reranking
In plain wordsA second, more careful pass over search results: a fast method grabs a few hundred candidates, then a slower, smarter model reorders them so the best come first.
The second stage of a multi-stage retrieval pipeline, in which a more expensive model re-scores a short list of candidates (typically 20–1,000) produced by a fast first-stage retriever such as BM25BM25, prop. n.: A decades-old formula that search engines use to score how well a document matches a query, rewarding rare query words and repeated matches — with diminishing returns — and correcting for long documents., dense retrievaldense retrieval, n.: Finding relevant documents by turning the question and every document into lists of numbers that capture meaning, then picking the documents whose numbers are closest to the question's. or hybrid searchhybrid search, n.: Searching two ways at once — by exact keywords and by meaning — and merging the results, so you catch precise matches like product codes as well as differently worded matches., and reorders them. Neural rerankers are usually cross-encoderscross-encoder, n.: An AI model that reads a question and a candidate passage together, side by side, and outputs one relevance score. Accurate but slow, so it is used to reorder a short list. that read the query and each candidate together, capturing fine-grained relevance that separate encodings miss; alternatives include late interactionlate interaction, n.: A search design that keeps a separate number-list for every word of a document and compares them with the question's words at the last moment — more precise than one list per document, yet still fast. models, gradient-boosted learning to ranklearning to rank, n.: Training a model to put items in the best order — search results, products, recommendations — by learning from examples of which items people judged more relevant or chose more often. models, and LLMslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. prompted to score or sort passages pointwise, pairwise or listwise. In retrieval-augmented generationretrieval-augmented generation, n.: Giving an AI an open-book exam: before answering, it looks up relevant passages in a set of documents and bases its reply on them, often citing where each point came from., reranking lets a system retrieve broadly for recall and then pass only the few most relevant passages to the generator, cutting cost, distraction and lost in the middlelost in the middle, phr.: AI models reading a long input behave a little like a skimmer: they reliably catch facts placed near its start or end but are much likelier to miss the same fact buried halfway through. effects. Its main cost is latency, which grows with the number of candidates.
Recall@100 was fine; the problem was ordering, so we added a reranker and kept the top five.
Origin
Multi-stage ranking — a cheap filter followed by a costlier ranker — was standard in web search by the 2000s, with learned rankers such as gradient-boosted trees. Rodrigo Nogueira and Kyunghyun Cho's 'Passage Re-ranking with BERT' (January 2019), which improved the best MS MARCO MRR@10 by 27% relative, launched neural reranking with transformerstransformer, n.: The design behind today's chatbots and much of modern AI. It handles text by letting every word look at every other word and decide which of them matter for understanding it..
Related
- cross-encodercross-encoder, n.: An AI model that reads a question and a candidate passage together, side by side, and outputs one relevance score. Accurate but slow, so it is used to reorder a short list.
- learning to ranklearning to rank, n.: Training a model to put items in the best order — search results, products, recommendations — by learning from examples of which items people judged more relevant or chose more often.
- hybrid searchhybrid search, n.: Searching two ways at once — by exact keywords and by meaning — and merging the results, so you catch precise matches like product codes as well as differently worded matches.
- dense retrievaldense retrieval, n.: Finding relevant documents by turning the question and every document into lists of numbers that capture meaning, then picking the documents whose numbers are closest to the question's.
- retrieval-augmented generationretrieval-augmented generation, n.: Giving an AI an open-book exam: before answering, it looks up relevant passages in a set of documents and bases its reply on them, often citing where each point came from.
- ColBERTColBERT, prop. n.: A search model that compares a question and a document word by word — matching each question word to its best partner in the document — while still letting documents be processed ahead of time for speed.
- normalized discounted cumulative gainnormalized discounted cumulative gain, n.: A score for a ranked list of results that rewards putting the most relevant items near the top, scaled so that a perfect ordering scores 1.
Sources
- Nogueira, R. & Cho, K. (2019). Passage Re-ranking with BERT. arXiv:1901.04085. arxiv.org/abs/1901.04085
Mentioned in
retrieval-augmented generationretrieval-augmented generation, n.: Giving an AI an open-book exam: before answering, it looks up relevant passages in a set of documents and bases its reply on them, often citing where each point came from., cross-encodercross-encoder, n.: An AI model that reads a question and a candidate passage together, side by side, and outputs one relevance score. Accurate but slow, so it is used to reorder a short list., hybrid searchhybrid search, n.: Searching two ways at once — by exact keywords and by meaning — and merging the results, so you catch precise matches like product codes as well as differently worded matches., lost in the middlelost in the middle, phr.: AI models reading a long input behave a little like a skimmer: they reliably catch facts placed near its start or end but are much likelier to miss the same fact buried halfway through., mean reciprocal rankmean reciprocal rank, n.: A score for search: for each question, note how far down the first correct answer appears, take one over that position, and average across questions. First place scores 1, second place 0.5., MTEBMTEB, prop. n.: A large public test suite that scores AI models which turn text into lists of numbers on many different jobs, such as searching, grouping, classifying and matching translations, so people can compare them on one leaderboard., normalized discounted cumulative gainnormalized discounted cumulative gain, n.: A score for a ranked list of results that rewards putting the most relevant items near the top, scaled so that a perfect ordering scores 1., semantic searchsemantic search, n.: Search that understands what you mean, not just the words you typed, so a search for 'cheap places to stay' also finds 'budget hotels'., BEIRBEIR, prop. n.: A collection of 18 different search tests — on topics from medicine to finance to fact-checking — used to see whether a search model works on kinds of data it was never trained on., ColBERTColBERT, prop. n.: A search model that compares a question and a document word by word — matching each question word to its best partner in the document — while still letting documents be processed ahead of time for speed., contextual retrievalcontextual retrieval, n.: A fix for searching cut-up documents: before storing each piece, an AI writes a short note saying where it came from and what it is about, so the piece still makes sense on its own., late interactionlate interaction, n.: A search design that keeps a separate number-list for every word of a document and compares them with the question's words at the last moment — more precise than one list per document, yet still fast., learning to ranklearning to rank, n.: Training a model to put items in the best order — search results, products, recommendations — by learning from examples of which items people judged more relevant or chose more often., LlamaIndexLlamaIndex, prop. n.: A free software toolkit, and the company behind it, for connecting AI language models to an organization's own documents and data so the models can find and use that information., MS MARCOMS MARCO, prop. n.: A large Microsoft dataset of real Bing search questions paired with web passages that answer them, used to train and test most modern AI search models., reciprocal rank fusionreciprocal rank fusion, n.: A simple recipe for merging several ranked lists of search results: each result earns points according to how high it ranks in each list, and the totals decide the final order.