reciprocal rank fusion
In plain wordsA simple recipe for merging several ranked lists of search results: each result earns points according to how high it ranks in each list, and the totals decide the final order.
A rank-aggregation method that scores each document by summing, over the input rankings, the reciprocal of a constant k plus the document's rank in that ranking, then sorts documents by the total. Because it uses only ranks, it needs no normalization of incompatible scores (such as BM25BM25, prop. n.: A decades-old formula that search engines use to score how well a document matches a query, rewarding rare query words and repeated matches — with diminishing returns — and correcting for long documents. values and cosine similaritiescosine similarity, n.: A score from −1 to 1 for how closely two lists of numbers point the same way, ignoring their size. It is the standard test of whether two texts mean similar things.) and has essentially one parameter. It rewards documents that appear near the top of several lists while keeping lower ranks from counting for nothing; k damps the influence of any single system's top picks. It is the most common fusion step in hybrid searchhybrid search, n.: Searching two ways at once — by exact keywords and by meaning — and merging the results, so you catch precise matches like product codes as well as differently worded matches. and is built into several search engines and vector databasesvector database, n.: A database built to store embeddings — lists of numbers that represent meaning — and to quickly find the stored items most similar to a new one. It is the memory behind many AI search tools.; weighted variants multiply each list's contribution by a tunable weight.
We merge the BM25 and embedding lists with reciprocal rank fusion, k = 60, then rerank the top 50.
Origin
Introduced by Gordon Cormack, Charles Clarke and Stefan Büttcher of the University of Waterloo in a SIGIR 2009 short paper, which found it outperformed Condorcet fusion and individual learning-to-rank methods. They fixed k = 60 in a pilot study, reporting the value near-optimal but not critical, and it remains the usual default.
Usage note
Not to be confused with mean reciprocal rankmean reciprocal rank, n.: A score for search: for each question, note how far down the first correct answer appears, take one over that position, and average across questions. First place scores 1, second place 0.5., an evaluation metric that also uses reciprocal ranks but measures a single ranking rather than combining several.
Related
- hybrid searchhybrid search, n.: Searching two ways at once — by exact keywords and by meaning — and merging the results, so you catch precise matches like product codes as well as differently worded matches.
- rerankingreranking, n.: A second, more careful pass over search results: a fast method grabs a few hundred candidates, then a slower, smarter model reorders them so the best come first.
- learning to ranklearning to rank, n.: Training a model to put items in the best order — search results, products, recommendations — by learning from examples of which items people judged more relevant or chose more often.
- BM25BM25, prop. n.: A decades-old formula that search engines use to score how well a document matches a query, rewarding rare query words and repeated matches — with diminishing returns — and correcting for long documents.
- dense retrievaldense retrieval, n.: Finding relevant documents by turning the question and every document into lists of numbers that capture meaning, then picking the documents whose numbers are closest to the question's.
- mean reciprocal rankmean reciprocal rank, n.: A score for search: for each question, note how far down the first correct answer appears, take one over that position, and average across questions. First place scores 1, second place 0.5.
- mean reciprocal rankmean reciprocal rank, n.: A score for search: for each question, note how far down the first correct answer appears, take one over that position, and average across questions. First place scores 1, second place 0.5.
Sources
- Cormack, G. V., Clarke, C. L. A. & Büttcher, S. (2009). Reciprocal rank fusion outperforms Condorcet and individual rank learning methods. SIGIR 2009: 758–759. plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf
Mentioned in
hybrid searchhybrid search, n.: Searching two ways at once — by exact keywords and by meaning — and merging the results, so you catch precise matches like product codes as well as differently worded matches., mean reciprocal rankmean reciprocal rank, n.: A score for search: for each question, note how far down the first correct answer appears, take one over that position, and average across questions. First place scores 1, second place 0.5., query rewritingquery rewriting, n.: Rephrasing a user's question before searching — fixing typos, adding missing details, splitting it into simpler questions — so the search engine finds better results.