Skip to content
Last Dictionary
Plane
AI Engineering & MLOpsMathematics & StatisticsMachine Learning

BM25

proper nounAlso: Okapi BM25, Best Matching 25, BM25F

In plain wordsA decades-old formula that search engines use to score how well a document matches a query, rewarding rare query words and repeated matches — with diminishing returns — and correcting for long documents.

  1. A formula for scoring documents against a keyword query, derived within the probabilistic relevance framework: a document's score sums, over the query terms it contains, an inverse document frequency weight multiplied by a saturating function of the term's frequency in the document, normalized for document length. The parameter k₁ sets how quickly repeated occurrences stop adding score (commonly 1.2–2), and b sets the strength of length normalization (commonly about 0.75). Needing no training, fast with an inverted indexinverted index, n.: A lookup table that lists, for every word, all the documents containing it — like the index at the back of a book — so a search engine can find matches almost instantly., and hard to beat on unfamiliar domains, BM25 is the default scorer in engines such as Lucene, Elasticsearch and OpenSearch, the standard baseline in information retrievalinformation retrieval, n.: The study and engineering of search: how to organize huge piles of documents so that the few that matter to someone's question can be found and ranked quickly. AI systems rely on it to look things up. research, and the usual sparsesparse retrieval, n.: Finding documents by the words they contain, giving more weight to rare, telling words — the classic way search engines work. half of hybrid searchhybrid search, n.: Searching two ways at once — by exact keywords and by meaning — and merging the results, so you catch precise matches like product codes as well as differently worded matches.. BM25F extends it to documents with separately weighted fields such as title and body.

    Before you ship that embedding model, check that it actually beats BM25 on our queries.

score⁡(D,Q)=∑t∈QIDF⁡(t) f(t,D) (k1+1)f(t,D)+k1(1−b+b ∣D∣avgdl)\operatorname{score}(D,Q)=\sum_{t\in Q}\operatorname{IDF}(t)\,\frac{f(t,D)\,(k_1+1)}{f(t,D)+k_1\left(1-b+b\,\frac{|D|}{\mathrm{avgdl}}\right)}

Origin

Developed by Stephen Robertson, Steve Walker and colleagues at City University, London, for the Okapi retrieval system, building on Robertson and Karen Spärck Jones's probabilistic relevance weighting (1976) and Stephen Harter's 2-Poisson model of term frequency (1975). BM25 emerged in 1994, in Robertson and Walker's SIGIR paper approximating the 2-Poisson model and in Okapi's runs at TREC-3. 'BM' stands for best matching; the number distinguishes it from sibling formulas such as BM11 and BM15.

Usage note

In the formula, f(t,D) is the count of term t in document D, |D| the document's length and avgdl the average length in the collection. Implementations use slightly different IDF variants, so BM25 scores differ between engines.

Related

See also
  • TF-IDFTF-IDF, abbr.: A simple score for picking out the words that characterize a document: a word counts for more the more often it appears there, and for less the more of the collection's other documents also use it.
  • sparse retrievalsparse retrieval, n.: Finding documents by the words they contain, giving more weight to rare, telling words — the classic way search engines work.
  • inverted indexinverted index, n.: A lookup table that lists, for every word, all the documents containing it — like the index at the back of a book — so a search engine can find matches almost instantly.
  • information retrievalinformation retrieval, n.: The study and engineering of search: how to organize huge piles of documents so that the few that matter to someone's question can be found and ranked quickly. AI systems rely on it to look things up.
  • hybrid searchhybrid search, n.: Searching two ways at once — by exact keywords and by meaning — and merging the results, so you catch precise matches like product codes as well as differently worded matches.
  • dense retrievaldense retrieval, n.: Finding relevant documents by turning the question and every document into lists of numbers that capture meaning, then picking the documents whose numbers are closest to the question's.

Sources

  1. Robertson, S. E., Walker, S., Jones, S., Hancock-Beaulieu, M. & Gatford, M. (1994). Okapi at TREC-3. Proceedings of the Third Text REtrieval Conference (TREC-3), NIST. ir.webis.de/anthology/1994.trec_conference-1994.11
  2. Robertson, S. & Zaragoza, H. (2009). The Probabilistic Relevance Framework: BM25 and Beyond. Foundations and Trends in Information Retrieval 3(4). staff.city.ac.uk/~sbrp622/papers/foundations_bm25_review.pdf

Mentioned in

baselinebaseline, n.: A simple reference method that a new model must beat to show it is worth anything, such as always guessing the most common answer or reusing yesterday's value., cross-encodercross-encoder, n.: An AI model that reads a question and a candidate passage together, side by side, and outputs one relevance score. Accurate but slow, so it is used to reorder a short list., dense retrievaldense retrieval, n.: Finding relevant documents by turning the question and every document into lists of numbers that capture meaning, then picking the documents whose numbers are closest to the question's., hybrid searchhybrid search, n.: Searching two ways at once — by exact keywords and by meaning — and merging the results, so you catch precise matches like product codes as well as differently worded matches., information retrievalinformation retrieval, n.: The study and engineering of search: how to organize huge piles of documents so that the few that matter to someone's question can be found and ranked quickly. AI systems rely on it to look things up., inverted indexinverted index, n.: A lookup table that lists, for every word, all the documents containing it — like the index at the back of a book — so a search engine can find matches almost instantly., rerankingreranking, n.: A second, more careful pass over search results: a fast method grabs a few hundred candidates, then a slower, smarter model reorders them so the best come first., sparse retrievalsparse retrieval, n.: Finding documents by the words they contain, giving more weight to rare, telling words — the classic way search engines work., TF-IDFTF-IDF, abbr.: A simple score for picking out the words that characterize a document: a word counts for more the more often it appears there, and for less the more of the collection's other documents also use it., TRECTREC, prop. n.: A yearly contest, run by a US government standards agency since 1992, in which search-engine researchers test their systems on the same documents and questions, with people judging which results are actually relevant., BEIRBEIR, prop. n.: A collection of 18 different search tests — on topics from medicine to finance to fact-checking — used to see whether a search model works on kinds of data it was never trained on., contextual retrievalcontextual retrieval, n.: A fix for searching cut-up documents: before storing each piece, an AI writes a short note saying where it came from and what it is about, so the piece still makes sense on its own., learning to ranklearning to rank, n.: Training a model to put items in the best order — search results, products, recommendations — by learning from examples of which items people judged more relevant or chose more often., reciprocal rank fusionreciprocal rank fusion, n.: A simple recipe for merging several ranked lists of search results: each result earns points according to how high it ranks in each list, and the totals decide the final order.