recall@k
In plain wordsA score for search systems: of all the right answers that exist, what fraction showed up in the top k results? Higher means fewer relevant items were missed.
- 1
Retrieval evaluation
The fraction of all relevant items for a query that appear among the top k retrieved results, averaged over a set of queries. It is the recallrecall, n.: Of all the items that really were positive, the share the model managed to find; a cancer screen with high recall misses very few people who actually have cancer. of a ranked list cut off at k and ignores the order within the top k. Recall@k is the key metric for first-stage retrievers and for the retrieval step of retrieval-augmented generationretrieval-augmented generation, n.: Giving an AI an open-book exam: before answering, it looks up relevant passages in a set of documents and bases its reply on them, often citing where each point came from., since a passage that is never retrieved cannot be used, whatever happens later; ordering quality is then measured with precisionprecision, n.: Of all the items a model flagged as positive, the share that really were positive; a spam filter with high precision rarely sends genuine emails to the spam folder.@k, mean reciprocal rankmean reciprocal rank, n.: A score for search: for each question, note how far down the first correct answer appears, take one over that position, and average across questions. First place scores 1, second place 0.5. or nDCGnormalized discounted cumulative gain, n.: A score for a ranked list of results that rewards putting the most relevant items near the top, scaled so that a perfect ordering scores 1.. A common simplification, hit rate@k (or success@k), records only whether at least one relevant item appears in the top k, and papers do not always say which of the two they report.
Recall@20 is 0.94, so retrieval isn't the bottleneck; the model is ignoring good passages.
- 2
Vector search
For approximate nearest neighbor searchapproximate nearest neighbor search, n.: A family of shortcuts for finding the most similar items in a huge collection without checking every one — accepting an occasional miss in exchange for answers thousands of times faster., the fraction of the true k nearest neighbors (as found by exact search) that the approximate index returns. It measures the index's fidelity to exact search, not relevance to users.
Origin
Recall and precision were adopted as retrieval measures in the 1950s and 1960s, notably in Cyril Cleverdon's Cranfield experiments; cutoff-based versions became standard with ranked retrieval and NIST's TREC evaluations from the 1990s.
Related
- recallrecall, n.: Of all the items that really were positive, the share the model managed to find; a cancer screen with high recall misses very few people who actually have cancer.
- precisionprecision, n.: Of all the items a model flagged as positive, the share that really were positive; a spam filter with high precision rarely sends genuine emails to the spam folder.
- mean reciprocal rankmean reciprocal rank, n.: A score for search: for each question, note how far down the first correct answer appears, take one over that position, and average across questions. First place scores 1, second place 0.5.
- normalized discounted cumulative gainnormalized discounted cumulative gain, n.: A score for a ranked list of results that rewards putting the most relevant items near the top, scaled so that a perfect ordering scores 1.
- approximate nearest neighbor searchapproximate nearest neighbor search, n.: A family of shortcuts for finding the most similar items in a huge collection without checking every one — accepting an occasional miss in exchange for answers thousands of times faster.
- information retrievalinformation retrieval, n.: The study and engineering of search: how to organize huge piles of documents so that the few that matter to someone's question can be found and ranked quickly. AI systems rely on it to look things up.
- evaluationevaluation, n.: Testing an AI system to find out what it can and cannot do, how well, and how safely — from exam-style quizzes to hands-on trials and expert reviews.
Mentioned in
recommender systemrecommender system, n.: Software that guesses which songs, products, posts or jobs you are most likely to want and shows those first, as when a video app lines up the next clip or a shop suggests an add-on., approximate nearest neighbor searchapproximate nearest neighbor search, n.: A family of shortcuts for finding the most similar items in a huge collection without checking every one — accepting an occasional miss in exchange for answers thousands of times faster., FAISSFAISS, prop. n.: A free software library from Meta for searching huge collections of embeddings very fast, on ordinary processors or graphics cards. Many AI search systems use it under the hood., HNSWHNSW, abbr.: A popular way to organize millions of embeddings for fast similarity search: a layered web of links where the search starts with long jumps on sparse top layers, then takes ever shorter steps., mean reciprocal rankmean reciprocal rank, n.: A score for search: for each question, note how far down the first correct answer appears, take one over that position, and average across questions. First place scores 1, second place 0.5., normalized discounted cumulative gainnormalized discounted cumulative gain, n.: A score for a ranked list of results that rewards putting the most relevant items near the top, scaled so that a perfect ordering scores 1., query rewritingquery rewriting, n.: Rephrasing a user's question before searching — fixing typos, adding missing details, splitting it into simpler questions — so the search engine finds better results., recallrecall, n.: Of all the items that really were positive, the share the model managed to find; a cancer screen with high recall misses very few people who actually have cancer., vector searchvector search, n.: Searching by closeness instead of exact match: the query and all stored items are lists of numbers, and the search returns the items whose numbers are most similar to the query's.