semantic search
In plain wordsSearch that understands what you mean, not just the words you typed, so a search for 'cheap places to stay' also finds 'budget hotels'.
- 1
Embedding-based search
Search that ranks results by how close their meaning is to the query's rather than by how many keywords they share, today usually by encoding both with an embedding modelembedding model, n.: An AI model whose only job is to turn text (or images, or code) into lists of numbers that capture meaning, so a computer can measure how similar two things are. and running vector searchvector search, n.: Searching by closeness instead of exact match: the query and all stored items are lists of numbers, and the search returns the items whose numbers are most similar to the query's. over text embeddingstext embedding, n.: A list of numbers that captures what a piece of text means, so that texts with similar meanings get similar lists. Computers use them to search, sort and group text by meaning. (dense retrievaldense retrieval, n.: Finding relevant documents by turning the question and every document into lists of numbers that capture meaning, then picking the documents whose numbers are closest to the question's.). It handles synonyms, paraphrases, misspellings and cross-language matches, and can search images or audio with the same machinery. Its weaknesses mirror its strengths: it can return items that are topically similar but wrong, miss exact identifiers, and is harder to explain than keyword matching, so production systems commonly combine it with keyword search (hybrid searchhybrid search, n.: Searching two ways at once — by exact keywords and by meaning — and merging the results, so you catch precise matches like product codes as well as differently worded matches.) and rerankingreranking, n.: A second, more careful pass over search results: a fast method grabs a few hundred candidates, then a slower, smarter model reorders them so the best come first.. Semantic search is the retrieval step in most retrieval-augmented generationretrieval-augmented generation, n.: Giving an AI an open-book exam: before answering, it looks up relevant passages in a set of documents and bases its reply on them, often citing where each point came from. systems.
Semantic search matched 'my card was charged twice' to the article titled 'Duplicate transactions'.
- 2
Semantic Web (earlier sense)
Search that exploits explicit, machine-readable meaning — ontologiesontology, n.: A carefully defined vocabulary for a subject such as medicine, spelling out each kind of thing, how kinds nest inside one another, and which connections between them are allowed., structured metadata and knowledge graphsknowledge graph, n.: A huge web of facts stored as connections between things — 'Paris is the capital of France', 'Marie Curie won a Nobel Prize' — that computers can search, combine and reason over. — to interpret queries and return entities and facts rather than documents, as envisioned for the semantic websemantic web, n.: An effort to label data on the web in a standard, precise way so that programs, not only human readers, can tell what it means and connect facts drawn from many different sites..
Origin
Latent semantic indexing (Scott Deerwester, Susan Dumais and colleagues, 1990) was an early statistical approach to matching by meaning. Ramanathan Guha, Rob McCool and Eric Miller's paper 'Semantic Search' (WWW 2003) used the phrase for Semantic Web search. The embedding sense became dominant after BERT-based models entered web search in 2019 and embedding APIs spread from 2022.
Usage note
Marketing applies 'semantic search' to almost any AI-assisted search; it is worth asking whether embeddings, knowledge graphs or simple query expansion are meant.
Related
- vector searchvector search, n.: Searching by closeness instead of exact match: the query and all stored items are lists of numbers, and the search returns the items whose numbers are most similar to the query's.
- dense retrievaldense retrieval, n.: Finding relevant documents by turning the question and every document into lists of numbers that capture meaning, then picking the documents whose numbers are closest to the question's.
- text embeddingtext embedding, n.: A list of numbers that captures what a piece of text means, so that texts with similar meanings get similar lists. Computers use them to search, sort and group text by meaning.
- hybrid searchhybrid search, n.: Searching two ways at once — by exact keywords and by meaning — and merging the results, so you catch precise matches like product codes as well as differently worded matches.
- latent semantic analysislatent semantic analysis, n.: An early technique for capturing word meaning by counting which words appear in which documents, then compressing those counts so that words used in similar contexts end up close together.
- knowledge graphknowledge graph, n.: A huge web of facts stored as connections between things — 'Paris is the capital of France', 'Marie Curie won a Nobel Prize' — that computers can search, combine and reason over.
- retrieval-augmented generationretrieval-augmented generation, n.: Giving an AI an open-book exam: before answering, it looks up relevant passages in a set of documents and bases its reply on them, often citing where each point came from.
- sparse retrievalsparse retrieval, n.: Finding documents by the words they contain, giving more weight to rare, telling words — the classic way search engines work.
Sources
- Guha, R., McCool, R. & Miller, E. (2003). Semantic Search. Proceedings of the 12th International World Wide Web Conference (WWW 2003).
- Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K. & Harshman, R. (1990). Indexing by latent semantic analysis. Journal of the American Society for Information Science 41(6): 391–407.
Mentioned in
embeddingembedding, n.: A list of numbers that stands for a word, picture or other item, arranged so that similar items get similar lists. It lets a computer measure similarity of meaning as distance., cosine similaritycosine similarity, n.: A score from −1 to 1 for how closely two lists of numbers point the same way, ignoring their size. It is the standard test of whether two texts mean similar things., generative engine optimizationgenerative engine optimization, n.: Shaping web content so that AI chatbots and AI search features are more likely to find it, quote it and recommend it — the AI-era version of search engine optimization., information retrievalinformation retrieval, n.: The study and engineering of search: how to organize huge piles of documents so that the few that matter to someone's question can be found and ranked quickly. AI systems rely on it to look things up., MTEBMTEB, prop. n.: A large public test suite that scores AI models which turn text into lists of numbers on many different jobs, such as searching, grouping, classifying and matching translations, so people can compare them on one leaderboard., query rewritingquery rewriting, n.: Rephrasing a user's question before searching — fixing typos, adding missing details, splitting it into simpler questions — so the search engine finds better results., semantic websemantic web, n.: An effort to label data on the web in a standard, precise way so that programs, not only human readers, can tell what it means and connect facts drawn from many different sites., Sentence-BERTSentence-BERT, prop. n.: A way of turning a whole sentence into a list of numbers so that sentences with similar meanings get similar numbers, which makes it fast to search millions of texts by meaning rather than by exact words., text embeddingtext embedding, n.: A list of numbers that captures what a piece of text means, so that texts with similar meanings get similar lists. Computers use them to search, sort and group text by meaning., vector databasevector database, n.: A database built to store embeddings — lists of numbers that represent meaning — and to quickly find the stored items most similar to a new one. It is the memory behind many AI search tools., vector searchvector search, n.: Searching by closeness instead of exact match: the query and all stored items are lists of numbers, and the search returns the items whose numbers are most similar to the query's., HyDEHyDE, prop. n.: A search trick: ask an AI to write a made-up answer to the question first, then search for real documents that resemble that imagined answer., semantic textual similaritysemantic textual similarity, n.: How close two sentences are in meaning, scored from 'about different things' to 'say exactly the same thing' — and the tests that check whether a computer's scores match people's judgments.