latent semantic analysis
In plain wordsAn early technique for capturing word meaning by counting which words appear in which documents, then compressing those counts so that words used in similar contexts end up close together.
A technique for information retrieval and text analysis that applies a truncated singular value decompositionsingular value decomposition, n.: A way of rewriting any grid of numbers as three simple moves in turn — turn, stretch along perpendicular axes, turn again — which reveals the directions that matter most in the data. to a term–document matrix, usually weighted by tf–idf, keeping the k largest singular values (commonly a few hundred). Terms and documents are thereby mapped into a shared low-dimensional 'semantic' space in which cosine similaritycosine similarity, n.: A score from −1 to 1 for how closely two lists of numbers point the same way, ignoring their size. It is the standard test of whether two texts mean similar things. reflects shared latent topics rather than literal word overlap, so a query about cars can retrieve a document that only says automobiles. Applied to search it is called latent semantic indexing (LSI). LSA partly handles synonymy but not polysemy, since each word receives a single vector. Its dense vectors anticipated word embeddingsword embedding, n.: A list of numbers standing for a word, learned so that words used in similar ways get similar lists. It lets a computer measure that 'cat' is closer to 'dog' than to 'car'. and dense retrievaldense retrieval, n.: Finding relevant documents by turning the question and every document into lists of numbers that capture meaning, then picking the documents whose numbers are closest to the question's., and probabilistic latent semantic analysis and latent Dirichlet allocationlatent Dirichlet allocation, n.: A popular topic-finding method that assumes every document is a blend of a few themes and every theme favors certain words, then works backward from the words to find both. recast it as a probabilistic topic modeltopic model, n.: A method that reads a large pile of documents and discovers the themes running through them, describing each theme by its typical words and each document by its mix of themes.. Its dimensions can take negative values and are hard to interpret, and adding documents requires recomputing or approximating the decomposition.
Latent semantic indexing let the 1990s search prototype match queries about physicians to documents about doctors.
Origin
Developed at Bellcore by Scott Deerwester, Susan Dumais, George Furnas, Richard Harshman, Thomas Landauer, Karen Lochbaum and Lynn Streeter, who filed a patent on it in 1988 (granted 1989). The landmark paper 'Indexing by Latent Semantic Analysis' by Deerwester, Dumais, Furnas, Landauer and Harshman appeared in the Journal of the American Society for Information Science in 1990; Landauer and Dumais later proposed LSA as a model of human vocabulary learning (1997).
Related
- singular value decompositionsingular value decomposition, n.: A way of rewriting any grid of numbers as three simple moves in turn — turn, stretch along perpendicular axes, turn again — which reveals the directions that matter most in the data.
- topic modeltopic model, n.: A method that reads a large pile of documents and discovers the themes running through them, describing each theme by its typical words and each document by its mix of themes.
- latent Dirichlet allocationlatent Dirichlet allocation, n.: A popular topic-finding method that assumes every document is a blend of a few themes and every theme favors certain words, then works backward from the words to find both.
- word embeddingword embedding, n.: A list of numbers standing for a word, learned so that words used in similar ways get similar lists. It lets a computer measure that 'cat' is closer to 'dog' than to 'car'.
- dense retrievaldense retrieval, n.: Finding relevant documents by turning the question and every document into lists of numbers that capture meaning, then picking the documents whose numbers are closest to the question's.
- information retrievalinformation retrieval, n.: The study and engineering of search: how to organize huge piles of documents so that the few that matter to someone's question can be found and ranked quickly. AI systems rely on it to look things up.
- dimensionality reductiondimensionality reduction, n.: Squeezing data that has many measurements per item into far fewer numbers while keeping the patterns that matter, so it is easier to visualize, store or learn from.
Sources
- Deerwester, S., Dumais, S. T., Furnas, G. W., Landauer, T. K. & Harshman, R. (1990). Indexing by Latent Semantic Analysis. Journal of the American Society for Information Science 41(6).
- Landauer, T. K. & Dumais, S. T. (1997). A Solution to Plato's Problem: The Latent Semantic Analysis Theory of Acquisition, Induction, and Representation of Knowledge. Psychological Review 104(2).
Mentioned in
latent Dirichlet allocationlatent Dirichlet allocation, n.: A popular topic-finding method that assumes every document is a blend of a few themes and every theme favors certain words, then works backward from the words to find both., low-rank approximationlow-rank approximation, n.: Replacing a big grid of numbers with a slimmer one that keeps most of the same information, the way a few pencil strokes can capture a face. It saves memory and computation., principal component analysisprincipal component analysis, n.: A technique that finds the few directions in which data vary the most and describes each item by its position along them, compressing many measurements into a handful of summary scores., semantic searchsemantic search, n.: Search that understands what you mean, not just the words you typed, so a search for 'cheap places to stay' also finds 'budget hotels'., singular value decompositionsingular value decomposition, n.: A way of rewriting any grid of numbers as three simple moves in turn — turn, stretch along perpendicular axes, turn again — which reveals the directions that matter most in the data., topic modeltopic model, n.: A method that reads a large pile of documents and discovers the themes running through them, describing each theme by its typical words and each document by its mix of themes., word embeddingword embedding, n.: A list of numbers standing for a word, learned so that words used in similar ways get similar lists. It lets a computer measure that 'cat' is closer to 'dog' than to 'car'., distributional hypothesisdistributional hypothesis, n.: The idea that words found in similar company — surrounded by similar words — tend to be similar in meaning, the insight that lets computers learn something about word meaning from large amounts of text alone., GloVeGloVe, prop. n.: A 2014 Stanford method for learning number lists that represent words, built from counts of how often each pair of words appears near each other across a very large body of text., vector space modelvector space model, n.: An early, still-influential idea for search: treat every document and query as an arrow in a space with one direction per word, and rank documents by how closely their arrows point the same way as the query's.