topic model
In plain wordsA method that reads a large pile of documents and discovers the themes running through them, describing each theme by its typical words and each document by its mix of themes.
A statistical model that uncovers recurring themes (topics) in an unlabeled collection of documents from patterns of word use alone. In the classic probabilistic formulation, each topic is a probability distributionprobability distribution, n.: A complete description of how likely each possible outcome is — for a fair die, one-sixth for each face. A language model produces one of these over its whole vocabulary at every step. over the vocabulary and each document is a mixture of topics; fitting the model to word counts might reveal a topic whose most probable words are 'gene', 'DNA' and 'sequencing', and documents that are 60% about it. The main models are latent semantic analysislatent semantic analysis, n.: An early technique for capturing word meaning by counting which words appear in which documents, then compressing those counts so that words used in similar contexts end up close together. (a linear-algebra precursor), probabilistic latent semantic analysis, latent Dirichlet allocationlatent Dirichlet allocation, n.: A popular topic-finding method that assumes every document is a blend of a few themes and every theme favors certain words, then works backward from the words to find both. and its extensions (correlated, dynamic, supervised and hierarchical topic models), and non-negative matrix factorizationmatrix factorization, n.: Breaking a big table of numbers, such as which users liked which films, into two smaller tables of hidden traits, so missing entries can be filled in by multiplying them back together.. Since the late 2010s, neural topic models and embedding-based pipelines, which cluster sentence embeddingsembedding, n.: A list of numbers that stands for a word, picture or other item, arranged so that similar items get similar lists. It lets a computer measure similarity of meaning as distance. and label each cluster with keywords or a large language modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code., have become common. Topic models are used to explore archives and in social science, the digital humanities and search; topics are judged by statistical coherence and human interpretability.
The topic model split ten years of parliamentary speeches into 40 themes, and the energy-prices topic spiked in 2022.
Origin
An early probabilistic model of documents as mixtures of topics appeared in a 1998 analysis of latent semantic indexing by Christos Papadimitriou, Prabhakar Raghavan, Hisao Tamaki and Santosh Vempala, building on latent semantic analysis (1988–1990). Thomas Hofmann introduced probabilistic latent semantic analysis in 1999, and the latent Dirichlet allocation of David Blei, Andrew Ng and Michael Jordan (2001–2003) became the field's standard model.
Usage note
Topics are statistical word groupings, not guaranteed concepts: results depend on preprocessing, the number of topics and random initialization, and some topics will be incoherent.
Related
- latent Dirichlet allocationlatent Dirichlet allocation, n.: A popular topic-finding method that assumes every document is a blend of a few themes and every theme favors certain words, then works backward from the words to find both.
- latent semantic analysislatent semantic analysis, n.: An early technique for capturing word meaning by counting which words appear in which documents, then compressing those counts so that words used in similar contexts end up close together.
- clusteringclustering, n.: Automatically sorting things into groups of similar items when nobody has said in advance what the groups should be, like sorting a drawer of odd socks into piles by color and size.
- dimensionality reductiondimensionality reduction, n.: Squeezing data that has many measurements per item into far fewer numbers while keeping the patterns that matter, so it is easier to visualize, store or learn from.
- natural language processingnatural language processing, n.: The field that teaches computers to handle human language — to read, interpret, translate, summarize and produce text and speech. It is one of the oldest and largest areas of artificial intelligence.
- text classificationtext classification, n.: Sorting pieces of text into categories automatically — what topic an article covers, whether a message is spam, which team should handle a customer's complaint.
Sources
- Blei, D. M. (2012). Probabilistic Topic Models. Communications of the ACM 55(4).
- Hofmann, T. (1999). Probabilistic Latent Semantic Analysis. Proc. 15th Conference on Uncertainty in Artificial Intelligence (UAI).
Mentioned in
bag of wordsbag of words, n.: A simple way of turning text into numbers by counting how often each word appears and ignoring word order — like describing a recipe only by its shopping list., latent Dirichlet allocationlatent Dirichlet allocation, n.: A popular topic-finding method that assumes every document is a blend of a few themes and every theme favors certain words, then works backward from the words to find both., latent semantic analysislatent semantic analysis, n.: An early technique for capturing word meaning by counting which words appear in which documents, then compressing those counts so that words used in similar contexts end up close together., stemmingstemming, n.: Cutting words down to a common root so that variants count as the same word — 'connected', 'connecting' and 'connection' all become 'connect' — to help search and text analysis.