Skip to content
Last Dictionary
Plane
Interpretability & SteeringMathematics & StatisticsMachine Learning

dictionary learning

nounAlso: sparse dictionary learning, sparse coding

In plain wordsA method that learns a large vocabulary of basic patterns so that any piece of data can be described using just a few of them. In AI interpretability it builds a dictionary of a model's own concepts.

  1. 1

    Signal processing and machine learning

    The problem of learning an overcomplete set of basis vectors (atoms), collected as the columns of a dictionary matrix D, such that each data vector x is approximated as D·a with a sparse coefficient vector a. Methods alternate between sparse coding — finding a for fixed D, for example by matching pursuit or L1-penalized regression (lassolasso, n.: A version of linear regression that keeps only the most useful inputs: it penalizes the total size of the weights so that many become exactly zero, leaving a simpler model.) — and updating D, as in the method of optimal directions or K-SVD. Unlike principal component analysisprincipal component analysis, n.: A technique that finds the few directions in which data vary the most and describes each item by its position along them, compressing many measurements into a handful of summary scores., atoms need not be orthogonal and may outnumber the dimensions of the data. Applications include image denoising, inpainting and compression, and models of receptive fields in the visual cortex.

  2. 2

    Interpretability

    The use of the same idea to decompose a neural network's activationsactivation, n.: The numbers a neural network computes inside itself as it processes an input — a reading of which of its artificial neurons are firing, and how strongly, at that moment. into many sparsely active, interpretable featuresfeature (interpretability), n.: A concept or property that a neural network represents internally — such as 'is French', 'mentions a bridge' or 'this code has a bug' — usually corresponding to a direction in its internal activity., undoing superpositionsuperposition, n.: The way a neural network squeezes more concepts into its internal activity than it has neurons, by letting them overlap — a main reason its insides are hard to read.. The dictionary is usually learned by a sparse autoencodersparse autoencoder, n.: A small add-on network trained to break an AI model's tangled internal signals into a much larger set of simpler pieces, most of which can be labeled with an understandable concept. or a relative such as a transcodertranscoder, n.: A stand-in for one part of an AI model that does the same job using a large set of simple, understandable pieces, so researchers can see what that part computes. or crosscodercrosscoder, n.: A concept-finding tool that looks at several layers, or several versions of a model, at once, so that one shared concept can be tracked wherever it appears., whose decoder columns are the atoms; each atom is then labeled by inspecting the inputs that activate it, often with automated interpretabilityautomated interpretability, n.: Using AI models to help explain other AI models — for instance, having a language model read examples that excite an internal unit and write a description of what it detects.. The result is, quite literally, a dictionary of the model's own concepts — in Anthropic's 2024 study of Claude 3 Sonnet, some 34 million entries.

    Dictionary learning on the MLP layer turned 512 tangled neurons into 4,096 readable features.

min⁡D, {ai}∑i∥xi−Dai∥22+λ∥ai∥1\min_{D,\,\{a_i\}} \sum_i \lVert x_i - D a_i \rVert_2^2 + \lambda \lVert a_i \rVert_1

Origin

Grew from Stéphane Mallat and Zhifeng Zhang's matching pursuit over 'time-frequency dictionaries' (1993) and Bruno Olshausen and David Field's sparse-coding model of visual cortex (Nature, 1996); 'dictionary learning' was a standard name by the early 2000s. Applied to word embeddings by Arora and colleagues (2018) and to transformers by Zeyu Yun and colleagues (2021); Anthropic's 'Towards Monosemanticity' (2023) made it the usual name in interpretability.

Usage note

'Sparse coding' sometimes names the whole problem and sometimes only the step of finding coefficients for a fixed dictionary.

Related

See also
  • sparse autoencodersparse autoencoder, n.: A small add-on network trained to break an AI model's tangled internal signals into a much larger set of simpler pieces, most of which can be labeled with an understandable concept.
  • superpositionsuperposition, n.: The way a neural network squeezes more concepts into its internal activity than it has neurons, by letting them overlap — a main reason its insides are hard to read.
  • feature (interpretability)feature (interpretability), n.: A concept or property that a neural network represents internally — such as 'is French', 'mentions a bridge' or 'this code has a bug' — usually corresponding to a direction in its internal activity.
  • principal component analysisprincipal component analysis, n.: A technique that finds the few directions in which data vary the most and describes each item by its position along them, compressing many measurements into a handful of summary scores.
  • lassolasso, n.: A version of linear regression that keeps only the most useful inputs: it penalizes the total size of the weights so that many become exactly zero, leaving a simpler model.
  • monosemanticitymonosemanticity, n.: The property of an internal unit of an AI model that stands for just one understandable thing, which makes it far easier to interpret than a unit that mixes several.

Sources

  1. Olshausen, B. A. & Field, D. J. (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607–609.
  2. Mallat, S. & Zhang, Z. (1993). Matching Pursuits with Time-Frequency Dictionaries. IEEE Transactions on Signal Processing 41(12).
  3. Yun, Z., Chen, Y., Olshausen, B. A. & LeCun, Y. (2021). Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factors. DeeLIO Workshop, NAACL 2021.

Mentioned in

Golden Gate ClaudeGolden Gate Claude, prop. n.: A one-day public demo in 2024 in which Anthropic turned up a single internal concept inside its Claude chatbot — the Golden Gate Bridge — so it steered almost every conversation back to the bridge., polysemanticitypolysemanticity, n.: When a single artificial neuron responds to several unrelated things — say, bird beaks, bicycle wheels and a particular typeface — so that no single label describes what it does., sparse autoencodersparse autoencoder, n.: A small add-on network trained to break an AI model's tangled internal signals into a much larger set of simpler pieces, most of which can be labeled with an understandable concept., superpositionsuperposition, n.: The way a neural network squeezes more concepts into its internal activity than it has neurons, by letting them overlap — a main reason its insides are hard to read., The Last DictionaryThe Last Dictionary, prop. n.: This book: a dictionary of intelligence, human and otherwise, which explains each word first in plain language, then precisely, and says when it was born., crosscodercrosscoder, n.: A concept-finding tool that looks at several layers, or several versions of a model, at once, so that one shared concept can be tracked wherever it appears., enumerative safetyenumerative safety, n.: A proposed safety strategy: list every concept an AI model represents internally, then check the list for dangerous ones, such as deception, before trusting the model., feature absorptionfeature absorption, n.: A flaw in concept-finding tools where a general concept goes missing in some cases because a more specific concept swallows it — like a 'starts with S' detector staying silent on the word 'snake'., feature splittingfeature splitting, n.: What happens when a concept-finding tool is given more room: a broad concept it found before breaks into several narrower ones, much as 'animal' might split into 'dog', 'cat' and 'bird'., Gemma ScopeGemma Scope, prop. n.: A free collection of ready-made concept-finding tools released by Google DeepMind for its open Gemma language models, so outside researchers can study their insides without training the tools themselves., independent component analysisindependent component analysis, n.: A technique for unmixing signals that have been blended together, such as separating individual voices from several microphones recording a crowded room., monosemanticitymonosemanticity, n.: The property of an internal unit of an AI model that stands for just one understandable thing, which makes it far easier to interpret than a unit that mixes several., parameter decompositionparameter decomposition, n.: An approach that tries to understand a neural network by splitting its weights — rather than its internal activity — into many simple pieces, each responsible for one mechanism., replacement modelreplacement model, n.: A more understandable copy of an AI model, built from simple labeled parts, that behaves almost like the original, so researchers can study the copy's workings instead., transcodertranscoder, n.: A stand-in for one part of an AI model that does the same job using a large set of simple, understandable pieces, so researchers can see what that part computes., Transformer Circuits ThreadTransformer Circuits Thread, prop. n.: Anthropic's online publication for research on how language models work inside, presented as long interactive web articles rather than conventional papers.