transcoder
In plain wordsA stand-in for one part of an AI model that does the same job using a large set of simple, understandable pieces, so researchers can see what that part computes.
A sparse dictionary model trained to imitate a component of a network — usually an MLPmultilayer perceptron, n.: A neural network made of several layers of artificial neurons, each connected to every neuron in the next layer. It is the simplest kind of deep network and a component of almost every modern model. sublayer — by mapping the component's input to a wide, sparse hidden layer and decoding from it the component's output, rather than reconstructing its own input as a sparse autoencodersparse autoencoder, n.: A small add-on network trained to break an AI model's tangled internal signals into a much larger set of simpler pieces, most of which can be labeled with an understandable concept. does. Because the transcoder replaces a computation rather than merely re-describing a representation, the connections between its features in successive layers can be read from the weights, enabling input-independent circuitcircuit, n.: A small, identifiable piece of wiring inside a neural network — a few internal parts and the connections between them — that together carry out one specific job, such as spotting curves or copying a name. analysis. A skip transcoder adds an affine skip connection, lowering reconstruction error. A cross-layer transcoder (CLT) lets each feature read from one layer's residual streamresidual stream, n.: The running internal state that flows through a transformer model from layer to layer; each layer reads from it and adds its own contribution, like editors annotating a shared document. and write to the outputs of all later MLP layers, capturing computation spread across depth; Anthropic's 2025 circuit tracingcircuit tracing, n.: Methods that map, step by step, which internal concepts an AI model used to produce a particular answer and how they fed into one another, like tracing wires on a circuit board. work used CLTs to build the replacement modelreplacement model, n.: A more understandable copy of an AI model, built from simple labeled parts, that behaves almost like the original, so researchers can study the copy's workings instead. from which attribution graphsattribution graph, n.: A diagram showing which internal concepts led an AI model to a particular answer, with arrows showing how strongly each concept pushed the next. are computed. Transcoders trade some reconstruction accuracy for this structural readability.
Swap in the layer-8 transcoder and check that the model's loss on the task barely moves.
Origin
Named and first studied in detail by Jacob Dunefsky, Philippe Chlenski and Neel Nanda in 'Transcoders Find Interpretable LLM Feature Circuits' (June 2024; NeurIPS 2024), who credit earlier, unexplored proposals of the idea as 'predicting future activations' (Adly Templeton and colleagues at Anthropic) and 'MLP stretchers'. Gonçalo Paulo, Stepan Shabalin and Nora Belrose of EleutherAI proposed skip transcoders (January 2025); cross-layer transcoders were the basis of Anthropic's 'Circuit Tracing' (Ameisen et al., March 2025).
Related
- sparse autoencodersparse autoencoder, n.: A small add-on network trained to break an AI model's tangled internal signals into a much larger set of simpler pieces, most of which can be labeled with an understandable concept.
- crosscodercrosscoder, n.: A concept-finding tool that looks at several layers, or several versions of a model, at once, so that one shared concept can be tracked wherever it appears.
- circuit tracingcircuit tracing, n.: Methods that map, step by step, which internal concepts an AI model used to produce a particular answer and how they fed into one another, like tracing wires on a circuit board.
- replacement modelreplacement model, n.: A more understandable copy of an AI model, built from simple labeled parts, that behaves almost like the original, so researchers can study the copy's workings instead.
- attribution graphattribution graph, n.: A diagram showing which internal concepts led an AI model to a particular answer, with arrows showing how strongly each concept pushed the next.
- dictionary learningdictionary learning, n.: A method that learns a large vocabulary of basic patterns so that any piece of data can be described using just a few of them. In AI interpretability it builds a dictionary of a model's own concepts.
Sources
- Dunefsky, J., Chlenski, P. & Nanda, N. (2024). Transcoders Find Interpretable LLM Feature Circuits. NeurIPS 2024. arXiv:2406.11944. arxiv.org/abs/2406.11944
- Paulo, G., Shabalin, S. & Belrose, N. (2025). Transcoders Beat Sparse Autoencoders for Interpretability. arXiv:2501.18823. arxiv.org/abs/2501.18823
- Ameisen, E., Lindsey, J., Pearce, A. et al. (2025). Circuit Tracing: Revealing Computational Graphs in Language Models. Transformer Circuits Thread.
Mentioned in
feature (interpretability)feature (interpretability), n.: A concept or property that a neural network represents internally — such as 'is French', 'mentions a bridge' or 'this code has a bug' — usually corresponding to a direction in its internal activity., sparse autoencodersparse autoencoder, n.: A small add-on network trained to break an AI model's tangled internal signals into a much larger set of simpler pieces, most of which can be labeled with an understandable concept., attribution graphattribution graph, n.: A diagram showing which internal concepts led an AI model to a particular answer, with arrows showing how strongly each concept pushed the next., circuit tracingcircuit tracing, n.: Methods that map, step by step, which internal concepts an AI model used to produce a particular answer and how they fed into one another, like tracing wires on a circuit board., crosscodercrosscoder, n.: A concept-finding tool that looks at several layers, or several versions of a model, at once, so that one shared concept can be tracked wherever it appears., dictionary learningdictionary learning, n.: A method that learns a large vocabulary of basic patterns so that any piece of data can be described using just a few of them. In AI interpretability it builds a dictionary of a model's own concepts., feature steeringfeature steering, n.: Steering an AI model by turning up or down specific concepts that interpretability tools have found inside it — for example, making a model obsessed with a bridge by maxing out its 'bridge' concept., Gemma ScopeGemma Scope, prop. n.: A free collection of ready-made concept-finding tools released by Google DeepMind for its open Gemma language models, so outside researchers can study their insides without training the tools themselves., NeuronpediaNeuronpedia, prop. n.: A free website where anyone can browse, search and test the internal concepts that researchers have found inside AI models, with examples of the text that switches each one on., replacement modelreplacement model, n.: A more understandable copy of an AI model, built from simple labeled parts, that behaves almost like the original, so researchers can study the copy's workings instead., sparse feature circuitsparse feature circuit, n.: A map of how a language model performs a task, drawn in terms of understandable concepts found by concept-finding tools, rather than hard-to-read neurons or whole layers.