Neuronpedia
In plain wordsA free website where anyone can browse, search and test the internal concepts that researchers have found inside AI models, with examples of the text that switches each one on.
An open platform for mechanistic interpretability research that hosts sparse autoencodersparse autoencoder, n.: A small add-on network trained to break an AI model's tangled internal signals into a much larger set of simpler pieces, most of which can be labeled with an understandable concept., transcodertranscoder, n.: A stand-in for one part of an AI model that does the same job using a large set of simple, understandable pieces, so researchers can see what that part computes. and other dictionary features for many open-weight models. For each feature it shows a dashboard of top-activating text examples, activation histograms, the output tokens the feature promotes and suppresses, and natural-language explanations generated by automated interpretabilityautomated interpretability, n.: Using AI models to help explain other AI models — for instance, having a language model read examples that excite an internal unit and write a description of what it detects.. Users can search features by meaning, run their own text through hosted models, steerfeature steering, n.: Steering an AI model by turning up or down specific concepts that interpretability tools have found inside it — for example, making a model obsessed with a bridge by maxing out its 'bridge' concept. outputs with chosen features, and build and share lists. Since 2025 it has also hosted interactive attribution graphsattribution graph, n.: A diagram showing which internal concepts led an AI model to a particular answer, with arrows showing how strongly each concept pushed the next. generated with open-source circuit tracingcircuit tracing, n.: Methods that map, step by step, which internal concepts an AI model used to produce a particular answer and how they fed into one another, like tracing wires on a circuit board. tools. Large public releases such as Google DeepMind's Gemma ScopeGemma Scope, prop. n.: A free collection of ready-made concept-finding tools released by Google DeepMind for its open Gemma language models, so outside researchers can study their insides without training the tools themselves. have been made explorable through it, and its own code has been released as open source.
Search Neuronpedia for 'deception' and check whether the top feature really fires on lies.
Origin
Created by Johnny Lin, who launched it in July 2023 as a game for crowdsourcing explanations of individual neurons in GPT-2; on 25 March 2024 Lin and Joseph Bloom announced its relaunch as a platform for sparse-autoencoder research, then hosting over 600,000 feature dashboards. Its code was open-sourced in 2025.
Related
- sparse autoencodersparse autoencoder, n.: A small add-on network trained to break an AI model's tangled internal signals into a much larger set of simpler pieces, most of which can be labeled with an understandable concept.
- automated interpretabilityautomated interpretability, n.: Using AI models to help explain other AI models — for instance, having a language model read examples that excite an internal unit and write a description of what it detects.
- feature (interpretability)feature (interpretability), n.: A concept or property that a neural network represents internally — such as 'is French', 'mentions a bridge' or 'this code has a bug' — usually corresponding to a direction in its internal activity.
- circuit tracingcircuit tracing, n.: Methods that map, step by step, which internal concepts an AI model used to produce a particular answer and how they fed into one another, like tracing wires on a circuit board.
- feature steeringfeature steering, n.: Steering an AI model by turning up or down specific concepts that interpretability tools have found inside it — for example, making a model obsessed with a bridge by maxing out its 'bridge' concept.
- Gemma ScopeGemma Scope, prop. n.: A free collection of ready-made concept-finding tools released by Google DeepMind for its open Gemma language models, so outside researchers can study their insides without training the tools themselves.
Sources
- Lin, J. & Bloom, J. (2024). Announcing Neuronpedia: Platform for accelerating research into Sparse Autoencoders. LessWrong / AI Alignment Forum. lesswrong.com/posts/BaEQoxHhWPrkinmxd/announcing-neuronpedia-as-a-platform-to-accelerate-research
Mentioned in
sparse autoencodersparse autoencoder, n.: A small add-on network trained to break an AI model's tangled internal signals into a much larger set of simpler pieces, most of which can be labeled with an understandable concept., automated interpretabilityautomated interpretability, n.: Using AI models to help explain other AI models — for instance, having a language model read examples that excite an internal unit and write a description of what it detects., circuit tracingcircuit tracing, n.: Methods that map, step by step, which internal concepts an AI model used to produce a particular answer and how they fed into one another, like tracing wires on a circuit board., Gemma ScopeGemma Scope, prop. n.: A free collection of ready-made concept-finding tools released by Google DeepMind for its open Gemma language models, so outside researchers can study their insides without training the tools themselves., GoodfireGoodfire, prop. n.: A San Francisco company, founded in 2024, that builds tools to look inside AI models, understand what they have learned and adjust how they behave., TransformerLensTransformerLens, prop. n.: A free software library that lets researchers open up language models and record, inspect or alter what happens inside them at every step while the model runs.