concept erasure
In plain wordsRemoving a specific idea — such as a person's gender, or an artist's style — from what an AI model can represent or produce, so the model can no longer use or show it.
- 1
Representations
The removal of information about a target concept from a model's embeddingsembedding, n.: A list of numbers that stands for a word, picture or other item, arranged so that similar items get similar lists. It lets a computer measure similarity of meaning as distance. or activationsactivation, n.: The numbers a neural network computes inside itself as it processes an input — a reading of which of its artificial neurons are firing, and how strongly, at that moment., so that downstream components cannot use it, while disturbing other information as little as possible. Linear concept erasure requires that no linear classifier can predict the concept from the edited representation better than a constant guess. Methods include iterative nullspace projectioniterative nullspace projection, n.: An early method for removing a trait such as gender from a model's internal representations by repeatedly training a simple detector for it and then flattening away whatever the detector relied on., which repeatedly trains and projects out linear classifiers; adversarial min–max approaches such as R-LACE; and LEACELEACE, abbr.: A one-step mathematical method for scrubbing a concept, such as gender, from an AI model's internal data so that no simple detector can find it, while changing everything else as little as possible., a closed-form edit that provably achieves linear erasure with minimal change. Uses include removing protected attributesprotected attribute, n.: A personal trait, such as race, sex, religion, age or disability, that the law says must not be used to treat people unfairly. for fairness, testing whether a model actually relies on a concept (amnesic probing), and suppressing unwanted behaviors.
After concept erasure, a linear probe could no longer recover gender from the résumé embeddings.
- 2
Generative models
Editing a generative model, usually a text-to-image diffusion modeldiffusion model, n.: An AI that learns to create images, sound or video by practicing noise removal: shown pictures buried under static, it learns to clean them up, then builds new ones starting from pure static., so that it no longer produces a specified concept — an artist's style, a named person, nudity or a copyrighted character — even when asked to. Methods fine-tune the model so its predictions for the concept are steered toward a neutral one, or edit cross-attention weights in closed form. Erasure is often incomplete: adversarially chosen prompts or light fine-tuning can bring 'erased' concepts back.
Origin
Grew out of debiasing word embeddings, notably Tolga Bolukbasi and colleagues' projection of gender directions (2016). Shauli Ravfogel and colleagues framed linear erasure with INLP (2020) and gave the phrase early prominence in the title of 'Linear Adversarial Concept Erasure' (arXiv, January 2022; ICML 2022), though its first use has not been established; Nora Belrose and colleagues' LEACE (2023) gave a closed-form solution. Rohit Gandikota and colleagues carried the idea to diffusion models in 2023.
Usage note
Related to but distinct from machine unlearningmachine unlearning, n.: Erasing specific things a trained AI model has learned, such as dangerous lab techniques, a copyrighted novel or one person's records, far more cheaply than training the model all over again., which removes the influence of particular training data, and from directional ablationdirectional ablation, n.: Erasing one specific direction from an AI model's internal signals while it runs, so whatever that direction encodes — such as the tendency to refuse — can no longer influence what the model does., which removes one direction without guaranteeing that a concept is linearly unrecoverable. Erasure that defeats linear probes can still leave nonlinearly recoverable traces.
Related
- LEACELEACE, abbr.: A one-step mathematical method for scrubbing a concept, such as gender, from an AI model's internal data so that no simple detector can find it, while changing everything else as little as possible.
- iterative nullspace projectioniterative nullspace projection, n.: An early method for removing a trait such as gender from a model's internal representations by repeatedly training a simple detector for it and then flattening away whatever the detector relied on.
- directional ablationdirectional ablation, n.: Erasing one specific direction from an AI model's internal signals while it runs, so whatever that direction encodes — such as the tendency to refuse — can no longer influence what the model does.
- machine unlearningmachine unlearning, n.: Erasing specific things a trained AI model has learned, such as dangerous lab techniques, a copyrighted novel or one person's records, far more cheaply than training the model all over again.
- protected attributeprotected attribute, n.: A personal trait, such as race, sex, religion, age or disability, that the law says must not be used to treat people unfairly.
- diffusion modeldiffusion model, n.: An AI that learns to create images, sound or video by practicing noise removal: shown pictures buried under static, it learns to clean them up, then builds new ones starting from pure static.
- machine unlearningmachine unlearning, n.: Erasing specific things a trained AI model has learned, such as dangerous lab techniques, a copyrighted novel or one person's records, far more cheaply than training the model all over again.
Sources
- Ravfogel, S., Twiton, M., Goldberg, Y. & Cotterell, R. (2022). Linear Adversarial Concept Erasure. ICML 2022.
- Gandikota, R., Materzyńska, J., Fiotto-Kaufman, J. & Bau, D. (2023). Erasing Concepts from Diffusion Models. ICCV 2023.
Mentioned in
ablationablation, n.: Removing or switching off a part of an AI system to see what changes — whether an ingredient of an experiment's design or a single unit inside a trained network., linear probelinear probe, n.: The simplest kind of probe: a straight-line classifier trained on an AI model's internal activity to test whether some piece of information can be read off directly., probingprobing, n.: Testing what information is hidden inside an AI model by training a small, separate predictor on the model's internal activity: if the predictor can recover something, the model has stored it somewhere., directional ablationdirectional ablation, n.: Erasing one specific direction from an AI model's internal signals while it runs, so whatever that direction encodes — such as the tendency to refuse — can no longer influence what the model does., iterative nullspace projectioniterative nullspace projection, n.: An early method for removing a trait such as gender from a model's internal representations by repeatedly training a simple detector for it and then flattening away whatever the detector relied on., LEACELEACE, abbr.: A one-step mathematical method for scrubbing a concept, such as gender, from an AI model's internal data so that no simple detector can find it, while changing everything else as little as possible., linear representation hypothesislinear representation hypothesis, n.: The idea that a neural network stores many concepts as simple directions inside its internal activity, so how much of a concept is present can be read off by measuring along its direction., sparse feature circuitsparse feature circuit, n.: A map of how a language model performs a task, drawn in terms of understandable concepts found by concept-finding tools, rather than hard-to-read neurons or whole layers.