control vector
In plain wordsA steering signal saved as a small file that can be loaded alongside an open AI model to turn a trait — say, cheerfulness or laziness — up or down while it generates text.
A set of steering vectorssteering vector, n.: A direction inside an AI model's internal activity that, when added while the model runs, nudges its behavior — say, more upbeat, more honest or fixated on a topic — without retraining it., usually one for each of many layers, stored as a reusable artifact and applied with an adjustable signed strength to shift a large language modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code.'s style or disposition at inference time. Control vectors are typically trained with the representation-reading recipe of representation engineeringrepresentation engineering, n.: An approach to understanding and controlling AI models by finding where high-level ideas such as honesty or danger show up in their internal activity, then reading or adjusting those signals directly.: generate prompt pairs that differ only in a persona instruction ('act extremely happy' versus 'act extremely sad'), collect hidden states, and take the first principal component of their differences at each layer. Positive strengths push toward one pole and negative strengths toward the other. Loading them became a feature of local inference software, letting users of open-weight modelsopen-weight model, n.: An AI model whose trained internal numbers are published for anyone to download and run on their own computers, even if the data and code used to make it stay private. adjust traits such as honesty, creativity or verbosity without fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way..
Load the happiness control vector at strength 1.5, or at −1.5 for a gloomy narrator.
Origin
The technique comes from representation engineeringrepresentation engineering, n.: An approach to understanding and controlling AI models by finding where high-level ideas such as honesty or danger show up in their internal activity, then reading or adjusting those signals directly. (Andy Zou and colleagues, 2023), whose representation-control methods compute such vectors. The name was popularized by Theia Vogel's blog post 'Representation Engineering Mistral-7B an Acid Trip' (January 22, 2024) and her open-source repeng library; in March 2024 Vogel also added support for loading control vectors to llama.cppllama.cpp, prop. n.: A free, open-source program that runs AI language models on ordinary computers and phones, without needing a powerful graphics card, by using compact low-precision versions of the models..
Usage note
In substance a control vector and a steering vector are the same kind of object; 'control vector' usually implies the multi-layer, PCA-trained, file-based form used in open-source tooling.
Related
- steering vectorsteering vector, n.: A direction inside an AI model's internal activity that, when added while the model runs, nudges its behavior — say, more upbeat, more honest or fixated on a topic — without retraining it.
- representation engineeringrepresentation engineering, n.: An approach to understanding and controlling AI models by finding where high-level ideas such as honesty or danger show up in their internal activity, then reading or adjusting those signals directly.
- activation steeringactivation steering, n.: Changing an AI model's behavior by adjusting the internal numbers it computes while it runs — like turning a hidden dial — instead of rewording the prompt or retraining the model.
- llama.cppllama.cpp, prop. n.: A free, open-source program that runs AI language models on ordinary computers and phones, without needing a powerful graphics card, by using compact low-precision versions of the models.
- persona vectorpersona vector, n.: A direction inside a language model's internal activity linked to a character trait, such as being flattering, malicious or prone to making things up, used to watch for or control that trait.
Sources
- Vogel, T. (2024). Representation Engineering Mistral-7B an Acid Trip. Blog post, January 22, 2024. vgel.me/posts/representation-engineering
Mentioned in
representation engineeringrepresentation engineering, n.: An approach to understanding and controlling AI models by finding where high-level ideas such as honesty or danger show up in their internal activity, then reading or adjusting those signals directly.