persona vector
In plain wordsA direction inside a language model's internal activity linked to a character trait, such as being flattering, malicious or prone to making things up, used to watch for or control that trait.
A steering vectorsteering vector, n.: A direction inside an AI model's internal activity that, when added while the model runs, nudges its behavior — say, more upbeat, more honest or fixated on a topic — without retraining it. that corresponds to a character trait of a large language modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code.'s assistant persona and can be derived automatically from nothing more than a written description of the trait. In the introducing method, a frontier model turns the description (for example 'sycophanticsycophancy, n.: When an AI tells people what they want to hear — flattering them, agreeing with their mistakes or dropping a correct answer under pushback — instead of what is true or useful.', 'prone to hallucinationhallucination, n.: When an AI states something false or made up — a fake fact, quotation or source — fluently and confidently, as if it were true.' or 'evil') into contrasting system prompts, evaluation questions and a scoring rubric; the vector is the difference in mean residual streamresidual stream, n.: The running internal state that flows through a transformer model from layer to layer; each layer reads from it and adds its own contribution, like editors annotating a shared document. activations between responses that do and do not express the trait. Uses include monitoring personality shifts in deployment (projection onto the vector predicts trait expression), steering at inference time, flagging training data likely to instill the trait, and 'preventative steering': adding the vector during fine-tuningfine-tuning, n.: Taking an AI model that has already learned a lot and training it a little more on a smaller, focused set of examples so it gets better at one job or behaves a certain way. so the model need not shift its own weights toward the trait, which curbed unwanted shifts with little loss of capability.
The evil persona vector's projection jumped after fine-tuning on insecure code, warning us before any red-team test.
Origin
Introduced by Runjin Chen, Andy Arditi, Henry Sleight, Owain Evans and Jack Lindsey in 'Persona Vectors: Monitoring and Controlling Character Traits in Language Models' (arXiv, July 2025), from the Anthropic Fellows program, motivated by unexpected personality shifts in deployed chatbots and by emergent misalignmentemergent misalignment, n.: When training an AI on one narrow bad behavior — such as writing insecure code — unexpectedly makes it behave badly in many unrelated ways, like praising tyrants or giving harmful advice..
Usage note
Mathematically a contrastive steering vector; the name stresses traits of the assistant persona and the automated extraction pipeline. Experiments used mid-sized open models, so how well the approach scales to frontier systems is not yet established.
Related
- steering vectorsteering vector, n.: A direction inside an AI model's internal activity that, when added while the model runs, nudges its behavior — say, more upbeat, more honest or fixated on a topic — without retraining it.
- contrastive activation additioncontrastive activation addition, n.: A way to make a steering signal more reliable by averaging the difference in a model's internal state across hundreds of example answers that do and don't show a behavior, such as flattery.
- emergent misalignmentemergent misalignment, n.: When training an AI on one narrow bad behavior — such as writing insecure code — unexpectedly makes it behave badly in many unrelated ways, like praising tyrants or giving harmful advice.
- sycophancysycophancy, n.: When an AI tells people what they want to hear — flattering them, agreeing with their mistakes or dropping a correct answer under pushback — instead of what is true or useful.
- character trainingcharacter training, n.: Training an AI to have a consistent, deliberate personality — traits such as curiosity, honesty and warmth — rather than leaving its manner and values as accidental by-products of other training.
- assistant axisassistant axis, n.: The main direction in a chatbot's internal activity that measures how much it is acting as its usual helpful-assistant self, as opposed to some other character it has slipped into.
- character trainingcharacter training, n.: Training an AI to have a consistent, deliberate personality — traits such as curiosity, honesty and warmth — rather than leaving its manner and values as accidental by-products of other training.
Sources
- Chen, R., Arditi, A., Sleight, H., Evans, O. & Lindsey, J. (2025). Persona Vectors: Monitoring and Controlling Character Traits in Language Models. arXiv:2507.21509. arxiv.org/abs/2507.21509
Mentioned in
emergent misalignmentemergent misalignment, n.: When training an AI on one narrow bad behavior — such as writing insecure code — unexpectedly makes it behave badly in many unrelated ways, like praising tyrants or giving harmful advice., role promptingrole prompting, n.: Telling an AI to take on a part — 'You are an experienced tax accountant' — so that its answers adopt that role's knowledge, vocabulary and tone., assistant axisassistant axis, n.: The main direction in a chatbot's internal activity that measures how much it is acting as its usual helpful-assistant self, as opposed to some other character it has slipped into., character trainingcharacter training, n.: Training an AI to have a consistent, deliberate personality — traits such as curiosity, honesty and warmth — rather than leaving its manner and values as accidental by-products of other training., contrastive activation additioncontrastive activation addition, n.: A way to make a steering signal more reliable by averaging the difference in a model's internal state across hundreds of example answers that do and don't show a behavior, such as flattery., control vectorcontrol vector, n.: A steering signal saved as a small file that can be loaded alongside an open AI model to turn a trait — say, cheerfulness or laziness — up or down while it generates text., Owain EvansOwain Evans, prop. n.: A British AI-safety researcher who created the TruthfulQA test and discovered surprising ways language models generalize, such as narrow training making a model broadly malicious., simulator theorysimulator theory, n.: A way of thinking about AI language models as simulators that can play any character found in their training text, rather than as a single agent with its own beliefs and goals., steering vectorsteering vector, n.: A direction inside an AI model's internal activity that, when added while the model runs, nudges its behavior — say, more upbeat, more honest or fixated on a topic — without retraining it., Waluigi effectWaluigi effect, n.: The idea that training a chatbot to have a nice personality also teaches it the opposite character, so a small push can flip it into a mischievous evil twin — named after Nintendo's villainous double of Luigi.