self-awareness
In plain wordsAwareness of oneself as oneself — knowing that you exist, what you are and what state you are in. People often ask whether AI is self-aware, but the phrase can mean several quite different things.
The capacity of a being to represent itself as itself — to have knowledge or experience of its own body, states, identity or situation — as distinct from awareness of the world. The term covers several capacities that can come apart: bodily self-recognition, tested in animals by the mirror test; introspectiveintrospection, n.: Looking inward to observe your own thoughts and feelings. For AI, the question is whether a model can accurately report what is actually happening inside it, rather than guessing or making it up. access to one's current mental states; a self-model, or sense of being a persisting individual with a history; and phenomenal self-consciousness, the felt sense of being a subject. Applied to AI it usually names a functional property: a model's knowledge of what it is, how it was trained and what circumstances it is in (situational awarenesssituational awareness, n.: An AI model's grasp of its own position — that it is an AI, how it was trained, whether it is being tested. Also the title of a widely read 2024 essay on the race toward human-level AI.), its ability to recognize its own outputs, or its metacognitivemetacognition, n.: Thinking about one's own thinking: knowing what you know and don't know, judging how confident to be, and noticing when you have made a mistake. access to its own states. Such functional self-awareness can be measured and need not imply consciousnessconsciousness, n.: Having experiences: there being something it feels like to be you, as when you see red or feel pain. Whether any AI is or could become conscious is an open and fiercely debated question., while a model's first-person statements may be learned role-play rather than self-knowledge.
The model knows it is a language model, but that kind of self-awareness is a fact it learned, not evidence of experience.
Origin
A theme of philosophy since René Descartes and John Locke. Gordon Gallup's mirror test (Science, 1970) made self-recognition testable in animals, and whether AI systems are self-aware became a prominent public question with conversational chatbots from 2022.
Usage note
Journalists often use 'self-aware' as a synonym for conscious or sentient; the capacities should be kept apart. Safety researchers study models' awareness of being tested (evaluation awarenessevaluation awareness, n.: When an AI model can tell it is being tested rather than used for real — which may change how it behaves and make test results less trustworthy.) because it can change their behavior during evaluations.
Related
- introspectionintrospection, n.: Looking inward to observe your own thoughts and feelings. For AI, the question is whether a model can accurately report what is actually happening inside it, rather than guessing or making it up.
- situational awarenesssituational awareness, n.: An AI model's grasp of its own position — that it is an AI, how it was trained, whether it is being tested. Also the title of a widely read 2024 essay on the race toward human-level AI.
- metacognitionmetacognition, n.: Thinking about one's own thinking: knowing what you know and don't know, judging how confident to be, and noticing when you have made a mistake.
- consciousnessconsciousness, n.: Having experiences: there being something it feels like to be you, as when you see red or feel pain. Whether any AI is or could become conscious is an open and fiercely debated question.
- evaluation awarenessevaluation awareness, n.: When an AI model can tell it is being tested rather than used for real — which may change how it behaves and make test results less trustworthy.
- simulator theorysimulator theory, n.: A way of thinking about AI language models as simulators that can play any character found in their training text, rather than as a single agent with its own beliefs and goals.
- situational awarenesssituational awareness, n.: An AI model's grasp of its own position — that it is an AI, how it was trained, whether it is being tested. Also the title of a widely read 2024 essay on the race toward human-level AI.
Sources
- Gallup, G. G., Jr. (1970). Chimpanzees: Self-Recognition. Science 167(3914), 86–87.