Situational Awareness (essay)
In plain wordsA long 2024 essay by a former OpenAI researcher arguing that AGI could arrive around 2027, that superintelligence could follow soon after, and that the US government should run a national AGI project.
An essay series published in June 2024 arguing that AGIartificial general intelligence, n.: AI that could learn and do almost any mental task a capable adult can, rather than excelling at one narrow job. Experts disagree on the exact bar and on when, or whether, it will arrive. by 2027 is 'strikingly plausible' and that superintelligencesuperintelligence, n.: A mind, usually imagined as an AI, that would leave even the cleverest people far behind in nearly every kind of thinking, not just in one game or skill. None is known to exist. could follow within about a year through automated AI research. Its forecast counts orders of magnitude ('OOMs') of effective computeeffective compute, n.: A way of measuring AI progress that counts better methods as if they were extra computing power: if a new technique lets a model match one trained with twice the computing, it counts as doubling effective compute. from three sources: growth in physical training compute, gains in algorithmic efficiency, and 'unhobbling', the capability gained by turning chatbots into agents with tools, memory and long-horizon reasoning, which it projected would yield drop-in remote workers. It predicted training clusters costing around a trillion dollars by about 2030 and treated AGI as a matter of national security: laboratories must be secured against state espionage, especially by China, and around 2027–2028 the US government would, and should, take over frontier development in a Manhattan Project-style effort the author called the Project. Widely read in Washington and Silicon Valley, it was criticized for its straight-line extrapolations and its race framing.
Half the arguments about AGI in 2024 were arguments with Situational Awareness.
Origin
Written by Leopold Aschenbrenner, a former member of OpenAI's superalignment team, and published online in June 2024 as Situational Awareness: The Decade Ahead; he then founded an investment firm of the same name.
Usage note
Not to be confused with situational awarenesssituational awareness, n.: An AI model's grasp of its own position — that it is an AI, how it was trained, whether it is being tested. Also the title of a widely read 2024 essay on the race toward human-level AI., a model's knowledge of its own nature and circumstances, which is a separate concept in AI safety.
Related
- AI timelinesAI timelines, n.: Forecasts of when AI will reach major milestones, such as matching humans at most work. People with short timelines expect this within years; others expect decades, or never.
- intelligence explosionintelligence explosion, n.: The idea that once AI gets good enough at improving AI, each smarter version could build a smarter one faster still, so abilities would snowball far beyond human level in a short time.
- AI arms raceAI arms race, n.: A competition among countries or companies to build the most powerful AI first, in which fear of falling behind pushes everyone to move faster and cut corners on safety.
- effective computeeffective compute, n.: A way of measuring AI progress that counts better methods as if they were extra computing power: if a new technique lets a model match one trained with twice the computing, it counts as doubling effective compute.
- automated AI researchautomated AI research, n.: Using AI systems to do the work of AI researchers, such as running experiments and writing code for new models, so that AI helps build its own successors.
- superalignmentsuperalignment, n.: The problem of keeping AI systems far smarter than humans aligned with human intentions — and the name of OpenAI's short-lived 2023–2024 research team devoted to it.
- situational awarenesssituational awareness, n.: An AI model's grasp of its own position — that it is an AI, how it was trained, whether it is being tested. Also the title of a widely read 2024 essay on the race toward human-level AI.
Sources
- Aschenbrenner, L. (2024). Situational Awareness: The Decade Ahead. Online essay series.
Mentioned in
Leopold AschenbrennerLeopold Aschenbrenner, prop. n.: A German-born former OpenAI researcher whose 2024 essay series 'Situational Awareness' predicted human-level AI around 2027 and a government-led race to build superintelligence.