Sparks of AGI
In plain wordsA 2023 Microsoft research paper arguing that an early version of GPT-4 showed the first glimmers of general, humanlike intelligence. Its claim provoked sharp debate about how to judge AI abilities.
'Sparks of Artificial General Intelligence: Early experiments with GPT-4', a 155-page report by Microsoft Research scientists who had studied a pre-release version of GPT-4GPT-4, prop. n.: OpenAI's 2023 language model, a big step up from the one behind the first ChatGPT. It scored well on many professional and academic exams and could also read images. in late 2022. Using qualitative probes meant to avoid memorized material — drawing a unicorn by writing TikZ graphics code, proving the infinitude of primes as a poem, and novel coding, mathematics, theory-of-mind and planning tasks — the authors argued that its abilities were broad and general enough that it 'could reasonably be viewed as an early (yet still incomplete)' form of artificial general intelligenceartificial general intelligence, n.: AI that could learn and do almost any mental task a capable adult can, rather than excelling at one narrow job. Experts disagree on the exact bar and on when, or whether, it will arrive.. They also catalogued failures in planning, arithmetic and consistency. Critics objected that the claims were not peer reviewed, could not be reproduced on the unreleased model, rested on anecdotes rather than systematic evaluation, and came from a company with a commercial stake in the result.
Half the field cited Sparks of AGI as a milestone; the other half cited it as a cautionary tale.
Origin
Posted to arXiv on 22 March 2023, eight days after GPT-4's release, by Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro and Yi Zhang.
Usage note
The paper's hedged AGI framing was widely reported as a claim that AGI had arrived, and 'sparks of AGI' became shorthand for the debate over whether large language models generalize or mainly interpolate their training data.
Related
- GPT-4GPT-4, prop. n.: OpenAI's 2023 language model, a big step up from the one behind the first ChatGPT. It scored well on many professional and academic exams and could also read images.
- artificial general intelligenceartificial general intelligence, n.: AI that could learn and do almost any mental task a capable adult can, rather than excelling at one narrow job. Experts disagree on the exact bar and on when, or whether, it will arrive.
- emergent abilitiesemergent abilities, n.: Skills that seem to switch on suddenly as AI models grow: small versions fail at them, then larger ones past a certain size succeed. Some researchers argue the suddenness is partly an artifact of how skills are measured.
- benchmarkbenchmark, n.: A fixed, shared test — a set of questions or tasks with an agreed way of scoring — used to compare AI systems with one another and to track progress over time.
- Microsoft ResearchMicrosoft Research, prop. n.: Microsoft's research division, with laboratories on several continents, whose scientists have produced influential work on speech, vision and language AI.
Sources
- Bubeck, S. et al. (2023). Sparks of Artificial General Intelligence: Early experiments with GPT-4. arXiv:2303.12712.
Mentioned in
artificial general intelligenceartificial general intelligence, n.: AI that could learn and do almost any mental task a capable adult can, rather than excelling at one narrow job. Experts disagree on the exact bar and on when, or whether, it will arrive., GPT-4GPT-4, prop. n.: OpenAI's 2023 language model, a big step up from the one behind the first ChatGPT. It scored well on many professional and academic exams and could also read images., Microsoft ResearchMicrosoft Research, prop. n.: Microsoft's research division, with laboratories on several continents, whose scientists have produced influential work on speech, vision and language AI.