stochastic parrot
In plain wordsA critical label for AI language systems, likening them to parrots that echo patterns in the text they were trained on, fluently but with no grasp of what the words mean.
A characterization of large language modelslarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code. that denies them understanding. Its coiners pictured such a model as 'haphazardly stitching together sequences of linguistic forms' from its training data according to how they tend to co-occur, with no access to meaning or to anyone's intentions, so that whatever coherence a reader finds in the output is the reader's own contribution; they paired the image with warnings about the environmental and financial cost of ever-larger models, undocumented training data and encoded biasbias, n.: A word with several AI meanings: an adjustable number added inside a model; a systematic error in estimates; or unfair treatment of some groups of people by an AI system.. The phrase became the standard shorthand for the skeptical side of the debate over whether LLMs understand language. Opponents point to evidence that models build internal world modelsworld model, n.: An AI's internal simulation of how its world works, which lets it predict what will happen next — especially in response to its own actions — and so plan, imagine or practice without acting for real. and generalize in ways simple parroting cannot explain; defenders reply that such evidence is contested and that fluency is easily mistaken for comprehension.
Calling it a stochastic parrot does not explain how it solved a problem that appears nowhere online.
Origin
Coined by Emily M. Bender, Timnit Gebru, Angelina McMillan-Major and Margaret Mitchell (credited as 'Shmargaret Shmitchell') in 'On the Dangers of Stochastic Parrots: Can Language Models Be Too Big?', presented at the ACM FAccT conference in March 2021. Google's objections to the paper preceded the departures of Gebru (December 2020) and Mitchell (February 2021), which made the phrase widely known; ChatGPT's release in 2022 made it ubiquitous.
Usage note
Sometimes adopted ironically: in December 2022 OpenAI's Sam Altman posted that he, and everyone, was a stochastic parrot. Related dismissive terms include fancy autocompletefancy autocomplete, n.: A dismissive description of AI chatbots as nothing more than a souped-up version of the feature on a phone keyboard that suggests your next word..
Related
- fancy autocompletefancy autocomplete, n.: A dismissive description of AI chatbots as nothing more than a souped-up version of the feature on a phone keyboard that suggests your next word.
- Emily M. BenderEmily M. Bender, prop. n.: An American linguist who argues that language models only mimic the form of language without understanding it, and who co-coined the phrase 'stochastic parrots'.
- Timnit GebruTimnit Gebru, prop. n.: An Ethiopian-born American computer scientist who exposed racial and gender bias in AI systems, co-wrote the 'stochastic parrots' paper, and founded an independent AI research institute.
- Chinese roomChinese room, n.: A thought experiment: a person who knows no Chinese follows a rulebook to answer Chinese messages so well that outsiders think they understand. If the person doesn't understand, does a computer running a program?
- world modelworld model, n.: An AI's internal simulation of how its world works, which lets it predict what will happen next — especially in response to its own actions — and so plan, imagine or practice without acting for real.
- ELIZA effectELIZA effect, n.: Our tendency to believe that a computer understands us, cares or thinks just because it produces humanlike words, even when it is only following simple rules.
- Potemkin understandingPotemkin understanding, n.: When an AI can correctly explain a concept yet cannot use it properly, revealing that its apparent understanding is a facade — like the fake village fronts said to have been built to impress a Russian empress.
Sources
- Bender, E. M., Gebru, T., McMillan-Major, A. & Shmitchell, S. (2021). On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? Proceedings of FAccT 2021, ACM.
Mentioned in
large language modellarge language model, n.: A computer program trained on enormous amounts of text to predict the words that come next. At vast scale, that simple skill lets it chat, answer questions, translate, summarize and write code., next-token predictionnext-token prediction, n.: The simple task most AI chatbots were trained on: given some text, guess the next small piece of it. Repeating that guess, piece by piece, produces whole answers., AI hypeAI hype, n.: Exaggerated claims about what AI can do or will soon do, made to attract money, attention or customers, which can mislead the public and policymakers., ELIZA effectELIZA effect, n.: Our tendency to believe that a computer understands us, cares or thinks just because it produces humanlike words, even when it is only following simple rules., Emily M. BenderEmily M. Bender, prop. n.: An American linguist who argues that language models only mimic the form of language without understanding it, and who co-coined the phrase 'stochastic parrots'., fancy autocompletefancy autocomplete, n.: A dismissive description of AI chatbots as nothing more than a souped-up version of the feature on a phone keyboard that suggests your next word., GPT-3GPT-3, prop. n.: A huge 2020 OpenAI language model that could perform new tasks simply from instructions or a few examples written into its prompt, without retraining. It made prompting a new way of programming., machine understandingmachine understanding, n.: Whether, and in what sense, an AI system grasps what words, situations or problems mean — rather than just producing answers that look as though it does., Noam ChomskyNoam Chomsky, prop. n.: An American linguist whose theories of grammar reshaped linguistics and cognitive science, and who argues that chatbots like ChatGPT imitate language without understanding it., stochasticstochastic, adj.: Involving chance, so the same starting point can lead to different outcomes. AI chatbots are stochastic: ask the same question twice and you may get two different answers., Timnit GebruTimnit Gebru, prop. n.: An Ethiopian-born American computer scientist who exposed racial and gender bias in AI systems, co-wrote the 'stochastic parrots' paper, and founded an independent AI research institute., blurry JPEG of the webblurry JPEG of the web, phr.: Writer Ted Chiang's 2023 comparison of ChatGPT to a low-quality compressed photo of the internet: it keeps the gist of what it learned but blurs the details and fills gaps with plausible guesses., Distributed AI Research InstituteDistributed AI Research Institute, prop. n.: Timnit Gebru set up this institute in 2021, a year after her departure from Google, to research the damage AI can do, especially to marginalized communities, without relying on big-tech money., FAccTFAccT, prop. n.: A yearly academic conference on the fairness, accountability and transparency of computer systems, where much influential research on AI bias and ethics has been published., Margaret MitchellMargaret Mitchell, prop. n.: An American computer scientist who proposed 'model cards' for documenting AI systems, co-led Google's Ethical AI team, and is chief ethics scientist at Hugging Face., octopus testoctopus test, n.: A thought experiment in which a clever octopus learns to impersonate a stranded islander by eavesdropping on an undersea cable, without ever knowing what the words refer to. It asks whether text-trained AI can grasp meaning., Potemkin understandingPotemkin understanding, n.: When an AI can correctly explain a concept yet cannot use it properly, revealing that its apparent understanding is a facade — like the fake village fronts said to have been built to impress a Russian empress.