octopus test
In plain wordsA thought experiment in which a clever octopus learns to impersonate a stranded islander by eavesdropping on an undersea cable, without ever knowing what the words refer to. It asks whether text-trained AI can grasp meaning.
A thought experiment by Emily M. Bender and Alexander Koller arguing that a system trained only on linguistic form cannot learn meaning. Two people, A and B, stranded on separate islands, communicate by an undersea cable; a hyperintelligent deep-sea octopus, O, taps the cable, learns the statistical patterns of their messages, then cuts B off and impersonates B. O can sustain small talk, but when A urgently asks how to fend off a bear with some sticks, O, never having encountered the things the words refer to, cannot help. The authors' general claim is that meaning relates linguistic form to something outside language — the world and speakers' communicative intentions — so it cannot be learned from form alone. Critics reply that text encodes much of the world's structure, that the bear scenario tests knowledge rather than meaning, and that multimodal models weaken the premise.
Origin
Introduced by Emily M. Bender and Alexander Koller in 'Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data' at ACL 2020, the annual meeting of the Association for Computational Linguistics.
Related
- symbol grounding problemsymbol grounding problem, n.: The puzzle of how words or symbols inside a machine can mean anything by themselves, rather than only pointing to more symbols — like a dictionary that defines every word using other words.
- machine understandingmachine understanding, n.: Whether, and in what sense, an AI system grasps what words, situations or problems mean — rather than just producing answers that look as though it does.
- stochastic parrotstochastic parrot, n.: A critical label for AI language systems, likening them to parrots that echo patterns in the text they were trained on, fluently but with no grasp of what the words mean.
- Chinese roomChinese room, n.: A thought experiment: a person who knows no Chinese follows a rulebook to answer Chinese messages so well that outsiders think they understand. If the person doesn't understand, does a computer running a program?
- distributional hypothesisdistributional hypothesis, n.: The idea that words found in similar company — surrounded by similar words — tend to be similar in meaning, the insight that lets computers learn something about word meaning from large amounts of text alone.
- Emily M. BenderEmily M. Bender, prop. n.: An American linguist who argues that language models only mimic the form of language without understanding it, and who co-coined the phrase 'stochastic parrots'.
Sources
- Bender, E. M. & Koller, A. (2020). Climbing towards NLU: On Meaning, Form, and Understanding in the Age of Data. Proceedings of ACL 2020, 5185–5198.
Mentioned in
Emily M. BenderEmily M. Bender, prop. n.: An American linguist who argues that language models only mimic the form of language without understanding it, and who co-coined the phrase 'stochastic parrots'., machine understandingmachine understanding, n.: Whether, and in what sense, an AI system grasps what words, situations or problems mean — rather than just producing answers that look as though it does., natural language understandingnatural language understanding, n.: The part of language AI concerned with working out what a piece of text means — who did what to whom, what the writer wants and what follows from it — rather than with producing text., symbol grounding problemsymbol grounding problem, n.: The puzzle of how words or symbols inside a machine can mean anything by themselves, rather than only pointing to more symbols — like a dictionary that defines every word using other words., distributional hypothesisdistributional hypothesis, n.: The idea that words found in similar company — surrounded by similar words — tend to be similar in meaning, the insight that lets computers learn something about word meaning from large amounts of text alone., knowledge argumentknowledge argument, n.: A thought experiment about Mary, a scientist who knows every physical fact about color but has lived all her life in a black-and-white room. When she first sees red, does she learn something new?