Loebner Prize
In plain wordsA yearly contest, held from 1991 to 2019, in which chatbots tried to convince judges they were human in short text chats. Its grand prize was never won, and critics called it a publicity stunt.
A yearly contest, run from 1991 to 2019, in which judges chatted by keyboard with hidden humans and chatbotschatbot, n.: Software you chat with in ordinary language, by typing or talking, as you would with a person. The newest ones, such as ChatGPT, can answer questions, draft writing and keep long conversations going. and then ranked the programs by how humanlike they seemed — a restricted form of the Turing testTuring test, n.: Alan Turing's proposed test of machine intelligence: a judge holds typed conversations with a hidden person and a hidden computer, and the computer passes if the judge cannot consistently pick it out.. A bronze medal and cash prize went each year to the most humanlike program; a silver prize for passing a text-only test and a gold prize for a test including audio and video were never awarded. Early contests limited conversation to chosen topics. Researchers criticized the contest for rewarding tricks — deliberate typos, evasions, scripted personality quirks — rather than intelligence, and most AI laboratories ignored it. Steve Worswick's chatbot Mitsuku (later Kuki) won five times, the last in 2019.
Winning the Loebner Prize mostly required a chatbot skilled at changing the subject.
Origin
Founded by the American inventor Hugh Loebner with the Cambridge Center for Behavioral Studies; the first contest was held at the Computer Museum in Boston in November 1991. In its final years it was organized by the Society for the Study of Artificial Intelligence and Simulation of Behaviour (AISB), which continued it after Loebner's death in 2016 until it lapsed after 2019.
Usage note
Marvin Minsky mockingly offered a prize to anyone who could persuade Loebner to end the contest. Stuart Shieber's 1994 article 'Lessons from a Restricted Turing Test' set out the academic objections.
Related
- Turing testTuring test, n.: Alan Turing's proposed test of machine intelligence: a judge holds typed conversations with a hidden person and a hidden computer, and the computer passes if the judge cannot consistently pick it out.
- chatbotchatbot, n.: Software you chat with in ordinary language, by typing or talking, as you would with a person. The newest ones, such as ChatGPT, can answer questions, draft writing and keep long conversations going.
- ELIZAELIZA, prop. n.: A 1960s computer program that chatted by turning a user's own words back into questions, like a therapist. Many users felt understood, though the program understood nothing.
- ELIZA effectELIZA effect, n.: Our tendency to believe that a computer understands us, cares or thinks just because it produces humanlike words, even when it is only following simple rules.
- Winograd schema challengeWinograd schema challenge, prop. n.: A test of machine common sense built from pairs of sentences in which changing one word changes what a pronoun refers to — something people resolve instantly using everyday knowledge.
Sources
- Shieber, S. M. (1994). Lessons from a Restricted Turing Test. Communications of the ACM 37(6), 70–78.
Mentioned in
chatbotchatbot, n.: Software you chat with in ordinary language, by typing or talking, as you would with a person. The newest ones, such as ChatGPT, can answer questions, draft writing and keep long conversations going., Turing testTuring test, n.: Alan Turing's proposed test of machine intelligence: a judge holds typed conversations with a hidden person and a hidden computer, and the computer passes if the judge cannot consistently pick it out., Daniel DennettDaniel Dennett, prop. n.: An American philosopher who argued that minds are what brains do, that consciousness can in principle be explained scientifically, and that we understand machines by treating them as if they had beliefs and goals., PARRYPARRY, prop. n.: An early-1970s chatbot built to talk like a psychiatric patient with paranoid schizophrenia. In tests, psychiatrists reading its conversations did little better than chance at telling it from real patients — an early Turing-test-style result.