Penn Treebank
In plain wordsA collection of English text, best known for its million words of newspaper articles, in which experts marked each word's grammatical role and drew each sentence's structure as a tree; it trained and tested a generation of language software.
- 1
Corpus
A corpuscorpus, n.: A large collection of written or spoken language, gathered so that people or computers can study it or learn from it — like a library assembled for a language model. of American English annotated at the University of Pennsylvania with part-of-speech tagspart-of-speech tagging, n.: Labeling each word in a sentence with its grammatical role — noun, verb, adjective and so on — as in 'Time (noun) flies (verb) like (preposition) an arrow'. and phrase-structure (constituency) trees. Its best-known portion is about one million words of 1989 Wall Street Journal text bracketed into labeled constituents (S, NP, VP …), with empty elements and, from the 1995 release, function tags and predicate–argument information. Its 36 word-class tags (NN, VBZ, JJ …, plus punctuation tags) became the de facto English standard. The conventional split, with WSJ sections 02–21 for training, 22 for development and 23 for testing, made it the benchmark for statistical syntactic parsingsyntactic parsing, n.: Working out the grammatical structure of a sentence — which words group together and which word modifies which — usually drawn as a tree. for two decades, and it seeded derived resources such as PropBank and converted dependency treebanks. It is licensed through the Linguistic Data Consortium.
Parsers in the 2000s were ranked by their labeled F1 on Section 23 of the Penn Treebank.
- 2
Language modeling
A small language-modelinglanguage model, n.: A system that has learned how likely different sequences of words are, so it can predict the next word in a text and, by doing that repeatedly, write text of its own. benchmark made from the same Wall Street Journal text and distributed by Tomas Mikolov and colleagues around 2010: lowercased, with numbers replaced by a placeholder, most punctuation removed and the vocabulary capped at 10,000 words, leaving about 930,000 training words. Word-level perplexityperplexity, n.: A score for how surprised a language model is by real text; lower is better. A perplexity of 10 means guessing each next word is, on average, as hard for the model as guessing a ten-sided die's roll. on it was the standard yardstick for recurrent language models until larger corpora such as WikiText displaced it.
Their regularized LSTM brought test perplexity on PTB below 60.
Origin
Built from 1989 to 1992 by Mitchell Marcus, Beatrice Santorini and Mary Ann Marcinkiewicz at the University of Pennsylvania and described in Computational Linguistics in 1993. Treebank II (1995) added predicate–argument structure, and Treebank-3 (1999) is the release most used today. The word 'treebank' for a parsed corpus is attributed to the linguist Geoffrey Leech in the 1980s.
Usage note
Being newswire from one newspaper over a short period, it underrepresents other genres and varieties of English, so parsers tuned to it degrade on web text, speech and specialist domains. Its tagset and bracketing guidelines remain widely used English annotation standards, alongside the later Universal Dependencies scheme.
Related
- corpuscorpus, n.: A large collection of written or spoken language, gathered so that people or computers can study it or learn from it — like a library assembled for a language model.
- syntactic parsingsyntactic parsing, n.: Working out the grammatical structure of a sentence — which words group together and which word modifies which — usually drawn as a tree.
- part-of-speech taggingpart-of-speech tagging, n.: Labeling each word in a sentence with its grammatical role — noun, verb, adjective and so on — as in 'Time (noun) flies (verb) like (preposition) an arrow'.
- perplexityperplexity, n.: A score for how surprised a language model is by real text; lower is better. A perplexity of 10 means guessing each next word is, on average, as hard for the model as guessing a ten-sided die's roll.
- Brown CorpusBrown Corpus, prop. n.: The first large collection of English text stored on computer for research: about a million words from 500 American publications of 1961, assembled at Brown University in the 1960s.
- WordNetWordNet, prop. n.: A hand-made map of English vocabulary in which words sharing a meaning are bundled together and the bundles are linked — recording, say, that a dog is a kind of animal and a wheel is part of a car.
Sources
- Marcus, M. P., Santorini, B. & Marcinkiewicz, M. A. (1993). Building a Large Annotated Corpus of English: The Penn Treebank. Computational Linguistics 19(2), 313–330.
- Mikolov, T. et al. (2010). Recurrent Neural Network Based Language Model. Proc. Interspeech 2010.
Mentioned in
WordNetWordNet, prop. n.: A hand-made map of English vocabulary in which words sharing a meaning are bundled together and the bundles are linked — recording, say, that a dog is a kind of animal and a wheel is part of a car.