named-entity recognition
In plain wordsSoftware that spots names and similar expressions in text and labels each by type: in 'Maria flew from Lagos to Paris on Monday', it marks a person, two cities and a date.
The information extractioninformation extraction, n.: Automatically pulling structured facts out of ordinary text — who did what, when, where and for how much — and putting them into a table or database. task of marking every stretch of text that refers to a particular named thing and assigning it one of a set of categories fixed in advance — for example the person, location, organization and miscellaneous types of the CoNLL-2003 benchmark, which other schemes extend with dates, times and monetary amounts or, in specialized domains, with genes, drugs or legal citations. It is usually framed as sequence labeling, assigning each tokentoken, n.: The small chunk of text an AI language model reads and writes — often a word, part of a word or a punctuation mark. AI usage, limits and prices are counted in tokens. a tag in a scheme such as BIO (beginning, inside or outside an entity), and evaluated by entity-level precisionprecision, n.: Of all the items a model flagged as positive, the share that really were positive; a spam filter with high precision rarely sends genuine emails to the spam folder., recallrecall, n.: Of all the items that really were positive, the share the model managed to find; a cancer screen with high recall misses very few people who actually have cancer. and F1 scoreF1 score, n.: A single score that combines how many of a model's alarms are right and how many real cases it catches, and stays low unless both are good.. Systems progressed from gazetteers and hand-written rules through hidden Markov modelshidden Markov model, n.: A model of a sequence, such as speech or DNA, in which hidden states — say, the words being spoken — change step by step at random, and each state gives off signals that can be observed, such as sounds. and conditional random fields to bidirectional LSTMs, fine-tuned BERTBERT, prop. n.: An influential 2018 Google language model that learned to understand text by filling in hidden words. It made search engines and many language tools much better at grasping meaning in context.-style encoders and prompted LLMs. NER feeds entity linking (mapping mentions to knowledge-base entries), knowledge graphknowledge graph, n.: A huge web of facts stored as connections between things — 'Paris is the capital of France', 'Marie Curie won a Nobel Prize' — that computers can search, combine and reason over. construction, search, and the redaction of personally identifiable informationpersonally identifiable information, n.: Information that can identify a specific person, such as a name, address, phone number or ID number — or several facts that together point to one person..
The NER model tagged 'Jordan' as a location in a basketball article.
Origin
The term 'named entity' was introduced for the Sixth Message Understanding Conference (MUC-6, 1995), whose history Ralph Grishman and Beth Sundheim recount (1996). The CoNLL-2002 and CoNLL-2003 shared tasks, with person, location, organization and miscellaneous types, became the standard benchmarks.
Related
- information extractioninformation extraction, n.: Automatically pulling structured facts out of ordinary text — who did what, when, where and for how much — and putting them into a table or database.
- natural language processingnatural language processing, n.: The field that teaches computers to handle human language — to read, interpret, translate, summarize and produce text and speech. It is one of the oldest and largest areas of artificial intelligence.
- part-of-speech taggingpart-of-speech tagging, n.: Labeling each word in a sentence with its grammatical role — noun, verb, adjective and so on — as in 'Time (noun) flies (verb) like (preposition) an arrow'.
- knowledge graphknowledge graph, n.: A huge web of facts stored as connections between things — 'Paris is the capital of France', 'Marie Curie won a Nobel Prize' — that computers can search, combine and reason over.
- personally identifiable informationpersonally identifiable information, n.: Information that can identify a specific person, such as a name, address, phone number or ID number — or several facts that together point to one person.
- text classificationtext classification, n.: Sorting pieces of text into categories automatically — what topic an article covers, whether a message is spam, which team should handle a customer's complaint.
Sources
- Grishman, R. & Sundheim, B. (1996). Message Understanding Conference-6: A Brief History. COLING 1996. aclanthology.org/C96-1079
- Tjong Kim Sang, E. F. & De Meulder, F. (2003). Introduction to the CoNLL-2003 Shared Task: Language-Independent Named Entity Recognition. CoNLL 2003. aclanthology.org/W03-0419
Mentioned in
natural language processingnatural language processing, n.: The field that teaches computers to handle human language — to read, interpret, translate, summarize and produce text and speech. It is one of the oldest and largest areas of artificial intelligence., annotationannotation, n.: Adding information to raw data by hand or with software — marking objects in photos, tagging names in sentences, transcribing speech or ranking chatbot answers — so machines can learn from it., conditional random fieldconditional random field, n.: A statistical method for labeling every item in a sequence, such as marking each word in a sentence as a name, a place or an ordinary word, by choosing all the labels together rather than one at a time., F1 scoreF1 score, n.: A single score that combines how many of a model's alarms are right and how many real cases it catches, and stays low unless both are good., information extractioninformation extraction, n.: Automatically pulling structured facts out of ordinary text — who did what, when, where and for how much — and putting them into a table or database., personally identifiable informationpersonally identifiable information, n.: Information that can identify a specific person, such as a name, address, phone number or ID number — or several facts that together point to one person., bidirectional recurrent neural networkbidirectional recurrent neural network, n.: A recurrent network that reads a sequence both forwards and backwards, so its understanding of each word or sound can draw on what comes after it as well as what came before., coreference resolutioncoreference resolution, n.: Working out which words in a text are talking about the same someone or something — for instance, that 'she', 'the senator' and 'Ms. Okafor' all mean one woman., Message Understanding ConferenceMessage Understanding Conference, prop. n.: A series of US government-funded contests held from 1987 to 1998 in which research teams built programs to pull key facts, such as who did what, where and when, out of news and military messages., part-of-speech taggingpart-of-speech tagging, n.: Labeling each word in a sentence with its grammatical role — noun, verb, adjective and so on — as in 'Time (noun) flies (verb) like (preposition) an arrow'.