discriminative model
In plain wordsA model that learns only to tell things apart — spam or not spam, cat or dog — without learning how to produce new examples of those things.
A model that maps inputs directly to predictions, learning either p(y | x) or a decision rule with no probabilities at all, while leaving the distribution of the inputs themselves unmodeled. Most neural classifiersclassifier, n.: A program or model that sorts inputs into categories, for example flagging an email as spam or a chatbot message as harmful. Many AI safety filters are classifiers. trained with cross-entropycross-entropy, n.: A score of how surprised a model's predictions are by what actually happens; lower is better. Language models are trained mainly by driving this number down on real text. are discriminative, as are logistic regressionlogistic regression, n.: A simple, widely used method for yes-or-no predictions that outputs a probability — say, a 70% chance a customer cancels — by passing a weighted sum of inputs through an S-shaped curve., support vector machinessupport vector machine, n.: A classic method that separates two groups of examples with the widest possible gap — like laying the broadest possible road between them. A mathematical shortcut lets it draw curved boundaries too. and conditional random fields. Because they spend no capacity on modeling the inputs, discriminative models usually reach lower error when labeled data are plentiful, whereas generative classifiers such as naive Bayesnaive Bayes, n.: A simple, fast way of sorting items into categories using probabilities, which pretends that each clue is independent of the others. It is the classic technique behind early email spam filters. can approach their (higher) asymptotic error with fewer examples and handle missing inputs more naturally. In a generative adversarial networkgenerative adversarial network, n.: Two neural networks trained as rivals: one forges fake images, the other tries to spot the fakes. As both improve, the forger's output becomes realistic enough to fool people., the discriminator is a discriminative model trained to separate real data from generated samples.
For the fraud flag a discriminative model is enough; nobody needs to generate fake transactions.
Origin
The distinction is long-standing in statistical pattern recognition and speech recognition, where discriminative training of acoustic models was studied from the 1980s. Andrew Ng and Michael Jordan's NeurIPS 2001 paper, showing that either naive Bayes or logistic regression can win depending on training-set size, made the generative–discriminative pairing standard vocabulary.
Related
- generative modelgenerative model, n.: A model that learns what a kind of data looks like — faces, sentences, songs — well enough to produce new examples that resemble the originals without copying them.
- classifierclassifier, n.: A program or model that sorts inputs into categories, for example flagging an email as spam or a chatbot message as harmful. Many AI safety filters are classifiers.
- logistic regressionlogistic regression, n.: A simple, widely used method for yes-or-no predictions that outputs a probability — say, a 70% chance a customer cancels — by passing a weighted sum of inputs through an S-shaped curve.
- naive Bayesnaive Bayes, n.: A simple, fast way of sorting items into categories using probabilities, which pretends that each clue is independent of the others. It is the classic technique behind early email spam filters.
- decision boundarydecision boundary, n.: The dividing line, or surface, where a classifier switches from predicting one category to another. Simple models draw straight lines; flexible ones draw curves.
- generative adversarial networkgenerative adversarial network, n.: Two neural networks trained as rivals: one forges fake images, the other tries to spot the fakes. As both improve, the forger's output becomes realistic enough to fool people.
- generative modelgenerative model, n.: A model that learns what a kind of data looks like — faces, sentences, songs — well enough to produce new examples that resemble the originals without copying them.
Sources
- Ng, A. Y. & Jordan, M. I. (2002). On Discriminative vs. Generative Classifiers: A comparison of logistic regression and naive Bayes. Advances in Neural Information Processing Systems 14 (NIPS 2001).
Mentioned in
classifierclassifier, n.: A program or model that sorts inputs into categories, for example flagging an email as spam or a chatbot message as harmful. Many AI safety filters are classifiers., generative AIgenerative AI, n.: AI that produces something new on request, such as an essay, a picture, a song or a piece of code, having absorbed from enormous numbers of existing examples what such things typically look like., modelmodel, n.: A simplified working version of something. In AI it usually means the trained program, a big set of learned numbers, that takes an input and produces an answer, prediction or piece of content., conditional random fieldconditional random field, n.: A statistical method for labeling every item in a sequence, such as marking each word in a sentence as a name, a place or an ordinary word, by choosing all the labels together rather than one at a time., generative adversarial networkgenerative adversarial network, n.: Two neural networks trained as rivals: one forges fake images, the other tries to spot the fakes. As both improve, the forger's output becomes realistic enough to fool people., generative modelgenerative model, n.: A model that learns what a kind of data looks like — faces, sentences, songs — well enough to produce new examples that resemble the originals without copying them., naive Bayesnaive Bayes, n.: A simple, fast way of sorting items into categories using probabilities, which pretends that each clue is independent of the others. It is the classic technique behind early email spam filters., predictive AIpredictive AI, n.: AI used to forecast outcomes about people or events, such as who will repay a loan, which patient may get sicker or which machine will fail, in order to guide decisions.