MNIST
In plain wordsA famous collection of 70,000 small images of handwritten digits used to test image-recognition programs. Getting a model to read MNIST digits is the 'hello world' of machine learning.
A datasetdataset, n.: An organized collection of examples gathered for a purpose — such as photos with their captions, or questions with answers — used to teach or test an AI system. of 70,000 handwritten digits labeled 0 to 9 and split into 60,000 training and 10,000 test images, each a 28 × 28 grayscale picture with the digit scaled to a standard size and placed at the center. Its source was two collections from the US National Institute of Standards and Technology (NIST), one written by Census Bureau staff and the other by high-school students; MNIST's makers drew on both for each split, and kept the writers of training and test digits separate. Small, clean and fast to train on, MNIST became the default benchmarkbenchmark, n.: A fixed, shared test — a set of questions or tasks with an agreed way of scoring — used to compare AI systems with one another and to track progress over time. for classifiers and the first exercise in countless courses. Error rates fell from about 12 percent for a linear classifier to roughly 0.2 percent for ensembles of convolutional networksconvolutional neural network, n.: A kind of neural network built for images and other grid-like data. It slides small pattern detectors across the input, finding edges first, then shapes, then whole objects. by the early 2010s, after which it was considered solved and too easy to separate methods; harder drop-in replacements include Fashion-MNIST (2017).
If your model can't reach 98 percent on MNIST, something in the pipeline is broken.
Origin
Created at AT&T Bell Laboratories by Yann LeCunYann LeCun, prop. n.: A French-American computer scientist who pioneered the neural networks that recognize images, led Meta's AI research for over a decade, and argues that chatbots alone will not reach human-level AI., Corinna Cortes and Christopher Burges; it was in use in Bell Labs classifier comparisons by 1994 and was later distributed freely from LeCun's website, becoming widely known through the 1998 LeNet-5 paper.
Usage note
A standard example of benchmark saturationbenchmark saturation, n.: When the best AI systems score so close to the maximum on a test that it can no longer tell them apart or show further progress.. In 2019 Chhavi Yadav and Léon Bottou reconstructed 50,000 further test digits that had been left out of the original release (QMNIST).
Related
- LeNetLeNet, prop. n.: A family of early image-recognition neural networks built at Bell Labs from 1989 that learned to read handwritten digits. A version read huge numbers of bank checks, and its design underlies today's image models.
- benchmarkbenchmark, n.: A fixed, shared test — a set of questions or tasks with an agreed way of scoring — used to compare AI systems with one another and to track progress over time.
- datasetdataset, n.: An organized collection of examples gathered for a purpose — such as photos with their captions, or questions with answers — used to teach or test an AI system.
- ImageNetImageNet, prop. n.: A huge collection of more than 14 million labeled photos, sorted into object categories, whose yearly recognition contest from 2010 to 2017 sparked the deep learning boom.
- benchmark saturationbenchmark saturation, n.: When the best AI systems score so close to the maximum on a test that it can no longer tell them apart or show further progress.
- image classificationimage classification, n.: Teaching a computer to say what a picture shows by choosing a label, such as 'cat', 'pneumonia' or 'stop sign', from a fixed list of possibilities.
Sources
- LeCun, Y., Bottou, L., Bengio, Y. & Haffner, P. (1998). Gradient-Based Learning Applied to Document Recognition. Proceedings of the IEEE 86(11), 2278–2324.
- Bottou, L., Cortes, C., Denker, J. S. et al. (1994). Comparison of Classifier Methods: A Case Study in Handwritten Digit Recognition. Proceedings of the 12th International Conference on Pattern Recognition (ICPR 1994).
Mentioned in
benchmarkbenchmark, n.: A fixed, shared test — a set of questions or tasks with an agreed way of scoring — used to compare AI systems with one another and to track progress over time., datasetdataset, n.: An organized collection of examples gathered for a purpose — such as photos with their captions, or questions with answers — used to teach or test an AI system., image classificationimage classification, n.: Teaching a computer to say what a picture shows by choosing a label, such as 'cat', 'pneumonia' or 'stop sign', from a fixed list of possibilities., optical character recognitionoptical character recognition, n.: Software that picks out the letters and words in a picture — a road sign, a handwritten note, a page of an old book — and turns them into real text a computer can copy, search or read aloud., Yann LeCunYann LeCun, prop. n.: A French-American computer scientist who pioneered the neural networks that recognize images, led Meta's AI research for over a decade, and argues that chatbots alone will not reach human-level AI., ImageNetImageNet, prop. n.: A huge collection of more than 14 million labeled photos, sorted into object categories, whose yearly recognition contest from 2010 to 2017 sparked the deep learning boom., LeNetLeNet, prop. n.: A family of early image-recognition neural networks built at Bell Labs from 1989 that learned to read handwritten digits. A version read huge numbers of bank checks, and its design underlies today's image models., Bell LabsBell Labs, prop. n.: A famous American industrial research laboratory, long owned by the telephone company AT&T, where the transistor, information theory and key early machine-learning methods were invented., deep belief networkdeep belief network, n.: A many-layered neural network trained one layer at a time, introduced in 2006. It showed that deep networks could be trained after all and helped launch the deep learning boom, before simpler methods replaced it.