Skip to content
Last Dictionary
Plane
Core ConceptsMachine Learning

semi-supervised learning

nounAlso: semisupervised learning, self-training, pseudo-labeling

In plain wordsMachine learning that combines a small set of labeled examples with a much larger pile of unlabeled ones, which are cheaper to collect, to learn better than from the labeled data alone.

  1. The machine learningmachine learning, n.: A way of building software that learns from examples instead of following rules written by a programmer. Shown thousands of labeled photos, for instance, a program can learn to recognize cats. setting in which a model is trained on a small set of labeled examples plus a large set of unlabeled ones, using the unlabeled data to improve on purely supervised learningsupervised learning, n.: Teaching a computer by example: you give it many inputs paired with the right answers, such as photos labeled 'cat' or 'dog', and it learns to answer correctly for new inputs.. It works only if the shape of the unlabeled data says something about the labels: for instance, that the data lie near a low-dimensional manifold (the manifold assumption), that points close together tend to share a label (smoothness), or that classes form clusters separated by sparse regions (the cluster assumption). Methods include self-training or pseudo-labeling (the model labels unlabeled data with its own confident predictions and retrains), co-training on two views of the data, graph-based label propagation, generative models, and consistency regularization, which demands stable predictions under perturbations. Transductive variants label only the given unlabeled points rather than learning a general rule.

    With only 500 labeled emails, semi-supervised learning on the 2 million unlabeled ones cut our error rate by a third.

Origin

Self-training appears in H. J. Scudder's 1965 work on adaptive pattern-recognition machines, and Vladimir Vapnik formalized transductive inference in the 1970s. Avrim Blum and Tom Mitchell's co-training (1998) and the 2006 volume Semi-Supervised Learning, edited by Olivier Chapelle, Bernhard Schölkopf and Alexander Zien, consolidated the field and its vocabulary.

Usage note

The abbreviation SSL is ambiguous, since it also stands for self-supervised learningself-supervised learning, n.: A way for AI to learn from huge amounts of unlabeled data by setting itself fill-in-the-blank puzzles, such as guessing a hidden word or a missing patch of an image., which needs no labels at all. Since large-scale self-supervised pretraining became standard, semi-supervised methods are usually combined with it rather than used alone.

Related

See also
  • supervised learningsupervised learning, n.: Teaching a computer by example: you give it many inputs paired with the right answers, such as photos labeled 'cat' or 'dog', and it learns to answer correctly for new inputs.
  • unsupervised learningunsupervised learning, n.: Machine learning without answer keys: the computer looks at unlabeled data and finds structure on its own, such as groups of similar customers or the main ways in which data vary.
  • weak supervisionweak supervision, n.: Training AI with labels that are cheap but imperfect, such as rough rules of thumb, partial tags or noisy guesses, instead of carefully hand-checked answers.
  • self-supervised learningself-supervised learning, n.: A way for AI to learn from huge amounts of unlabeled data by setting itself fill-in-the-blank puzzles, such as guessing a hidden word or a missing patch of an image.
  • active learningactive learning, n.: A way to train a model with fewer labeled examples by letting it choose which unlabeled examples a person should label next, usually the ones it is most unsure about.
  • labellabel, n.: The correct answer attached to an example in a dataset — 'cat' on a photo of a cat, 'positive' on an upbeat review — that a learning system is trained to predict.
  • manifold hypothesismanifold hypothesis, n.: The idea that real-world data, such as photos of faces, fills only a thin, smoothly curved sliver of the vast space of all possible data — which is why learning from examples works at all.
Contrast with
  • self-supervised learningself-supervised learning, n.: A way for AI to learn from huge amounts of unlabeled data by setting itself fill-in-the-blank puzzles, such as guessing a hidden word or a missing patch of an image.

Sources

  1. Scudder, H. J. (1965). Probability of Error of Some Adaptive Pattern-Recognition Machines. IEEE Transactions on Information Theory, 11(3), 363–371.
  2. Blum, A. & Mitchell, T. (1998). Combining Labeled and Unlabeled Data with Co-Training. Proceedings of COLT 1998.
  3. Chapelle, O., Schölkopf, B. & Zien, A. (eds) (2006). Semi-Supervised Learning. MIT Press.

Mentioned in

active learningactive learning, n.: A way to train a model with fewer labeled examples by letting it choose which unlabeled examples a person should label next, usually the ones it is most unsure about., bootstrapbootstrap, n.: Estimating how much a result might vary by repeatedly re-drawing samples, with replacement, from the data you already have — like re-running a study many times without collecting new data., learninglearning, n.: Getting better at something through experience. For a machine, it means changing its internal settings based on data so that it performs a task better than before., self-supervised learningself-supervised learning, n.: A way for AI to learn from huge amounts of unlabeled data by setting itself fill-in-the-blank puzzles, such as guessing a hidden word or a missing patch of an image., supervised learningsupervised learning, n.: Teaching a computer by example: you give it many inputs paired with the right answers, such as photos labeled 'cat' or 'dog', and it learns to answer correctly for new inputs., weak supervisionweak supervision, n.: Training AI with labels that are cheap but imperfect, such as rough rules of thumb, partial tags or noisy guesses, instead of carefully hand-checked answers.