Skip to content
Last Dictionary
Plane
Machine Learning

one-vs-rest

nounAlso: OvR, one-vs-all, OvA, one-against-all, one-vs-one, OvO, all-pairs

In plain wordsA way to handle many categories with a method that only gives yes-or-no answers: train one detector per category ('this one or not?') and pick the category whose detector is most confident.

  1. A strategy that reduces a multiclass classificationmulticlass classification, n.: Sorting each item into exactly one of three or more categories — for example, deciding which of the ten digits a handwritten number shows. problem with K classes to K binary classificationbinary classification, n.: Sorting things into exactly two groups, such as spam versus not spam or sick versus healthy. Most yes-or-no decisions made by AI systems are of this kind. problems, the k-th separating class k from all the others; at prediction time every classifier scores the input and the class with the highest score wins. It lets inherently binary learners such as support vector machinessupport vector machine, n.: A classic method that separates two groups of examples with the widest possible gap — like laying the broadest possible road between them. A mathematical shortcut lets it draw curved boundaries too. handle many classes, and the same per-label construction handles multilabel classificationmultilabel classification, n.: Tagging each item with any number of labels at once — a photo can show a beach, a dog and a sunset — instead of picking just one category.. Each binary problem is imbalanced (one class against K − 1), and the classifiers' scores may not be on comparable scales, so they are often calibrated before comparison. The main alternative, one-vs-one, trains K(K − 1)/2 classifiers on pairs of classes and predicts by majority vote: each subproblem is smaller and balanced, but their number grows quadratically. Ryan Rifkin and Aldebaro Klautau (2004) showed that well-tuned one-vs-rest is as accurate as more elaborate schemes.

    Scikit-learn's LinearSVC trains one-vs-rest by default, so ten digit classes means ten classifiers.

Origin

Used informally since early multiclass pattern recognition; the one-vs-all and one-vs-one decompositions became standard with multiclass support vector machines in the 1990s, and Rifkin and Klautau's 'In Defense of One-Vs-All Classification' (Journal of Machine Learning Research, 2004) made an influential case for the simpler scheme, though libraries such as LIBSVM use one-vs-one.

Related

See also
  • multiclass classificationmulticlass classification, n.: Sorting each item into exactly one of three or more categories — for example, deciding which of the ten digits a handwritten number shows.
  • binary classificationbinary classification, n.: Sorting things into exactly two groups, such as spam versus not spam or sick versus healthy. Most yes-or-no decisions made by AI systems are of this kind.
  • multilabel classificationmultilabel classification, n.: Tagging each item with any number of labels at once — a photo can show a beach, a dog and a sunset — instead of picking just one category.
  • support vector machinesupport vector machine, n.: A classic method that separates two groups of examples with the widest possible gap — like laying the broadest possible road between them. A mathematical shortcut lets it draw curved boundaries too.
  • class imbalanceclass imbalance, n.: When one category is far rarer than the others in the data — say, one fraud case per thousand transactions — so a model can look accurate while missing nearly every rare case that matters.
  • calibrationcalibration, n.: How well a model's stated confidence matches reality: when a well-calibrated system says it is 80% sure, it turns out to be right about 80% of the time.

Sources

  1. Rifkin, R. & Klautau, A. (2004). In Defense of One-Vs-All Classification. Journal of Machine Learning Research, 5, 101–141.

Mentioned in

binary classificationbinary classification, n.: Sorting things into exactly two groups, such as spam versus not spam or sick versus healthy. Most yes-or-no decisions made by AI systems are of this kind., multiclass classificationmulticlass classification, n.: Sorting each item into exactly one of three or more categories — for example, deciding which of the ten digits a handwritten number shows., multilabel classificationmultilabel classification, n.: Tagging each item with any number of labels at once — a photo can show a beach, a dog and a sunset — instead of picking just one category., support vector machinesupport vector machine, n.: A classic method that separates two groups of examples with the widest possible gap — like laying the broadest possible road between them. A mathematical shortcut lets it draw curved boundaries too.