Skip to content
Last Dictionary
Plane
Deep LearningMachine Learning

metric learning

nounAlso: deep metric learning, similarity learning, distance metric learning, triplet loss

In plain wordsTeaching a model to measure similarity: it learns to place items so that ones that should count as alike end up close together and different ones far apart.

  1. The learning of a distance or similarity function from examples of which items should be considered similar, usually by training an encoder so that ordinary Euclidean distanceEuclidean distance, n.: How far apart two points are in a straight line, as a ruler would measure it, extended from two or three dimensions to lists of numbers of any length. or cosine similaritycosine similarity, n.: A score from −1 to 1 for how closely two lists of numbers point the same way, ignoring their size. It is the standard test of whether two texts mean similar things. between embeddingsembedding, n.: A list of numbers that stands for a word, picture or other item, arranged so that similar items get similar lists. It lets a computer measure similarity of meaning as distance. reflects the desired notion of similarity. Classical methods learn a Mahalanobis matrix, as in large-margin nearest neighbor; deep metric learning trains neural networks with pairwise contrastive losses, the triplet loss, which requires an anchor to be closer to a positive than to a negative by a margin α, or proxy- and softmax-based losses. Mining informative 'hard' negatives is critical. Applications include face and speaker verificationspeaker verification, n.: Checking whether a voice really belongs to the person it claims to be, as when a bank confirms your identity by the sound of your voice on the phone., image retrieval, person re-identification and few-shot classification; SiameseSiamese network, n.: A matching machine built from twin copies of one neural network: each copy turns one of two inputs, say two signatures or two face photos, into a summary, and the two summaries are compared. encoders and contrastive learningcontrastive learning, n.: A way to learn without labels by comparison: the model learns to pull together two versions of the same thing, such as two crops of one photo, and push apart unrelated things. are its main tools.

    Switching from triplet loss to a proxy-based metric learning loss doubled training speed.

Ltriplet=max⁡(0,  ∥f(a)−f(p)∥2−∥f(a)−f(n)∥2+α)\mathcal{L}_{\text{triplet}} = \max\big(0,\; \lVert f(a) - f(p) \rVert^2 - \lVert f(a) - f(n) \rVert^2 + \alpha\big)

Origin

Eric Xing, Andrew Ng, Michael Jordan and Stuart Russell posed distance metric learning as convex optimization at NIPS 2002. Kilian Weinberger, John Blitzer and Lawrence Saul's large-margin nearest neighbor method (2005) used triplet-style constraints, and Florian Schroff, Dmitry Kalenichenko and James Philbin's FaceNet (2015) popularized the triplet loss for deep networks.

Related

See also
  • Siamese networkSiamese network, n.: A matching machine built from twin copies of one neural network: each copy turns one of two inputs, say two signatures or two face photos, into a summary, and the two summaries are compared.
  • contrastive learningcontrastive learning, n.: A way to learn without labels by comparison: the model learns to pull together two versions of the same thing, such as two crops of one photo, and push apart unrelated things.
  • embeddingembedding, n.: A list of numbers that stands for a word, picture or other item, arranged so that similar items get similar lists. It lets a computer measure similarity of meaning as distance.
  • k-nearest neighborsk-nearest neighbors, n.: Predicting something about a new example by finding the few most similar examples already known and going with their majority answer, or their average.
  • few-shot learningfew-shot learning, n.: Getting an AI system to handle a new task or recognize a new kind of thing from just a handful of examples, the way a person can learn a new word after hearing it once or twice.
  • speaker verificationspeaker verification, n.: Checking whether a voice really belongs to the person it claims to be, as when a bank confirms your identity by the sound of your voice on the phone.

Mentioned in

contrastive learningcontrastive learning, n.: A way to learn without labels by comparison: the model learns to pull together two versions of the same thing, such as two crops of one photo, and push apart unrelated things., Siamese networkSiamese network, n.: A matching machine built from twin copies of one neural network: each copy turns one of two inputs, say two signatures or two face photos, into a summary, and the two summaries are compared.