hinge loss

A loss functionloss function, n.: A score that measures how wrong a model's predictions are. Training means adjusting the model, step by step, to push this number down. for classification with labels y ∈ {−1, +1} and a real-valued score f(x), defined as max(0, 1 − y·f(x)). It is zero for examples classified correctly with a marginmargin, n.: How far, and on which side, an example lies from a classifier's dividing line. A wide gap between the line and the nearest examples suggests the classifier will handle new data well. of at least 1 and grows linearly as the margin shrinks or turns negative, so easy examples contribute nothing — which is why only support vectors shape a support vector machinesupport vector machine, n.: A classic method that separates two groups of examples with the widest possible gap — like laying the broadest possible road between them. A mathematical shortcut lets it draw curved boundaries too., whose training objective is the hinge loss plus an L2 penalty. The loss is convex, allowing efficient optimization, but not differentiable at the hinge point, so subgradient methods or smoothed variants are used. It is a convex upper bound on the 0–1 classification error but, unlike log losslog loss, n.: A score for probability predictions that punishes confident mistakes severely: saying '99% sure' and being wrong costs far more than saying '60% sure' and being wrong. Lower is better., does not yield probability estimates. Multiclass versions (Weston–Watkins, Crammer–Singer) penalize the gap between the true class's score and the others'; the squared hinge penalizes violations quadratically.

With hinge loss the model stops pushing on examples once they clear the margin, unlike with cross-entropy.