decision threshold

The value of a classifier's score or predicted probability above which an example is assigned to the positive class. Many models output a continuous score, and the default threshold of 0.5 on a probability is optimal only if the probabilities are calibrated and both kinds of error cost the same. Raising the threshold generally increases precisionprecision, n.: Of all the items a model flagged as positive, the share that really were positive; a spam filter with high precision rarely sends genuine emails to the spam folder. and specificityspecificity, n.: Of all the cases that really were negative, the share a test correctly cleared; a highly specific test rarely raises false alarms on healthy people. and lowers recallrecall, n.: Of all the items that really were positive, the share the model managed to find; a cancer screen with high recall misses very few people who actually have cancer.; lowering it does the reverse. Each threshold yields one confusion matrixconfusion matrix, n.: A table that counts a model's right and wrong answers by type, showing for each true category how often each category was predicted, so you can see exactly which things it mixes up., and sweeping it traces the ROC curveROC curve, n.: A graph showing how a detector's hit rate and false-alarm rate change together as it is made more or less trigger-happy; the closer the curve hugs the top-left corner, the better. and the precision–recall curveprecision–recall curve, n.: A graph showing how a model's share of correct alarms and its share of cases caught trade off as it is made more or less cautious; especially useful when positive cases are rare.. Choosing the operating point is a decision rather than a learning problem: with calibrated probabilities, the threshold that minimizes expected cost is the cost of a false positive divided by the sum of the costs of a false positive and a false negative. In practice thresholds are tuned on a validation setvalidation set, n.: A set of examples kept out of training and used during development to compare model versions and settings, like a practice exam taken before the real one. to meet a constraint, such as 95% recall or at most 1% of transactions flagged, to maximize a metric such as the F1 scoreF1 score, n.: A single score that combines how many of a model's alarms are right and how many real cases it catches, and stays low unless both are good., or separately per group to satisfy fairness criteria. They should be re-checked when class prevalence or score distributions drift.

We lowered the decision threshold from 0.5 to 0.2 because a missed tumor costs far more than an extra biopsy.