Skip to content
Last Dictionary
Plane
Machine LearningEvaluation & Benchmarks

Matthews correlation coefficient

nounAlso: MCC, phi coefficient, φ coefficient

In plain wordsA single score from −1 to +1 for a yes-or-no classifier that uses all four kinds of right and wrong answers, so it stays honest even when one answer is far more common than the other.

  1. A summary measure of binary classification quality computed from all four cells of the confusion matrixconfusion matrix, n.: A table that counts a model's right and wrong answers by type, showing for each true category how often each category was predicted, so you can see exactly which things it mixes up.. Numerically it equals the Pearson correlationcorrelation, n.: A measure of how closely two quantities move together, from −1 (one rises as the other falls) through 0 (no straight-line link) to +1 (they rise together). It does not show that one causes the other. (the phi coefficient) between the predicted and actual labels, so +1 means every case is classified correctly, 0 means the predictions carry no information about the truth, and −1 means they are exactly inverted. Unlike accuracyaccuracy, n.: The share of a model's answers that are right, such as 95 out of 100. Simple and intuitive, but misleading when one kind of answer is far more common than the others., it is not inflated by class imbalanceclass imbalance, n.: When one category is far rarer than the others in the data — say, one fraud case per thousand transactions — so a model can look accurate while missing nearly every rare case that matters., and unlike the F1 scoreF1 score, n.: A single score that combines how many of a model's alarms are right and how many real cases it catches, and stays low unless both are good. it treats the positive and negative classes symmetrically and takes true negatives into account, so a high value requires good performance on both classes. A multiclass generalization, Jan Gorodkin's Rk statistic, exists. Like other threshold-based metrics, MCC describes a single decision thresholddecision threshold, n.: The cut-off score above which a model's prediction counts as 'yes'; moving it trades missed cases against false alarms, like setting how sensitive a smoke alarm should be. and ignores probability calibrationcalibration, n.: How well a model's stated confidence matches reality: when a well-calibrated system says it is 80% sure, it turns out to be right about 80% of the time.. It is undefined when any row or column of the confusion matrix sums to zero, and is then conventionally set to 0. Long standard in bioinformatics, it is increasingly recommended for imbalanced classification generally.

    Accuracy was 97% on the imbalanced test set, but an MCC of 0.21 showed the model barely beat chance.

MCC=TP⋅TN−FP⋅FN(TP+FP)(TP+FN)(TN+FP)(TN+FN)\mathrm{MCC} = \frac{TP \cdot TN - FP \cdot FN}{\sqrt{(TP+FP)(TP+FN)(TN+FP)(TN+FN)}}

Origin

Introduced by the biochemist Brian W. Matthews in 1975 to compare predicted and observed protein secondary structure; mathematically it is the phi coefficient for 2 × 2 tables, associated with Karl Pearson (1904) and George Udny Yule (1912). Davide Chicco and Giuseppe Jurman's 2020 paper argued for preferring it to F1 and accuracy in binary classification.

Related

See also
  • confusion matrixconfusion matrix, n.: A table that counts a model's right and wrong answers by type, showing for each true category how often each category was predicted, so you can see exactly which things it mixes up.
  • F1 scoreF1 score, n.: A single score that combines how many of a model's alarms are right and how many real cases it catches, and stays low unless both are good.
  • accuracyaccuracy, n.: The share of a model's answers that are right, such as 95 out of 100. Simple and intuitive, but misleading when one kind of answer is far more common than the others.
  • class imbalanceclass imbalance, n.: When one category is far rarer than the others in the data — say, one fraud case per thousand transactions — so a model can look accurate while missing nearly every rare case that matters.
  • correlationcorrelation, n.: A measure of how closely two quantities move together, from −1 (one rises as the other falls) through 0 (no straight-line link) to +1 (they rise together). It does not show that one causes the other.
  • decision thresholddecision threshold, n.: The cut-off score above which a model's prediction counts as 'yes'; moving it trades missed cases against false alarms, like setting how sensitive a smoke alarm should be.

Sources

  1. Matthews, B. W. (1975). Comparison of the Predicted and Observed Secondary Structure of T4 Phage Lysozyme. Biochimica et Biophysica Acta – Protein Structure 405(2).
  2. Chicco, D. & Jurman, G. (2020). The Advantages of the Matthews Correlation Coefficient (MCC) over F1 Score and Accuracy in Binary Classification Evaluation. BMC Genomics 21.

Mentioned in

confusion matrixconfusion matrix, n.: A table that counts a model's right and wrong answers by type, showing for each true category how often each category was predicted, so you can see exactly which things it mixes up., F1 scoreF1 score, n.: A single score that combines how many of a model's alarms are right and how many real cases it catches, and stays low unless both are good.