Confusion Matrix

Every prediction falls into one of four mutually exclusive categories:

• True Positive (TP): Model correctly predicts the positive class (e.g., disease present).
• True Negative (TN): Model correctly predicts the negative class (e.g., disease absent).
• False Positive (FP): A Type I Error; model predicts positive, but the ground truth is negative (false alarm).
• False Negative (FN): A Type II Error; model predicts negative, but the ground truth is positive (missed diagnosis). The sum of all four cells equals the total number of predictions.

Sensitivity, or the True Positive Rate (TPR), quantifies a model's ability to avoid false negatives. It is calculated directly from the matrix columns:

• Formula: TP / (TP + FN)
• Clinical Context: A highly sensitive test is ideal for screening where missing a disease is catastrophic. If sensitivity is 99%, only 1% of diseased patients are missed.
• Trade-off: Maximizing sensitivity often increases false positives, lowering specificity.

Specificity, or the True Negative Rate (TNR), measures a model's ability to correctly identify healthy patients and avoid false positives.

• Formula: TN / (TN + FP)
• Clinical Context: High specificity is crucial for confirmatory tests to prevent unnecessary invasive procedures. A specificity of 95% means only 5% of healthy patients receive a false alarm.
• Independence: Unlike predictive values, specificity is mathematically independent of disease prevalence in the study population.

Precision answers: 'Given a positive prediction, what is the probability the patient actually has the disease?' It is heavily influenced by disease prevalence.

• Formula: TP / (TP + FP)
• Clinical Utility: A low precision indicates a high number of false alarms, leading to clinician alert fatigue.
• Prevalence Paradox: A test with 99% sensitivity and 99% specificity can still have a low PPV if the disease is extremely rare, as the raw number of false positives can dwarf true positives.

The F1 Score provides a single metric balancing precision and recall, useful when classes are imbalanced.

• Formula: 2 * (Precision * Recall) / (Precision + Recall)
• Mechanism: It uses the harmonic mean, which penalizes extreme divergence between precision and recall more heavily than the arithmetic mean.
• Application: Preferred over raw accuracy when the cost of false negatives and false positives are both significant, but you need a single optimization target.

Overall accuracy (TP + TN) / Total can be dangerously misleading in medical imaging where disease prevalence is low.

• Example: A model that predicts 'no cancer' for every scan in a dataset with 1% cancer prevalence achieves 99% accuracy but 0% sensitivity.
• Best Practice: Never rely solely on accuracy for diagnostic validation. Always decompose the confusion matrix to report sensitivity and specificity alongside the prevalence rate.