Precision Recall Curve

The curve visualizes the inverse relationship between precision and recall as the classification threshold varies. A high threshold yields high precision (few false alarms) but low recall (many missed cases). Lowering the threshold increases recall at the cost of precision. The goal is to select an operating point that balances the clinical cost of false positives against the clinical cost of false negatives.

Average Precision (AP) summarizes the curve into a single scalar value. Unlike AUC-ROC, AUPRC is sensitive to class imbalance and does not reward a model for correctly classifying easy negatives. In a dataset where only 1% of patients have a condition, a baseline AUPRC equals the prevalence rate (0.01), not 0.5. This makes AUPRC a more honest metric for rare-event detection.

The PR curve is the preferred evaluation tool when the positive class is rare:

• Sepsis prediction: <5% of ICU stays
• 30-day readmission: 10-15% of discharges
• In-hospital mortality: 2-3% of admissions
• Adverse drug events: <1% of prescriptions In these scenarios, a model with 99% accuracy can be useless if it predicts 'no event' for every case. The PR curve exposes this failure.

A well-constructed PR curve is monotonically decreasing after interpolation. The no-skill baseline is a horizontal line at y = P/(P+N), representing the positive class prevalence. Any useful model must achieve an AUPRC significantly above this baseline. A curve that hugs the top-right corner indicates both high precision and high recall across thresholds.

While the ROC curve plots TPR vs. FPR and can be optimistic on imbalanced data, the PR curve ignores true negatives entirely. This is clinically appropriate: a doctor does not care how many healthy patients were correctly identified as healthy; they care about missed diagnoses and false alarms. When class imbalance exceeds 1:10, the PR curve provides a more discriminating view of model quality.

Clinicians use the curve to select an operating threshold based on decision-theoretic criteria:

• High recall priority: Screening tests where missing a case is unacceptable (e.g., HIV screening)
• High precision priority: Invasive follow-up procedures where false positives cause harm (e.g., biopsy recommendation)
• F1-max point: The threshold where the harmonic mean of precision and recall is maximized, often used as a default operating point

A graphical plot illustrating the diagnostic ability of a binary classifier as its discrimination threshold varies. It plots the True Positive Rate (Sensitivity) against the False Positive Rate (1-Specificity) . While widely used, the ROC curve can present an overly optimistic view of model performance on highly imbalanced clinical datasets where the negative class (healthy patients) vastly outnumbers the positive class (rare disease).

The harmonic mean of Precision and Recall, providing a single metric that balances both concerns. The F1 score reaches its best value at 1 (perfect precision and recall) and worst at 0. It is a crucial summary statistic derived from the precision-recall curve, often used to select an optimal probability threshold or compare models when seeking a balance between false positives and false negatives.

A single scalar value that summarizes the Precision-Recall curve by calculating the weighted mean of precisions achieved at each threshold, with the increase in recall from the previous threshold used as the weight. Average Precision is the preferred metric for evaluating models on imbalanced data because it focuses exclusively on the positive class's performance, unlike the Area Under the ROC Curve (AUC-ROC).

The process of adjusting a predictive model's output probabilities so they accurately reflect the true likelihood of an event. A model can have an excellent precision-recall curve but be poorly calibrated, meaning a predicted 90% risk of sepsis does not correspond to a 90% observed frequency. Reliability diagrams are used to assess this alignment, which is critical for clinical decision-making.

A method for evaluating the net benefit of a predictive model by quantifying the trade-off between true-positive classifications and the harm of false-positive interventions across a range of clinical threshold probabilities. It extends the concept of the precision-recall trade-off by incorporating the clinical consequences of decisions, helping determine if a model offers more benefit than a 'treat all' or 'treat none' strategy.

A condition in a dataset where the number of observations in one class is significantly lower than in others. In clinical contexts, this is the norm: a rare disease might affect 1% of a population. Standard accuracy metrics fail here, as a model predicting 'no disease' for everyone would be 99% accurate. The Precision-Recall curve is specifically designed to evaluate model performance under such skewed distributions.