Area Under the ROC Curve (AUC-ROC)

Unlike metrics such as accuracy or sensitivity that depend on a specific risk cutoff, AUC-ROC evaluates discriminative performance across all possible classification thresholds. This is critical for PRS models where the optimal threshold for clinical action is often unknown or varies by population. The ROC curve plots the true positive rate (sensitivity) against the false positive rate (1 - specificity) at every threshold, and the AUC summarizes this entire curve into a single scalar value between 0 and 1.

The AUC has a direct, intuitive interpretation: it is the probability that a randomly selected affected individual (case) receives a higher PRS than a randomly selected unaffected individual (control). An AUC of 0.50 indicates performance no better than random chance, while an AUC of 1.0 represents perfect discrimination. In practice, PRS models for complex diseases typically achieve AUCs between 0.55 and 0.75, reflecting the polygenic and multifactorial nature of these traits.

AUC-ROC is fundamentally a rank-order statistic, equivalent to the Mann-Whitney U test statistic normalized by the product of sample sizes. This means the metric depends only on the relative ordering of risk scores, not their absolute magnitudes. For PRS evaluation, this property is advantageous because raw polygenic scores are often on arbitrary scales that vary by construction method. The AUC remains invariant under any monotonic transformation of the score distribution.

While AUC-ROC measures discrimination and R² (variance explained) measures overall predictive accuracy, the two are mathematically linked under the liability threshold model. For a disease with a given population prevalence, the AUC can be converted to an approximate R² on the liability scale. This connection allows researchers to benchmark PRS performance across studies that report different metrics and to estimate the potential clinical utility of a score given the underlying heritability of the trait.

AUC-ROC can present an overly optimistic view of performance when cases and controls are highly imbalanced, which is common in population-based PRS studies of rare diseases. Because the false positive rate uses the large number of true negatives in its denominator, a model can achieve a high AUC even with poor positive predictive value. In these scenarios, AUC-PR (Precision-Recall AUC) is often a more informative complementary metric, as it focuses on the model's ability to identify the minority class correctly.

The DeLong test provides a non-parametric method for statistically comparing the AUCs of two correlated ROC curves, such as when evaluating whether adding a novel set of variants significantly improves a baseline PRS. This test accounts for the correlation induced by evaluating both models on the same set of individuals. Reporting confidence intervals for AUCs, typically via bootstrap resampling, is essential for conveying the uncertainty in discriminative ability, especially in small validation cohorts.

The graphical plot that visualizes the trade-off between the true positive rate (sensitivity) and the false positive rate (1 - specificity) across all possible classification thresholds. The curve is generated by varying the decision threshold for assigning case status and plotting the resulting TPR against FPR. A model with no discriminative ability produces a diagonal line from (0,0) to (1,1), while a perfect classifier hugs the upper-left corner. The ROC curve is threshold-independent, making it the foundational visualization from which the AUC-ROC metric is derived.

An alternative to the ROC curve that plots precision (positive predictive value) against recall (sensitivity) across thresholds. PR curves are particularly informative for evaluating PRS models on imbalanced datasets where controls vastly outnumber cases—a common scenario in population biobanks. While AUC-ROC can present an overly optimistic picture of performance under severe class imbalance, the area under the PR curve provides a more stringent assessment by penalizing models that generate many false positives among the top-ranked individuals.

A generalization of the AUC-ROC for censored time-to-event data, commonly used in survival analysis. The C-index estimates the probability that, for a randomly selected pair of individuals, the model correctly predicts which one experiences the event first. In the absence of censoring, the C-index is mathematically equivalent to the AUC-ROC. For PRS models predicting age-of-onset outcomes, the C-index accounts for individuals who drop out or remain event-free at the end of the study period, providing a more clinically relevant measure of discriminative ability.

A metric that quantifies the clinical utility of adding a PRS to an existing risk model by measuring how many individuals are correctly reassigned to more appropriate risk categories. Unlike AUC-ROC, which only assesses ranking ability, NRI evaluates whether the score meaningfully changes clinical decision-making. It partitions reclassification into:

• Event NRI: Improvement among cases correctly moved to higher risk tiers
• Non-event NRI: Improvement among controls correctly moved to lower risk tiers A significant NRI demonstrates that the PRS adds predictive value beyond traditional risk factors, even when AUC-ROC improvements appear modest.

A strictly proper scoring rule that measures both discrimination and calibration simultaneously by calculating the mean squared difference between predicted probabilities and actual binary outcomes. The Brier score ranges from 0 (perfect prediction) to 0.25 (uninformative model for a 50% base rate). For PRS evaluation, the Brier score complements AUC-ROC by penalizing models that achieve good ranking but produce miscalibrated probability estimates. Decomposing the Brier score into refinement and calibration components reveals whether poor performance stems from weak discrimination or systematic probability bias.

A statistical goodness-of-fit test that assesses calibration by comparing observed and expected event rates across deciles of predicted risk. Individuals are binned into groups based on their predicted probabilities, and a chi-squared statistic evaluates whether deviations between observed and expected counts are larger than expected by chance. A significant p-value indicates poor calibration. For PRS models used in clinical settings, passing the Hosmer-Lemeshow test is essential: a model with high AUC-ROC but poor calibration may systematically overestimate or underestimate absolute disease risk, undermining its utility for patient counseling.