Federated AUC

Unlike metrics tied to a specific decision threshold (e.g., accuracy), AUC evaluates a classifier's ability to rank positive instances above negative ones across all possible thresholds. In a federated setting, this is critical because the optimal clinical decision threshold may vary by institution. The metric is computed by aggregating true positive rates (TPR) and false positive rates (FPR) from each client without sharing individual prediction scores.

• Evaluates ranking quality, not a single operating point
• Robust to class imbalance, common in rare disease detection
• Enables apples-to-apples comparison across heterogeneous sites

The federated ROC curve is constructed by securely aggregating TPR and FPR pairs from each participating institution at various threshold levels. Using Secure Multi-Party Computation (SMPC) or homomorphic encryption, the central server computes the global curve without ever seeing individual patient predictions. The AUC is then numerically integrated using the trapezoidal rule on the aggregated curve.

• Each client computes local TPR/FPR pairs independently
• Coordinates are encrypted before transmission
• Global curve is assembled from aggregated, privacy-preserving counts

A significant challenge in federated AUC computation arises when prediction score distributions differ dramatically across sites due to non-IID data. A raw average of local AUCs can be misleading. Advanced techniques use vertical federated aggregation of the global ROC curve by pooling rank-order statistics rather than averaging scalar AUC values, ensuring the metric reflects true global discrimination.

• Local score calibration differences can distort naive averaging
• Pooling global ranks preserves the integrity of the ROC curve
• Requires careful normalization or isotonic regression per client

To prevent membership inference attacks through the AUC metric itself, Differential Privacy (DP) is applied during aggregation. Noise calibrated to a privacy budget (epsilon) is added to the TPR and FPR counts before they are shared. This ensures that the final federated AUC does not leak information about whether any single patient's record was in the evaluation set.

• Gaussian or Laplacian noise added to aggregated counts
• Privacy budget must be tracked across multiple evaluation rounds
• Trade-off exists between privacy guarantee and metric fidelity

A single point estimate of AUC is insufficient for clinical decision-making. Federated AUC computation often includes federated bootstrapping, where each client generates resampled datasets locally, computes AUC on each bootstrap sample, and the distribution of these values is securely aggregated to produce a confidence interval. This quantifies the uncertainty of the performance estimate without sharing raw data.

• Each client performs stratified bootstrap resampling locally
• Distribution of AUC values is aggregated across the network
• Provides 95% confidence intervals for regulatory submissions

Federated AUC is often computed alongside a federated confusion matrix, which securely aggregates true positive, false positive, true negative, and false negative counts across institutions. While the confusion matrix provides threshold-specific metrics (precision, recall), the AUC offers a complementary, threshold-independent summary. Together, they provide a complete picture of model performance.

• Confusion matrix enables calculation of federated precision and recall
• AUC summarizes the entire ROC curve in a single scalar
• Both are foundational to federated model evaluation pipelines