Federated Population Stability Index (PSI)

The continuous monitoring of a deployed federated model's performance to identify degradation over time caused by concept drift or data drift in distributed input streams. Unlike centralized monitoring, drift detection must operate without centralizing live inference data. Techniques include federated statistical tests that compare feature distributions between a reference window and a monitoring window across nodes, triggering alerts when the Federated PSI exceeds a predefined threshold. This is critical for maintaining clinical model safety in production.

A change in the statistical relationship between input features and the target variable over time, which can cause a previously accurate federated model to become unreliable. For example, a diagnostic model trained on pre-pandemic chest X-rays may exhibit concept drift when applied to post-pandemic populations with different disease presentations. In federated settings, concept drift must be detected without pooling patient data, often using federated performance metrics like Federated AUC or Federated F1-Score tracked over time at each institution.

A privacy-preserving method for aggregating classification performance metrics across multiple institutions by securely summing true positive, false positive, true negative, and false negative counts. Each institution computes its local confusion matrix on private validation data, then shares only the aggregate cell counts via Secure Aggregation (SecAgg). The central server reconstructs the global confusion matrix to derive metrics like precision, recall, and specificity without ever seeing individual patient predictions.

The process of auditing a federated model for unfair performance disparities across protected groups by securely computing fairness metrics on distributed validation data. Key metrics include:

• Demographic Parity: Equal positive prediction rates across groups
• Equalized Odds: Equal true positive and false positive rates across groups
• Federated computation ensures that sensitive attributes and outcomes remain local while only aggregated fairness statistics are shared for regulatory compliance.

Methods such as Monte Carlo Dropout or Deep Ensembles applied in a federated setting to estimate the confidence and epistemic uncertainty of a model's predictions. This is critical for high-stakes clinical decision support, where a model must flag ambiguous cases for human review. Expected Calibration Error (ECE) is often computed in a federated fashion to ensure probability estimates from a decentralized model are reliable and well-calibrated across all participating institutions.

The task of identifying inference-time inputs that differ fundamentally from the federated training data distribution. An OOD detector flags potentially unsafe predictions for clinical review, preventing a model from making confident but erroneous decisions on novel patient populations or imaging protocols. In federated settings, OOD detection boundaries must be established collaboratively, often using federated conformal prediction or energy-based scores aggregated across nodes without sharing raw patient data.