Concept Drift

Concept drift occurs when the posterior distribution P(Y|X) changes between the training phase and the operational phase. In a federated context, this means the relationship between clinical inputs (X) and diagnostic targets (Y) has shifted across one or more participating institutions. This is distinct from data drift (covariate shift), where only the input distribution P(X) changes but the decision boundary remains valid. True concept drift invalidates the model's learned mapping, requiring immediate remediation.

Not all detected performance degradation requires model retraining. Virtual concept drift occurs when the underlying data distribution changes but the existing decision boundary remains optimal—no action is needed. Real concept drift signifies a genuine change in the statistical relationship, demanding model adaptation. Federated monitoring systems must distinguish between these two to avoid unnecessary, costly retraining cycles across the network.

In healthcare federated learning, concept drift often manifests through non-stationary clinical environments:

• Pandemic Shifts: The presentation of respiratory illness changed fundamentally during COVID-19, breaking models trained on pre-pandemic data.
• Equipment Recalibration: A hospital upgrading its MRI machines alters the feature space, changing the relationship between pixel intensities and pathology.
• Treatment Evolution: New clinical guidelines that redefine what constitutes a positive diagnosis directly alter the target variable Y.

Federated drift detection relies on monitoring summary statistics without centralizing patient data. Common approaches include:

• Federated Population Stability Index (PSI): Measures distributional shifts in input features across institutions.
• Federated Expected Calibration Error (ECE): Tracks the divergence between predicted confidence and observed accuracy on local nodes.
• Sequential Analysis: The Page-Hinkley test or ADWIN algorithm can be applied to streaming federated performance metrics to detect abrupt change points with statistical rigor.

The temporal pattern of drift dictates the remediation strategy:

• Sudden (Abrupt) Drift: An instantaneous change, such as a new diagnostic coding standard going into effect. Requires immediate model rollback or emergency retraining.
• Incremental (Gradual) Drift: A slow, continuous evolution, like the changing demographics of a patient population. Can be addressed with scheduled, periodic federated retraining.
• Recurring Drift: Cyclical patterns, such as seasonal influenza variations, where historical model states become relevant again. Managed with model versioning and context-aware routing.

Once real concept drift is confirmed, federated systems can adapt through:

• Triggered Federated Retraining: Automatically initiating a new global training round using the most recent local data.
• Sliding Window Training: Weighting recent patient encounters more heavily than historical data during local optimization.
• Model Rollback: Reverting to a previously validated model version that performs better on the current data distribution, a strategy effective for recurring drift.
• Ensemble Adaptation: Dynamically adjusting the weighting of an ensemble of models based on each member's recent performance on local validation streams.

The continuous, privacy-preserving monitoring of a deployed federated model's predictive performance over time. Instead of centralizing live inference data, drift detection algorithms compute statistical divergence metrics—such as Population Stability Index (PSI) or Kolmogorov-Smirnov tests—locally at each institution. Only aggregated alert signals are shared with the central server. This architecture enables early warning systems for model degradation caused by concept drift or data drift without violating patient data locality constraints.

A decentralized metric for quantifying distributional shifts in a model's input features between a reference baseline period and a current monitoring window. Each client computes PSI locally by binning feature values and measuring the divergence using the formula: PSI = Σ (Actual% - Expected%) × ln(Actual% / Expected%). The resulting scores are securely aggregated to produce a global drift indicator. A PSI value > 0.25 typically signals significant feature drift requiring investigation, while values < 0.1 indicate stable distributions.

The task of identifying inference-time inputs that differ fundamentally from the federated training distribution. OOD detection methods—including Mahalanobis distance-based scoring, energy-based models, and density estimation—are deployed locally at each institution. When a patient case is flagged as OOD, the model can defer to human clinical review rather than producing a potentially unreliable prediction. This is critical for detecting concept drift where the relationship between symptoms and diagnoses shifts due to emerging disease variants or new treatment protocols.

Methods for estimating the confidence and epistemic uncertainty of a federated model's predictions. Techniques such as Monte Carlo Dropout and Deep Ensembles are applied in a decentralized manner to produce prediction intervals rather than point estimates. High uncertainty can serve as a proxy signal for concept drift, indicating that the model is encountering input patterns outside its training distribution. In clinical settings, calibrated uncertainty enables risk-stratified decision-making where low-confidence predictions trigger mandatory specialist review.

A metric measuring the discrepancy between a model's predicted confidence and its empirical accuracy, computed in a federated fashion. Predictions are binned by confidence level, and the weighted average of |accuracy - confidence| across bins yields the ECE score. A well-calibrated model with ECE < 0.05 produces reliable probability estimates. When concept drift occurs, calibration often degrades before accuracy drops, making federated ECE monitoring an effective early warning indicator for model staleness in distributed healthcare deployments.

A distribution-free framework for generating prediction sets with a guaranteed marginal coverage probability (e.g., 95%). Unlike Bayesian methods, conformal prediction makes no assumptions about the underlying data distribution, making it robust to the distributional shifts characteristic of concept drift. In a federated setting, each client computes nonconformity scores on a local calibration set, and these scores are aggregated to construct globally valid prediction intervals. This provides rigorous uncertainty guarantees even as clinical data distributions evolve.