Model Drift

The fundamental relationship between the input features (X) and the target variable (y) changes. The model's learned mapping becomes obsolete because the underlying phenomenon it predicts has shifted.

• Example: A spam filter trained on 'Nigerian prince' emails fails when attackers switch to legitimate-sounding business compromise language.
• Detection: Requires access to ground truth labels, often delayed. Monitor prediction error rates over time.
• Remediation: Full retraining on recent data or online learning with a forgetting factor.

The statistical distribution of the input features P(X) changes, but the conditional distribution P(y|X) remains stable. The model sees data unlike its training set.

• Example: A credit scoring model trained pre-pandemic encounters a population with suddenly different income distributions and employment types.
• Detection: Population Stability Index (PSI), Kullback-Leibler divergence, or two-sample Kolmogorov-Smirnov tests on feature distributions.
• Remediation: Feature engineering updates, retraining with representative samples, or calibration layer adjustments.

The prior probability of the target variable P(y) shifts independently of the input features. The base rate of the phenomenon being predicted changes.

• Example: A fraud detection model experiences a sudden spike in the fraud rate from 0.1% to 5% during a coordinated attack.
• Detection: Monitoring the mean prediction score over time. A shift without a corresponding feature distribution change signals label drift.
• Remediation: Threshold tuning, class re-weighting, or recalibration using isotonic regression or Platt scaling.

Individual input features exhibit distributional shifts that may or may not aggregate to full data drift. This granular view identifies which specific signals are decaying.

• Example: A sensor in an industrial IoT system slowly degrades, introducing a systematic bias in temperature readings while pressure features remain stable.
• Detection: Per-feature drift metrics using Wasserstein distance or Jensen-Shannon divergence, visualized in drift dashboards.
• Remediation: Sensor recalibration, feature removal, or engineering invariant representations resistant to specific sensor degradation.

The output distribution of the model's predictions P(ŷ) changes, serving as an aggregate symptom of any combination of concept, data, or label drift.

• Example: A churn prediction model suddenly classifies 80% of users as high-risk, up from a historical 15%, triggering a flood of false-positive retention offers.
• Detection: Monitoring the prediction distribution histogram, mean confidence, and rejection rate over rolling windows.
• Remediation: Serves as an early warning trigger. Root cause analysis must follow to determine if concept, data, or label drift is the source.

Drift is not always monotonic. It exhibits temporal structures that dictate the appropriate monitoring and response cadence.

• Sudden Drift: An abrupt shift caused by an external shock (e.g., a regulatory change). Requires immediate model rollback or emergency retraining.
• Gradual Drift: A slow, continuous shift due to evolving user behavior. Handled by scheduled retraining pipelines.
• Recurring Drift: Seasonal or cyclical patterns (e.g., holiday shopping behavior). Addressed with time-aware features or ensemble models.
• Incremental Drift: A permanent, stepwise change. Requires a model architecture update, not just retraining.

A change in the statistical distribution of input features (P(X)) between training and production environments. This is a leading indicator of potential model decay.

• Detection: Population Stability Index (PSI), Kolmogorov-Smirnov tests
• Example: A credit model trained on pre-pandemic income data encounters a production distribution with significantly lower average incomes
• Remediation: Does not always require retraining; may indicate a broken upstream data pipeline

A fundamental change in the relationship between inputs and the target variable (P(Y|X)). The same input now maps to a different correct output, rendering the learned decision boundary obsolete.

• Detection: Requires ground truth labels; monitors predictive accuracy degradation over time
• Example: A fraud model where fraudsters adopt new tactics, changing the pattern that defines 'fraudulent' behavior for the same transaction features
• Remediation: Mandates model retraining with fresh labeled data

A specific type of data drift where the distribution of independent variables changes, but the conditional distribution of the target given the features remains stable.

• Assumption: P(Y|X) is unchanged, so the model's logic is still valid for the regions of input space it covers
• Risk: The model may encounter regions of feature space with no training support, leading to high-variance extrapolations
• Example: A user demographic shift in a recommendation system where the underlying preference logic for each demographic remains constant

A change in the distribution of the target variable itself (P(Y)), independent of the input features. The base rate of the outcome class changes in production.

• Detection: Monitors the marginal distribution of predictions against expected class frequencies
• Example: A disease screening model deployed in a general population setting where disease prevalence is much lower than in the clinical trial training data
• Impact: Directly distorts precision and recall metrics without any change to the model's underlying discriminative power

A granular sub-category of data drift focusing on the distributional shift of a single input feature rather than the joint distribution. Critical for isolating the root cause of model degradation.

• Monitoring: Univariate statistical tests applied per feature in the serving pipeline
• Example: A sensor degradation causing a specific temperature reading to consistently report lower values than during training
• Action: Triggers targeted feature engineering or sensor recalibration rather than full model retraining

A change in the distribution of the model's output scores over time. This is a lagging indicator that synthesizes the effects of all upstream data and concept shifts.

• Detection: Monitors the distribution of predicted probabilities or scores without requiring ground truth labels
• Example: A churn model suddenly assigning high churn probability to 80% of the customer base, indicating a likely upstream data quality issue
• Utility: Serves as a real-time operational alarm before ground truth labels become available for accuracy measurement