Concept Drift

An abrupt change in the underlying data distribution triggered by an unforeseen external event. The model's decision boundary becomes instantly obsolete.

• Mechanism: A shock to the system, such as a new competitor entering the market, a global pandemic shifting consumer behavior, or a regulatory change redefining a financial instrument.
• Example: A credit risk model trained on pre-2020 data failing catastrophically during the COVID-19 economic shutdown because the relationship between employment status and repayment ability changed overnight.
• Detection: Typically caught by sharp spikes in error rates or data distribution distance metrics like Population Stability Index (PSI).

A gradual, continuous evolution of the target concept over time. The model's accuracy slowly erodes as the world changes around it.

• Mechanism: Slow-moving shifts in user preferences, inflation gradually changing purchasing power, or mechanical wear-and-tear in a sensor-monitored industrial asset.
• Example: A recommendation engine for an e-commerce platform where fashion trends slowly evolve seasonally. Last year's "high relevance" items become this year's "discount rack" items.
• Detection: Requires sequential analysis techniques like the Drift Detection Method (DDM) or monitoring the KL-divergence between prediction confidence distributions over rolling windows.

Cyclical patterns where the target concept oscillates between two or more distinct states. The model must recognize the current context to apply the correct logic.

• Mechanism: Predictable temporal patterns such as day/night cycles, weekday vs. weekend behavior, or seasonal weather patterns affecting demand.
• Example: A fraud detection model for a ride-sharing platform. The legitimate behavior patterns at 2 AM on a Saturday are statistically distinct from those at 9 AM on a Tuesday. A single static model will flag normal weekend behavior as anomalous.
• Detection: Often addressed by building context-aware features (time of day, season) or maintaining an ensemble of specialized sub-models rather than treating it as a drift problem.

A shift in the distribution of the input features (P(X)) that does not necessarily change the decision boundary (P(Y|X)). This is often confused with real concept drift.

• Mechanism: The input data changes, but the underlying relationship between the inputs and the target remains stable. The model's training data simply didn't cover this region of the feature space.
• Example: A loan application system suddenly receiving applications from a new demographic segment after a marketing campaign. The model is uncertain not because the rules of creditworthiness changed, but because it has no data on this population.
• Detection: Distinguish from real drift by checking if the model's confidence drops (uncertainty) rather than its accuracy dropping (error). This is a data coverage problem, not a concept problem.

A change in the statistical properties of one or more input variables, which may or may not cascade into prediction errors. This is a leading indicator of potential concept drift.

• Mechanism: A sensor degrades and reports values with a new bias, a data pipeline bug introduces a unit conversion error, or a categorical variable sees a new category appear.
• Example: A predictive maintenance model for a turbine where the temperature sensor slowly decalibrates, reporting values 5°C lower than reality. The model's input distribution shifts, eventually causing it to miss overheating warnings.
• Detection: Univariate statistical tests (Kolmogorov-Smirnov, Chi-squared) on each feature distribution between the training baseline and the current production window.

A direct change in the prior probability of the target variable P(Y), independent of the input features. The base rate of the event being predicted changes.

• Mechanism: The fundamental frequency of the outcome shifts. This is distinct from concept drift because the relationship P(Y|X) might be stable, but the overall probability P(Y) is not.
• Example: A click-through rate (CTR) prediction model during a major holiday sale. The base rate of clicks skyrockets not because the features are more predictive, but because user intent is universally higher.
• Detection: Monitoring the mean of the predicted probability against the actual observed frequency (calibration drift). A perfectly calibrated model will see its predictions systematically underestimate the true rate.

The automated process of monitoring for shifts in the distribution of input features (P(X)) that can degrade model performance. Unlike concept drift, data drift only examines input data, not the relationship to the target.

• Statistical tests: Kolmogorov-Smirnov, Chi-squared, Jensen-Shannon divergence
• Population Stability Index (PSI) is the most common industry metric
• Triggers alerts when feature distributions deviate from a baseline reference window
• Often a leading indicator that concept drift may follow

A specific type of data drift where the distribution of independent variables changes, but the conditional distribution of the target given the inputs—P(Y|X)—remains stable. This is distinct from true concept drift.

• Common in medical imaging when scanner hardware is upgraded
• Can often be corrected through importance weighting or domain adaptation
• Does not require retraining if properly handled via statistical correction
• Often confused with concept drift but requires different remediation

A drift pattern where the class distribution P(Y) changes while the class-conditional feature distributions remain constant. Common in fraud detection when attack rates spike seasonally.

• Also called label shift or target shift
• Detectable without ground truth labels using Black Box Shift Estimation (BBSE)
• Corrected via adjusting classification thresholds or recalibrating output probabilities
• Fundamentally different from concept drift because P(X|Y) stays fixed

A predefined threshold or condition that automatically initiates a model retraining pipeline when drift exceeds acceptable bounds. This operationalizes drift detection into automated remediation.

• Performance-based triggers: Retrain when F1 or accuracy drops below a threshold
• Distribution-based triggers: Retrain when PSI or drift test p-value crosses a limit
• Time-based triggers: Scheduled retraining as a fallback cadence
• Requires a champion-challenger deployment pattern to validate new models safely

Techniques that allow a model to update incrementally as new data arrives, without full batch retraining. Critical for high-velocity environments where concept drift is continuous.

• Stochastic gradient descent with a moving window of recent samples
• Adaptive learning rate methods like AdaGrad or Adam adjust step sizes dynamically
• Ensemble weighting: Dynamically re-weight an ensemble of models based on recent performance
• Trade-off: rapid adaptation vs. risk of catastrophic forgetting on older patterns

The latency between prediction and label availability that complicates drift detection. In fraud or credit risk, true outcomes may take weeks or months to materialize.

• Creates a verification gap where drift can occur undetected
• Mitigated by proxy metrics that correlate with eventual outcomes
• Replay testing simulates drift scenarios on historical data with known labels
• Requires careful design of monitoring windows to avoid false alarms during the delay period