Contextual Drift

A change in the distribution of the input features P(X) while the conditional distribution of the target given the features P(Y|X) remains stable. In a retail bandit, this occurs when the demographic mix of users visiting a site changes seasonally, but the purchasing behavior of a given demographic remains constant. The model sees data from a new region of the feature space where it has high uncertainty, triggering a natural need for exploration.

A change in the fundamental relationship between the context and the reward, meaning P(Y|X) itself changes over time. The same user context now yields a different expected outcome. For example, a product recommendation that historically led to a purchase may stop converting due to a viral social media trend altering consumer preferences. This is the most damaging form of drift because it invalidates the learned policy, requiring aggressive model retraining or forgetting mechanisms.

A change in the marginal distribution of the target variable P(Y) without a change in the input features or the decision boundary. In a click-through rate prediction model, this manifests as a sudden global increase or decrease in click rates across all users due to a site-wide user interface redesign. While the ranking of items might remain correct, the absolute predicted probabilities become miscalibrated, breaking downstream logic that relies on calibrated scores.

Drift is categorized by its temporal velocity. Sudden drift is an abrupt, discontinuous change often caused by an external shock like a competitor launch or a global pandemic. Incremental drift is a slow, gradual evolution of user behavior, such as the multi-year shift from desktop to mobile shopping. Detection systems must be tuned differently for each: sudden drift requires immediate alerts and rollback, while incremental drift is managed by continuous online learning and adaptive windowing.

A deceptive form of drift caused not by a changing world, but by a change in the data collection or logging policy. If the exploration strategy of a bandit is modified, the distribution of logged data shifts, creating the illusion of concept drift. Off-policy evaluation metrics will diverge from online performance. Distinguishing virtual drift from true environmental drift is critical to avoid retraining a model on biased data and amplifying the error.

The standard engineering approach to detecting drift involves using a two-sample statistical test on reference and production data windows. The Kolmogorov-Smirnov (KS) test is used for continuous features, while Chi-squared tests are used for categorical features. A more practical method for high-dimensional data is to monitor the performance of a domain classifier trained to distinguish between the reference and current data; high classifier accuracy indicates significant drift.

A bandit problem where the underlying reward distributions change over time, violating the static assumption of standard algorithms. In a non-stationary environment, historical data becomes obsolete, requiring the agent to continuously adapt. Strategies include discounting old observations with a sliding window or using a forgetting factor, and employing algorithms specifically designed to track a moving optimum rather than converging to a fixed one.

A measure of how recently a deployed model has been updated to reflect the latest user behavior. In the presence of contextual drift, model freshness is a critical operational metric. A stale model trained on outdated feature distributions will make suboptimal decisions. Monitoring freshness involves tracking the age of the model artifact and the distributional divergence between training and serving data to trigger automated retraining pipelines.

The continuous updating of machine learning models in production to adapt to shifting consumer behavior. Unlike batch retraining, online retraining ingests streaming data to incrementally adjust model parameters. This is the primary defense against contextual drift, ensuring that the model's learned relationships between context features and rewards remain valid as the environment evolves.

A critical distinction in non-stationary environments. Data drift refers to a change in the input feature distribution P(X), while concept drift refers to a change in the conditional relationship P(Y|X) between features and the target. Contextual drift can encompass both: a new user segment (data drift) or a shift in what a specific context means for a reward (concept drift).

The optimization objective in bandit problems that seeks to minimize the difference between the cumulative reward of the optimal policy and the reward accumulated by the learning algorithm. Under contextual drift, regret minimization becomes more challenging because the optimal policy itself is a moving target. Algorithms must balance rapid adaptation against the noise of reacting to transient fluctuations.

A statistical method for estimating the performance of a new policy using historical data collected under a different logging policy. In drifting environments, off-policy evaluation is complicated by the fact that historical data may no longer represent the current context distribution. Techniques like Inverse Propensity Scoring (IPS) must be augmented with time-decay weights to account for the diminishing relevance of older logged interactions.