Model Drift Detection

Occurs when the statistical relationship between the input features and the target variable changes. In a RAN context, this means the mapping from PRB utilization and CQI to actual cell congestion no longer holds. The model's predictions become systematically biased. Detection requires monitoring the prediction error distribution over time using techniques like the Page-Hinkley test or ADWIN.

A shift in the distribution of the input features themselves, even if the relationship to the target remains stable. For example, a sudden change in user traffic patterns due to a new event venue opening near a cell site. Detected via statistical distance metrics:

• Kullback-Leibler Divergence
• Population Stability Index (PSI)
• Kolmogorov-Smirnov test Comparing the training data distribution to the live inference window.

Monitors the model's output distribution directly. A significant shift in the range or frequency of predicted values—such as a load balancer suddenly predicting near-zero congestion for all cells—signals a failure. This is often the first observable symptom of an upstream problem. Threshold-based alerting on the mean and variance of predictions is a common first-line defense.

Drift is often caused not by a changing world, but by broken data pipelines. Detection systems must monitor:

• Schema violations: Missing features, type changes
• Range violations: CQI values outside 0-15
• Null rate spikes: Sudden gaps in telemetry These are not drift per se, but data quality incidents that manifest as drift. A robust system distinguishes between the two to route alerts correctly.

Drift detection is only valuable if it triggers a controlled response. Common automated actions include:

• Online retraining: Incrementally updating the model with recent data
• Full batch retraining: Triggering a CI/CD pipeline for a new model version
• Model rollback: Reverting to a previously stable checkpoint
• Human-in-the-loop: Alerting an MLOps engineer for investigation The trigger threshold must balance sensitivity against alert fatigue.

The choice of reference data against which drift is measured is critical. Common strategies:

• Fixed Window: Compare against the original training set (risks flagging legitimate seasonal changes)
• Sliding Window: Compare against the recent past (detects abrupt shifts but misses slow degradation)
• Golden Dataset: A curated, statistically representative holdout set For RAN load prediction, a weekly sliding window often captures daily periodicity while detecting true anomalies.

The fundamental phenomenon where the statistical properties of the target variable change over time, violating a model's core assumption that the future will resemble the past. In RAN load balancing, this occurs when user mobility patterns shift due to a new transit schedule or a permanent venue opening. Sudden drift happens abruptly, while incremental drift manifests as a slow, continuous degradation. Detecting concept drift is the primary objective of any model monitoring system.

A critical distinction in model monitoring. Data drift (or feature drift) refers to changes in the distribution of input features, such as a shift in the average reported Channel Quality Indicator (CQI) due to seasonal foliage. Concept drift refers to a change in the relationship between those inputs and the target prediction, such as the same CQI value now correlating with lower throughput due to a new interference source. Data drift does not always cause concept drift, but it is a leading indicator that warrants investigation.

A widely used statistical metric that quantifies how much a feature's distribution has shifted from a reference baseline, typically the training data. PSI works by binning the feature values and comparing the proportion of samples in each bin between the reference and production datasets. A PSI value below 0.1 indicates no significant shift, 0.1 to 0.25 suggests moderate drift, and above 0.25 signals significant drift requiring investigation. It is computationally efficient and easily interpretable by operations teams.

A non-symmetric measure from information theory that quantifies the difference between two probability distributions. In drift detection, it is used to calculate the information loss when the production data distribution is used to approximate the training data distribution. Unlike PSI, KL Divergence is not symmetric and can be applied to continuous distributions without arbitrary binning. It is particularly effective for detecting subtle shifts in high-dimensional telemetry data like multivariate time-series of PRB utilization and RRC connections.

A direct, performance-based approach that tracks the model's predictive accuracy in production. This requires a feedback loop where the actual outcome is eventually observed and compared against the forecast. For cell load prediction, the actual PRB utilization is recorded 15 minutes after the prediction horizon and compared to the forecast. A widening gap in metrics like Mean Absolute Percentage Error (MAPE) or Root Mean Square Error (RMSE) over a rolling window is a definitive signal of model degradation, triggering a retraining or rollback workflow.

An online drift detection algorithm that dynamically maintains a variable-length window of recent data points. ADWIN automatically grows the window when the data stream is stationary and shrinks it when a change is detected by comparing the means of two sub-windows. Its key advantage is that it requires no manual threshold tuning and provides theoretical guarantees on false positive and false negative rates. This makes it ideal for deployment in a Near-RT RIC xApp where autonomous, real-time reaction to drift is required.