Model Drift Detection

Distinguishes between two fundamental degradation patterns. Data Drift occurs when the statistical properties of the input features change (e.g., a shift in user device distribution from urban to rural). Concept Drift occurs when the relationship between the inputs and the target variable changes, even if the input data looks the same (e.g., a new interference pattern that wasn't present during training).

• Detection Method: Two-sample Kolmogorov-Smirnov tests for data drift
• Detection Method: Performance-based triggers for concept drift
• Impact: Unaddressed drift leads to suboptimal beamforming and throughput loss

A symmetric metric quantifying how much a variable's distribution has shifted from a reference baseline. In RAN applications, PSI is calculated on key performance indicators like Reference Signal Received Power (RSRP) distributions across cells.

• Formula: PSI = Σ (Actual% - Expected%) * ln(Actual% / Expected%)
• Thresholds: < 0.1 indicates no significant drift; > 0.25 signals a major shift requiring model retraining
• Use Case: Monitoring the input feature space of a Massive MIMO optimization xApp

Monitors the output of the deployed model for statistical anomalies. A sudden shift in the mean or variance of predictions—such as predicted Channel Quality Indicators (CQIs)—often signals that the model is encountering unfamiliar network states.

• Metric: Kullback-Leibler divergence between production and baseline prediction distributions
• Advantage: Captures degradation without requiring ground truth labels in real-time
• Integration: Implemented as a health-check microservice within the Non-RT RIC's AI/ML workflow

A continuous evaluation method that computes model accuracy metrics over a moving temporal window and compares them against a static holdout baseline. This detects gradual degradation caused by seasonal traffic pattern evolution.

• Window Size: Typically 1-hour to 24-hour windows for RAN traffic models
• Baseline: Frozen model performance on the original validation dataset
• Trigger: Automated retraining pipeline initiation when accuracy drops below 95% of baseline

Identifies model degradation caused by malicious actors attempting to poison the inference loop. In a RIC context, this involves detecting statistically anomalous E2 report streams designed to force incorrect resource allocation decisions.

• Technique: Robust covariance estimation to identify outliers in high-dimensional telemetry
• Defense: Isolating anomalous base stations and reverting to a safe, non-AI fallback policy
• Relevance: Critical for securing the closed-loop automation in O-RAN architectures

Uses explainability techniques like SHAP (SHapley Additive exPlanations) to monitor whether the model's decision-making rationale has changed. A shift in feature importance rankings—such as a model suddenly ignoring uplink interference metrics—indicates semantic drift.

• Method: Calculate SHAP values on a reference dataset and compare to production distributions
• Alert: Triggered when the top-3 most important features change order
• Benefit: Provides actionable diagnostics for engineers to understand why the model is failing