Covariate Shift

Compare the distributions of training and production data using two-sample tests. Kolmogorov-Smirnov (KS) test measures the maximum distance between empirical cumulative distribution functions, while Maximum Mean Discrepancy (MMD) operates in reproducing kernel Hilbert space for higher-dimensional feature comparisons. A significant p-value indicates a shift has occurred.

• KS test: sensitive to location and shape changes
• MMD: effective for high-dimensional embeddings
• Chi-squared: suitable for categorical or binned features

Train a binary classifier to distinguish between samples from the source (training) domain and the target (production) domain. If the classifier achieves accuracy significantly above chance, the input distributions differ. The classifier's confidence scores can serve as a continuous drift metric, and its learned weights can identify which features contribute most to the shift.

• Use logistic regression or a shallow neural network
• Monitor AUC over time as a drift indicator
• Feature importance reveals root cause

Correct for covariate shift by assigning weights to training samples based on the density ratio p_target(x) / p_source(x). Samples that are more representative of the target distribution receive higher weight during loss computation. This avoids full retraining and is particularly useful when target labels are scarce or expensive to obtain.

• Estimate density ratio via kernel mean matching
• Apply weights in the loss function: weighted cross-entropy
• Effective when P(y|x) remains unchanged

Align source and target feature distributions using domain adaptation techniques. CORAL (Correlation Alignment) matches second-order statistics by whitening and re-coloring feature covariances. Adversarial domain adaptation uses a gradient reversal layer to learn domain-invariant representations, forcing the feature extractor to produce embeddings indistinguishable between domains.

• CORAL: simple, closed-form solution
• Adversarial methods: learn non-linear alignments
• Apply at intermediate network layers

Deploy continuous monitoring pipelines that compute drift metrics on streaming inference data. Track the Population Stability Index (PSI) for binned features and the Wasserstein distance for continuous distributions. Set automated alerts when drift exceeds predefined thresholds, triggering model retraining or fallback to a heuristic baseline.

• PSI < 0.1: no significant shift
• PSI 0.1–0.25: moderate shift, investigate
• PSI > 0.25: significant shift, trigger remediation

Design input features that are inherently invariant to expected environmental changes. In RF applications, replace raw IQ samples with cyclostationary signatures or higher-order cumulants that remain stable across varying channel conditions. Normalization techniques like instance normalization or per-channel standardization can remove nuisance variation before it reaches the model.

• Use physics-informed signal representations
• Apply adaptive gain control preprocessing
• Prefer relative over absolute feature values