Wasserstein Distance

The Wasserstein distance is interpreted as the minimum cost required to transform one probability distribution into another. Imagine two piles of earth representing the training and production feature distributions. The distance is the total amount of mass multiplied by the average distance it must be moved. This geometric grounding means the metric respects the topology of the feature space—small shifts in feature values produce small distance values, while abrupt, distant changes produce large ones. This property is critical for fraud monitoring, where a gradual drift in transaction amounts should be flagged differently than a sudden, structural break in user behavior patterns.

Unlike Kullback-Leibler (KL) Divergence, the Wasserstein distance is a true metric satisfying symmetry and the triangle inequality. Critically, it provides meaningful gradients even when distributions have non-overlapping supports. KL divergence explodes to infinity when the production distribution contains samples in regions where the training distribution has zero probability mass—a common scenario in fraud detection with novel attack vectors. Wasserstein smoothly quantifies how far apart these disjoint supports are. This avoids the vanishing gradient problem in drift detection systems and provides a more stable, interpretable signal for triggering model retraining pipelines.

Computing the exact Wasserstein distance involves solving an optimal transport problem, which is computationally expensive. The Kantorovich-Rubinstein duality provides a tractable alternative for the 1-Wasserstein distance. It reformulates the problem as finding a 1-Lipschitz function that maximizes the difference in expectations between the two distributions. This dual form underpins the Wasserstein GAN (WGAN) critic, where a neural network is constrained to be 1-Lipschitz via weight clipping or gradient penalty. In model monitoring, this dual formulation enables efficient, differentiable estimation of distributional distance in high-dimensional feature spaces.

Wasserstein distance excels at detecting covariate shift in financial transaction features. Consider a fraud model trained on normal transaction amounts with a mean of $200. If a production window shows a mean shift to $450, the Wasserstein distance captures not just that the distributions differ, but by how much in absolute dollar terms. This magnitude sensitivity is invaluable for severity-based alerting. A drift of 0.05 in the Wasserstein metric might trigger a warning, while a drift of 0.5 triggers an immediate investigation. This contrasts with purely statistical tests like the KS test, which may only report a p-value without conveying the practical significance of the shift.

Computing the Wasserstein distance in high-dimensional feature spaces suffers from the curse of dimensionality and high computational cost. The Sliced Wasserstein Distance (SWD) addresses this by projecting distributions onto random one-dimensional lines, computing the 1D Wasserstein distance on each projection, and averaging the results. The 1D Wasserstein distance has a closed-form solution using the cumulative distribution functions, making SWD highly efficient. For fraud model monitoring with hundreds of features, SWD provides a scalable approximation that retains the geometric sensitivity of the full metric while being computable in streaming contexts.

In production MLOps pipelines, the Wasserstein distance is computed over sliding windows of inference data. A reference distribution is established from the validation set, and the distance to each production window is tracked over time. When the distance exceeds a statistically derived threshold—often set using bootstrapped confidence intervals from the training data—a drift alert is generated. This alert can trigger automated workflows: shadow deployment of a challenger model, feature importance analysis to identify the drifted dimensions, or a full retraining cycle. The metric's smooth behavior makes it well-suited for integration with Statistical Process Control (SPC) charts for visual monitoring.

An asymmetric measure of how one probability distribution diverges from a reference distribution. Unlike Wasserstein distance, KL divergence is not a true metric—it lacks symmetry and does not satisfy the triangle inequality.

• Formula: D_KL(P || Q) = Σ P(x) log(P(x)/Q(x))
• Key limitation: Undefined when Q(x) = 0 but P(x) > 0
• Use case: Quantifying information loss when approximating one distribution with another
• Contrast with Wasserstein: KL divergence fails to provide meaningful gradients when distributions have non-overlapping supports, a problem Wasserstein solves elegantly

A symmetric metric that quantifies distributional shift by binning continuous variables and comparing expected versus observed proportions. PSI is the industry standard in financial model monitoring.

• Thresholds: PSI < 0.1 indicates no significant shift; 0.1-0.25 suggests moderate drift; > 0.25 signals severe distribution change
• Calculation: PSI = Σ (Actual% - Expected%) × ln(Actual% / Expected%)
• Limitation: Sensitive to binning strategy and struggles with high-dimensional data
• Relationship to Wasserstein: PSI operates on discretized univariate distributions, while Wasserstein distance handles multivariate continuous distributions natively without binning artifacts

A kernel-based statistical test that distinguishes distributions by comparing mean embeddings in a reproducing kernel Hilbert space (RKHS). MMD is widely used in domain adaptation and generative model evaluation.

• Key property: MMD = 0 if and only if the two distributions are identical (given a characteristic kernel)
• Kernel choices: Gaussian RBF kernel is standard; kernel bandwidth selection critically affects sensitivity
• Advantage over Wasserstein: Computationally cheaper to estimate with closed-form expressions
• Trade-off: MMD may miss geometric structure that Wasserstein captures through its earth mover's interpretation of optimal transport cost

A nonparametric statistical test comparing empirical cumulative distribution functions (CDFs). The KS statistic measures the maximum vertical distance between two CDFs.

• Statistic: D = max |F₁(x) - F₂(x)|, where F₁ and F₂ are empirical CDFs
• Sensitivity: Most sensitive near the median of the distribution; less powerful at detecting tail differences
• Fraud relevance: Often used to monitor individual feature drift in transaction amount distributions or user velocity metrics
• Contrast with Wasserstein: KS test provides a single scalar from maximum deviation, while Wasserstein integrates differences across the entire distribution, making it more sensitive to subtle shape changes

The operational context where distributional distance metrics are deployed. Covariate shift occurs when P(X) changes but P(Y|X) remains stable—the input feature distribution drifts while the decision boundary stays valid.

• Detection pipeline: Compute Wasserstein distance between training and production feature distributions on a sliding window
• Fraud example: A new payment method emerges, shifting transaction amount distributions without changing the underlying fraud patterns
• Response strategy: Trigger model retraining when Wasserstein distance exceeds a calibrated threshold
• Multivariate advantage: Wasserstein distance detects joint distribution shifts that univariate tests like PSI or KS miss entirely

A technique that trains a binary classifier to distinguish between training and production data samples. If the classifier achieves high accuracy, significant distributional shift is confirmed.

• Method: Label training data as 0, production data as 1; train a classifier; AUC > 0.7 indicates problematic drift
• Feature importance: The classifier's feature importance scores reveal which features are driving the distribution shift
• Complement to Wasserstein: Adversarial validation identifies that drift exists and locates its source, while Wasserstein distance provides a quantitative magnitude of the shift
• Operational integration: Use Wasserstein distance for continuous monitoring alerts; run adversarial validation during incident diagnosis to pinpoint root causes