Maximum Mean Discrepancy (MMD)

MMD measures the distance between two probability distributions P and Q by comparing their mean embeddings in a reproducing kernel Hilbert space (RKHS). The squared MMD is defined as the squared RKHS distance between the kernel mean embeddings:

• Definition: MMD² = ||μ_P - μ_Q||²_H
• Key insight: If the kernel is characteristic (e.g., Gaussian RBF), MMD(P, Q) = 0 if and only if P = Q
• Unbiased estimator: Computed from finite samples using the V-statistic or U-statistic formulation
• Kernel choice: The Gaussian kernel k(x, y) = exp(-||x - y||² / 2σ²) is the most common characteristic kernel, with the bandwidth σ controlling the scale of distributional features captured

MMD serves as a non-parametric statistical test for the null hypothesis that two sets of samples are drawn from the same distribution. It is widely used in batch effect detection and correction validation:

• Null hypothesis (H₀): P = Q (no batch effect present)
• Test procedure: Compute the empirical MMD and compare against a null distribution estimated via permutation testing or a parametric bootstrap
• Quadratic-time computation: The naive estimator requires O(n²) operations, but linear-time approximations exist using random Fourier features
• Application in batch correction: MMD is used to quantitatively evaluate whether latent representations from different experimental batches have been successfully aligned, complementing metrics like kBET and LISI

MMD is fully differentiable with respect to its inputs, making it an ideal domain-invariant loss for training deep neural networks. This property enables its use in domain-adversarial and disentanglement architectures:

• MMD-AAE (Adversarial Autoencoders): Replaces the adversarial discriminator with an MMD loss to match the aggregated posterior of the latent code to a prior distribution, providing more stable training
• Domain-Adversarial Neural Networks (DANN): MMD can substitute the domain classifier, directly minimizing the distributional discrepancy between source and target domain features
• Batch correction via MMD: The loss L = L_task + λ · MMD²(Z_batch_A, Z_batch_B) is minimized to learn a latent representation Z that is invariant to batch identity while preserving biological signal
• Gradient flow: Backpropagation through the MMD computation requires computing kernel derivatives, which are well-defined for smooth kernels like the Gaussian RBF

The witness function f*(x) = μ_P(x) - μ_Q(x) identifies where in the input space the two distributions differ most. This provides interpretability for batch effect analysis:

• Definition: The function that maximizes the mean discrepancy, revealing the regions of feature space where P and Q diverge
• Interpretation: High absolute values of the witness function indicate systematic shifts between batches at specific feature coordinates
• Batch effect localization: By evaluating the witness function on latent representations, researchers can identify which cell types or gene programs are most affected by technical variation
• Visualization: Plotting the witness function over a low-dimensional embedding (e.g., t-SNE or UMAP) creates a heatmap of distributional discrepancy, guiding targeted correction strategies

Standard MMD with a fixed Gaussian kernel may lose statistical power in high-dimensional spaces. Deep kernel MMD addresses this by learning an optimal kernel representation:

• Deep kernel: k_ω(x, y) = k(φ_ω(x), φ_ω(y)), where φ_ω is a neural network feature extractor
• Optimization: The kernel parameters ω are optimized to maximize the test power of the two-sample test, effectively learning the most discriminative features for distinguishing distributions
• Application to scRNA-seq: Deep kernel MMD can learn a latent space where batch effects are maximally detectable, then be inverted to learn a batch-invariant representation
• Relationship to C2ST: This approach connects to classifier two-sample tests (C2ST), where a classifier's ability to discriminate batches is used as a proxy for distributional discrepancy

MMD and optimal transport (OT) are complementary frameworks for comparing distributions, each with distinct computational and theoretical properties relevant to batch correction:

• MMD: Compares distributions via kernel mean embeddings in RKHS; provides a closed-form, differentiable metric with O(n²) complexity
• Wasserstein distance (OT): Solves a transportation problem to find the minimal-cost mapping between distributions; respects the geometry of the underlying metric space
• Trade-offs: MMD is computationally cheaper and easier to integrate into deep learning pipelines, while OT provides a more geometrically faithful alignment that preserves local structure
• Hybrid approaches: Sinkhorn divergences and kernelized OT bridge the gap, combining the computational efficiency of MMD with the geometric sensitivity of optimal transport for robust batch correction

The theoretical foundation of MMD, representing a probability distribution as an element in a Reproducing Kernel Hilbert Space (RKHS). By mapping distributions to this high-dimensional feature space, complex distributional properties are captured by a single mean vector. The distance between two distributions is then simply the distance between their kernel mean embeddings, enabling a non-parametric test that leverages the kernel trick to implicitly compute inner products in infinite-dimensional spaces without explicit feature mapping.

A statistical hypothesis test evaluating whether two independent samples are drawn from the same underlying distribution. MMD provides a powerful, non-parametric alternative to classical tests like the Kolmogorov-Smirnov test or t-test. Its key advantage is the ability to detect any difference between distributions—not just shifts in mean or variance—by operating in a characteristic RKHS. The null hypothesis is rejected if the empirical MMD statistic exceeds a threshold determined by permutation or asymptotic approximations.

A kernel function, such as the Gaussian Radial Basis Function (RBF) or Laplacian kernel, whose associated kernel mean embedding is injective. This property guarantees that the MMD between two distributions is zero if and only if the distributions are identical. Without a characteristic kernel, MMD loses its theoretical guarantee as a proper metric. The bandwidth parameter of the RBF kernel critically controls the scale of features being compared, acting as a sensitivity knob for detecting distributional differences.

A deep learning architecture that shares MMD's goal of learning domain-invariant features. While MMD explicitly minimizes a kernel-based distance metric between latent distributions, DANN uses a gradient reversal layer and a domain classifier to achieve adversarial invariance. The feature extractor is trained to maximize the domain classifier's loss, forcing it to produce representations that are uninformative about the batch of origin. Both methods are cornerstones of unsupervised domain adaptation.

An alternative metric for comparing probability distributions, derived from optimal transport theory. Unlike MMD, which compares distributions via kernel mean embeddings, the Wasserstein distance computes the minimal cost of transporting mass from one distribution to another. It provides a geometrically meaningful distance even when distributions have non-overlapping support—a scenario where other divergences like KL-divergence fail. This makes it a popular alternative loss function in generative models like Wasserstein GANs.

A Hilbert space of functions where point evaluation is a continuous linear functional. This property, formalized by the reproducing property f(x) = <f, k(x, ·)>, is the mathematical engine behind MMD. By embedding distributions into an RKHS, the complex problem of comparing probability densities is reduced to comparing simple inner products between mean embeddings. The choice of kernel k defines the geometry of the RKHS and determines which statistical features of the data are emphasized during the comparison.