Maximum Mean Discrepancy (MMD)

MMD quantifies the difference between two distributions P and Q by computing the squared distance between their mean embeddings in a Reproducing Kernel Hilbert Space (RKHS). The statistic is defined as:

MMD² = ||μ_P - μ_Q||²_H

• μ_P and μ_Q are the kernel mean embeddings of the distributions
• The choice of kernel (e.g., Gaussian RBF, polynomial) determines which moments of the distributions are compared
• A characteristic kernel (like the Gaussian kernel) ensures MMD is zero if and only if P = Q, making it a true metric

In practice, MMD is computed from finite samples drawn from each distribution. The unbiased empirical estimator is:

MMD² = (1/m(m-1)) Σᵢ Σⱼ₌ᵢ k(xᵢ, xⱼ) + (1/n(n-1)) Σᵢ Σⱼ₌ᵢ k(yᵢ, yⱼ) - (2/mn) Σᵢ Σⱼ k(xᵢ, yⱼ)

• k(·,·) is the kernel function measuring similarity between samples
• The first two terms capture within-distribution similarity
• The cross-term penalizes between-distribution similarity
• Computational complexity is O((m+n)²), making it quadratic in sample size

In channel-robust RF fingerprinting, MMD is integrated into the training objective to align feature distributions across different channel conditions:

L_total = L_task + λ · MMD²(Z_source, Z_target)

• Z_source and Z_target are the learned feature representations from different domains
• λ controls the trade-off between task performance and distribution alignment
• Minimizing MMD forces the encoder to produce channel-invariant features
• Unlike adversarial methods, MMD does not require training a separate domain classifier, simplifying optimization

The effectiveness of MMD critically depends on the kernel function and its hyperparameters. Key considerations include:

• Gaussian RBF kernel: k(x,y) = exp(-||x-y||² / 2σ²) is the most common choice due to its characteristic property
• Bandwidth σ: Controls the scale at which distribution differences are detected
• Too small σ: MMD captures only local noise, leading to high variance
• Too large σ: MMD loses sensitivity to fine-grained distribution differences
• Median heuristic: Setting σ to the median pairwise distance among samples provides a robust default
• Multiple kernel learning can combine kernels at different scales for improved robustness

MMD enables formal two-sample tests to determine whether two sets of samples originate from the same distribution. The testing framework includes:

• Null hypothesis H₀: P = Q (distributions are identical)
• Test statistic: The empirical MMD² estimate
• Permutation testing: Shuffling sample labels repeatedly to estimate the null distribution without parametric assumptions
• Asymptotic distribution: Under H₀, MMD² converges to a weighted sum of χ² variables
• This provides a principled method to verify that domain adaptation has successfully aligned feature distributions

MMD offers distinct advantages over other distribution divergence measures in deep learning contexts:

• vs. KL Divergence: MMD does not require density estimation and is well-defined even when distributions have disjoint support
• vs. Wasserstein Distance: MMD is computationally simpler, avoiding the need for adversarial training or optimal transport solvers
• vs. CORAL Loss: CORAL aligns only second-order statistics (covariance), while MMD with a characteristic kernel matches all moments
• vs. Adversarial Methods: MMD provides a stable, non-adversarial objective that avoids mode collapse and discriminator training instability

A subfield of transfer learning that addresses the problem of training a model on a source domain with labeled data and deploying it on a different but related target domain with different data distributions. MMD is a primary tool for minimizing the distribution shift between source and target feature representations, ensuring a fingerprinting model trained in one channel condition remains accurate in another.

A metric derived from optimal transport theory that measures the minimum cost of transforming one probability distribution into another. Compared to MMD, the Wasserstein distance provides a more geometrically meaningful measure of distance between distributions, especially when their supports do not overlap. It is often used as an alternative loss function for aligning complex, high-dimensional feature distributions across varying channel conditions.

A domain adaptation loss function that aligns the second-order statistics of source and target feature distributions by minimizing the difference between their covariance matrices. While MMD aligns distributions in a reproducing kernel Hilbert space, CORAL provides a simpler, parameter-free alternative by directly matching feature correlations, making it computationally efficient for real-time RF fingerprinting applications.

A technique that trains neural networks to learn features that are discriminative for the primary task while being indistinguishable across different domains. A Gradient Reversal Layer is used to maximize domain classifier loss, forcing the feature extractor to produce domain-invariant representations. MMD-based alignment serves as a complementary or alternative approach to adversarial methods for achieving this invariance.

The process of separating a learned representation into independent, interpretable factors of variation. In RF fingerprinting, the goal is to isolate device-specific features from channel-induced distortions. MMD can be applied to enforce statistical independence between these latent factors, ensuring that the fingerprint embedding captures only hardware impairments and not environmental artifacts.

A self-supervised learning paradigm that trains models to pull representations of similar data points together and push dissimilar ones apart in the embedding space. While contrastive methods use instance-level discrimination, MMD provides a distribution-level alignment signal. Combining both approaches can yield representations that are both locally consistent and globally domain-invariant for robust emitter identification.