Federated Feature Alignment

A kernel-based statistical test used as a regularization loss during federated training. MMD measures the distance between two probability distributions by comparing their embeddings in a Reproducing Kernel Hilbert Space (RKHS). In federated learning, an MMD penalty is added to the local objective function to force the feature extractor to produce client-invariant representations. This directly combats feature distribution skew without requiring raw data sharing.

Correlation Alignment (CORAL) aligns the second-order statistics (covariance matrices) of feature maps from different client domains. The CORAL loss minimizes the Frobenius norm of the difference between source and target covariance matrices. It is computationally lighter than MMD and highly effective for mitigating covariate shift in medical imaging, where different scanner vendors produce systematic intensity variations.

Uses a Gradient Reversal Layer (GRL) and a domain classifier to learn features that are indistinguishable across clients. During backpropagation, the GRL reverses the gradient from the domain classifier, forcing the feature extractor to maximize domain confusion. This results in a shared representation space where a patient's data looks statistically similar regardless of which hospital generated it.

Instead of sharing gradients, clients exchange compact class-representative vectors (prototypes). The global server aligns these prototypes across clients by minimizing the distance between prototypes of the same class from different sites. This method is naturally robust to label distribution skew and significantly reduces communication overhead compared to full model transfer.

Applies Wasserstein distance minimization to map the local data distribution of one client onto another. By solving the optimal transport plan, this technique can handle complex, non-linear shifts in feature space. It is particularly useful for federated harmonization in neuroimaging, where subtle structural variations in MRI scans must be normalized across sites.

Leverages self-supervised contrastive learning to pull representations of semantically similar data points together across client boundaries while pushing dissimilar points apart. This method does not require labeled data for alignment and excels at learning robust features under extreme statistical heterogeneity, often outperforming supervised alignment when local label quality varies significantly.

Correlation Alignment (CORAL) aligns the second-order statistics of feature distributions by minimizing the difference between covariance matrices of different client domains.

• Mechanism: Matches the covariance of source and target feature activations
• Loss formulation: Frobenius norm of the difference between covariance matrices
• Federated benefit: Simple to implement and computationally efficient compared to adversarial methods
• Use case: Effective when feature distributions differ primarily in scale and correlation structure rather than higher-order moments

Uses a domain discriminator network with a gradient reversal layer to learn feature representations that are invariant to the client's identity.

• Architecture: A shared feature extractor is trained to fool a domain classifier that tries to predict which client generated the data
• Gradient reversal layer: Multiplies gradients by a negative constant during backpropagation, encouraging domain-invariant features
• Advantage: Can capture complex, non-linear distribution shifts beyond first and second-order statistics
• Challenge: Requires careful tuning of the adversarial balance to avoid mode collapse

An optimization framework that learns data representations where the optimal classifier is identical across all training clients, aiming to discover causal relationships robust to spurious correlations.

• Core principle: Seeks a representation such that the optimal linear classifier on top of it matches across all environments
• Federated application: Each client acts as a distinct environment with its own spurious correlations
• Penalty term: Gradients of the local loss with respect to a dummy classifier are penalized if they vary across clients
• Outcome: Models that generalize to new client sites by ignoring non-causal, site-specific patterns

Applies optimal transport theory to align probability distributions across clients by minimizing the Wasserstein distance between their data representations.

• Wasserstein distance: Measures the minimal cost of transforming one distribution into another
• Sinkhorn algorithm: An efficient entropic regularization approach for computing optimal transport plans
• Federated implementation: Clients compute local transport maps to a shared barycenter distribution without sharing raw data
• Medical imaging use: Highly effective for harmonizing MRI intensities across different scanner vendors and protocols

A communication-efficient method where clients share compact class-representative vectors (prototypes) instead of full model updates, naturally handling feature and label distribution skew.

• Prototype definition: The mean embedding vector for each class computed from local data
• Alignment process: Global prototypes are aggregated and redistributed; local models minimize distance to global prototypes
• Privacy benefit: Prototypes are aggregated statistics, not individual samples
• Regularization: Local training includes a cross-entropy loss with global prototypes as soft targets, encouraging inter-client feature consistency