Feature Distribution Skew

The most common real-world cause of feature skew in healthcare. Different hospitals serve populations with distinct demographic profiles, leading to divergent input distributions.

• Age Distribution: A pediatric hospital's patient features cluster in the 0-18 range, while a geriatric clinic's data centers on 65+
• Geographic Variation: Genetic and lifestyle factors tied to geography shift baseline lab values and biometrics
• Socioeconomic Factors: Tertiary care centers in urban areas see different disease presentations than rural community clinics
• Example: A model trained on predominantly Caucasian populations may fail to generalize to hospitals serving Asian or African demographics due to differing baseline melanin levels in dermatology imaging

Medical imaging and sensor data exhibit feature skew due to hardware variability across institutions, even when capturing the same pathology.

• Scanner Manufacturers: MRI machines from Siemens, GE, and Philips produce images with different intensity distributions, resolution, and noise characteristics
• Acquisition Protocols: Variations in slice thickness, contrast agent timing, and pulse sequences create systematic feature differences
• Sensor Calibration: Wearable devices and IoT medical sensors from different manufacturers report vital signs with distinct calibration curves
• Mitigation: Federated harmonization techniques like ComBat or optimal transport align feature distributions without sharing raw patient data

Standard batch normalization tracks running means and variances that inadvertently encode local feature distributions, creating both privacy risks and performance degradation under feature skew.

• Privacy Leakage: Local BN statistics can reveal demographic information about a client's patient population
• FedBN Variant: Maintains separate BN layers for each client while sharing only convolutional and linear layer weights, preserving local feature normalization
• Group Normalization Alternative: Replaces batch-level statistics with channel-group statistics, eliminating dependence on batch composition entirely
• Practical Impact: FedBN can recover up to 10-15% accuracy lost to feature skew in medical imaging tasks

Maximum Mean Discrepancy (MMD) explicitly minimizes the distance between feature distributions of different clients in a reproducing kernel Hilbert space.

• Statistical Foundation: MMD measures the distance between two probability distributions by comparing their mean embeddings in a high-dimensional feature space
• Federated Implementation: A regularization term is added to the local training objective that penalizes divergence between local feature representations and a global reference distribution
• Kernel Choice: Gaussian RBF kernels are standard; the bandwidth parameter controls the scale of distribution matching
• Communication Efficiency: Only aggregated MMD statistics are shared, not raw features, preserving privacy while enabling alignment

Hospital A uses Siemens MRI scanners while Hospital B uses GE scanners. Even with identical acquisition protocols, the raw pixel intensity histograms differ due to proprietary reconstruction algorithms and coil sensitivities.

• A model trained naively with FedAvg will learn scanner-specific artifacts rather than pathology
• Federated Harmonization techniques like ComBat or MMD-based alignment are required
• Without correction, the global model's accuracy on a third site with Philips scanners drops by 15-25%

A pediatric hospital contributes data with patient ages 0-18 years, while a Veterans Affairs hospital contributes data with ages 55-85 years. The marginal distribution of the 'age' feature is completely disjoint.

• A federated model for disease prediction must learn age-invariant representations
• Federated Invariant Risk Minimization can help discover causal features rather than spurious age correlations
• Simple normalization per client fails when distributions have no overlap

Lab A reports HbA1c in mmol/mol (IFCC units) while Lab B reports HbA1c as a percentage (NGSP units) . Both measure the same biomarker but the feature scales differ by a linear transformation.

• This is a tractable form of feature skew solvable via z-score normalization per client before aggregation
• More insidious: Lab A uses a high-performance liquid chromatography method, Lab B uses immunoassay—same units, different underlying distributions
• Federated feature alignment with maximum mean discrepancy loss can correct for assay-specific biases

Clinic X codes conditions using ICD-10-CM with high specificity, while Clinic Y uses truncated ICD-9 codes from legacy systems. The feature space cardinality and granularity differ fundamentally.

• One-hot encoding produces incompatible feature dimensions across clients
• Federated transfer learning with a shared embedding layer can map disparate coding systems to a common latent space
• Entity resolution across coding ontologies must be performed before federated training begins

A remote patient monitoring study collects ECG at 256 Hz from medical-grade Holter monitors at Site C, while Site D collects PPG at 25 Hz from consumer smartwatches. The temporal resolution and sensor modality differ entirely.

• Feature extraction pipelines produce time-series features with different Nyquist limits
• Domain generalization approaches must learn representations robust to sampling rate
• Federated multi-modal fusion can treat each sensor type as a separate view of the same physiological state

A lung cancer screening model is trained across sites in Shanghai, London, and Lagos. Chest CT radiomic features—texture, shape, intensity—vary due to population-specific granulomatous disease prevalence, air pollution artifacts, and nutritional status.

• Federated adversarial training with a domain discriminator can learn site-invariant representations
• Clustered federated learning may group sites by similar radiomic profiles before aggregation
• A model that fails to account for this skew will misclassify benign granulomas as malignancies in high-prevalence regions