Membership Inference Attack

The adversary trains a binary attack classifier to distinguish between the target model's outputs on member and non-member samples. Key signals include:

• Prediction confidence: Models are typically more confident on training data
• Loss values: Training samples exhibit lower cross-entropy loss
• Gradient norms: Member gradients often have smaller L2 norms
• Logit distributions: Output layer activations differ statistically

The attack exploits overfitting—the more a model memorizes its training data, the more vulnerable it is to membership inference.

Membership inference in federated learning presents unique attack surfaces:

• Honest-but-curious server: The central aggregator analyzes model updates to infer membership of local training samples
• Malicious participant: A compromised client observes global model updates to determine if specific records exist in other institutions' datasets
• External auditor: An entity with only query access to the final federated model attempts membership inference through prediction API calls

Each threat model requires different defense strategies and carries distinct regulatory implications under HIPAA and GDPR.

Differential privacy (DP) provides formal guarantees against membership inference by bounding the influence of any single training sample on the model output:

• ε (epsilon): The privacy budget parameter—lower values provide stronger protection but degrade model utility
• Gaussian mechanism: Adds calibrated noise to gradients during federated aggregation
• Clipping: Constrains the L2 norm of individual updates before noising

A model trained with (ε=8, δ=10⁻⁵)-DP limits an adversary's ability to determine membership with high confidence, though utility trade-offs must be carefully managed in clinical applications.

Likelihood ratio attacks provide a rigorous framework for empirically measuring membership inference vulnerability:

• LiRA (Likelihood Ratio Attack): Trains multiple shadow models on random subsets to estimate the distribution of model outputs conditioned on membership
• True positive rate at low false positive: Measures attack precision when the adversary is constrained to minimal false alarms
• ROC AUC: Quantifies overall distinguishability between members and non-members

LiRA-based auditing has become the de facto standard for evaluating privacy leakage in federated models before clinical deployment.

Multiple complementary defenses reduce membership inference risk:

• Knowledge distillation: Training a student model on softened outputs reduces memorization of individual samples
• Early stopping: Halting training before convergence limits overfitting to training data
• Regularization techniques: L2 weight decay, dropout, and label smoothing all reduce the gap between training and test behavior
• Model pruning: Removing low-magnitude weights can eliminate memorized patterns
• Federated averaging frequency: Less frequent aggregation with more local epochs increases privacy but may slow convergence

Membership inference attacks directly challenge compliance with healthcare privacy regulations:

• HIPAA: Successful membership inference may constitute a breach of protected health information if an individual's presence in a treatment cohort is revealed
• GDPR Article 22: Automated decisions based on inferred membership may violate rights to human review
• FDA SaMD guidelines: Software as a Medical Device submissions increasingly require privacy risk assessments including membership inference vulnerability

Organizations deploying federated learning must document attack surface analyses and demonstrate that residual membership inference risk falls within acceptable thresholds.

An adversarial technique that reconstructs representative features of a target class from model outputs. Unlike membership inference, which asks 'was this record used?', model inversion asks 'what does the model remember?'

• Gradient-based inversion reconstructs training inputs from shared model updates
• Confidence-based inversion uses prediction scores to generate class prototypes
• Particularly dangerous in medical imaging where faces or scans can be recovered
• Defenses include differential privacy and limiting output granularity

A privacy attack that predicts sensitive attributes about individuals from model outputs and non-sensitive auxiliary data. The adversary leverages correlations learned during training.

• Infers genetic markers from medication response predictions
• Exploits statistical correlations between demographic features and health outcomes
• Differs from membership inference by targeting specific traits rather than training set presence
• Mitigation requires fairness constraints and feature suppression techniques

The primary mathematical defense against membership inference. It provides a formal guarantee by bounding the influence of any single record on model outputs through calibrated noise injection.

• Epsilon (ε) parameter controls the privacy-utility tradeoff
• Gaussian mechanism adds noise proportional to query sensitivity
• DP-SGD clips and noises gradients during federated training
• A privacy budget tracks cumulative information leakage across queries

The foundational methodology for executing membership inference attacks. The adversary trains shadow models that mimic the target model's behavior on known data to learn the signature of membership.

• Requires auxiliary data from the same distribution as the private training set
• Each shadow model is trained on a known split of member and non-member records
• Attack model learns to distinguish confidence score patterns between members and non-members
• Effectiveness correlates with overfitting in the target model

An integrity attack where adversaries inject malicious samples into training data to compromise model behavior. When combined with membership inference, poisoning can amplify privacy leakage.

• Backdoor attacks create targeted misclassifications triggered by specific patterns
• Gradient poisoning in federated learning corrupts the global model through malicious updates
• Poisoned models often exhibit higher overfitting, increasing membership inference vulnerability
• Defenses include robust aggregation and anomaly detection on local updates

The degree to which a model memorizes rather than generalizes from training data directly correlates with membership inference attack success. Overfit models exhibit distinct behavior on seen versus unseen records.

• Memorization of rare or unique training examples creates high-confidence predictions
• Loss gap between training and test sets serves as an attack signal
• Regularization techniques like dropout and weight decay reduce vulnerability
• Early stopping during federated training limits memorization without sacrificing utility