Membership Inference Attack

A privacy attack that infers sensitive personal attributes of individuals in the training data by analyzing a model's predictions and confidence scores. Unlike membership inference, which asks 'was this record in the training set?', attribute inference asks 'what hidden characteristics does this record have?'

• Exploits correlations between model outputs and sensitive features
• Particularly dangerous when models unintentionally memorize demographic patterns
• Can reveal protected attributes like health status, income, or political affiliation
• Often uses shadow models trained on auxiliary data to calibrate the attack

A privacy breach where an attacker reconstructs sensitive training data or statistical features from a model's parameters or outputs. This goes beyond membership inference by actually recovering the content of training records rather than just their presence.

• Can generate realistic synthetic faces from facial recognition models
• Exploits confidence scores to iteratively refine reconstructed inputs
• Particularly effective against high-dimensional output models like image classifiers
• Defenses include gradient clipping and limiting output precision

An attack that reconstructs private local training data from the shared model gradients during collaborative learning. In federated settings, gradients are transmitted instead of raw data, but these gradients carry detailed information about the local batch.

• Can recover pixel-accurate images and exact text sequences from gradients
• Exploits the fact that gradients are direct functions of input data
• Deep leakage from gradients (DLG) achieves high-fidelity reconstruction in <100 iterations
• Defenses include gradient perturbation, compression, and secure aggregation

An attack where an adversary systematically queries a target model to reconstruct its functionality or steal its learned parameters. While not directly a privacy attack, extracted models can enable subsequent membership inference against the original training data.

• Attacker builds a substitute model that mimics the victim's decision boundary
• Requires only black-box API access to the target model
• Can steal proprietary models trained on millions of dollars of compute
• Defenses include rate limiting, query auditing, and output rounding

A mathematical framework that provides a provable guarantee against information leakage by adding calibrated statistical noise to data or model outputs. It is the primary defense against membership inference attacks, bounding what any adversary can learn about individual records.

• Controlled by epsilon (ε), the privacy budget parameter
• Lower epsilon = stronger privacy but reduced model utility
• DP-SGD clips and noises gradients during training for formal guarantees
• Provides a quantifiable, auditable privacy-utility tradeoff

The foundational attack methodology behind most membership inference implementations. The attacker trains multiple replica models on datasets that mimic the target model's training distribution, then uses their known membership labels to train a binary classifier that distinguishes members from non-members.

• Requires auxiliary data from the same distribution as the target training set
• Each shadow model's training data is split into member and non-member sets
• The attack model learns to detect subtle differences in confidence, entropy, and loss
• Effective even against black-box models that only return top-1 predictions