Membership Inference Attack

The attacker trains multiple shadow models that mimic the target model's behavior on known datasets. These shadow models learn to distinguish between members (data used in training) and non-members (holdout data). The attack model is then trained on the shadow models' prediction outputs—such as confidence scores, loss values, or logit vectors—to create a binary classifier that generalizes to the target model.

• Requires query access to the target model (black-box or white-box)
• Shadow models approximate the target's decision boundary
• Effective even when only top-k prediction scores are available

Membership inference success is directly proportional to model overfitting. Overfitted models memorize training examples rather than learning generalizable patterns, producing distinctively higher confidence scores on training data. This memorization creates a detectable signal: the model's predictions on members exhibit lower entropy and higher maximum class probabilities compared to non-members.

• Dropout and weight decay reduce attack surface
• Early stopping is a critical defense mechanism
• Differential privacy bounds memorization mathematically

The per-example loss value is the most informative feature for membership inference. Models typically exhibit lower loss on training examples because they were explicitly optimized to minimize error on those records. Attackers compute loss for the target record and compare it against a calibrated threshold derived from shadow model behavior.

• Cross-entropy loss is the standard signal
• Likelihood ratio attacks formalize this comparison
• Works even without confidence score access (label-only attacks)

In the most restrictive threat model, attackers only observe the predicted class label—not confidence scores or loss values. Label-only attacks exploit adversarial perturbation: by adding small, carefully crafted noise to the target record and observing whether the prediction changes, attackers infer membership. Training records are more robust to perturbation because they lie further from the decision boundary.

• Requires multiple queries with perturbed inputs
• Measures prediction robustness as a proxy for membership
• Effective against API-only deployments with minimal output

Differential privacy (DP) provides a formal mathematical guarantee against membership inference. By clipping gradients and adding calibrated Gaussian noise during training, DP bounds the influence of any single training example on the final model parameters. The privacy budget epsilon (ε) quantifies the guarantee: lower epsilon values provide stronger protection but may degrade model utility.

• DP-SGD is the standard training algorithm
• Privacy-utility trade-off must be carefully tuned
• Provides provable bounds on attack success rate

Generative models trained to produce synthetic data are doubly susceptible to membership inference. First, the generator itself may memorize training records and reproduce them verbatim. Second, downstream models trained on synthetic data can leak information about the original training set. Nearest neighbor adversarial accuracy (NNAA) specifically measures this risk by testing whether synthetic records are closer to training data than holdout data.

• GANs and diffusion models both exhibit memorization
• Data duplication auditing detects overfitted generators
• TSTR evaluation should include privacy metrics