Membership Inference Attack

Attackers train shadow models to mimic the target model's behavior. By creating multiple models on known datasets, the attacker generates labeled training data for the attack classifier.

• Synthetic Data: Shadow models are trained on data synthesized from the target model's output distribution.
• Black-box Access: This technique requires only API-level query access, making it highly practical against commercial ML services.
• Binary Classifier: The attack model learns to distinguish 'In' members from 'Out' non-members based on prediction confidence vectors.

The attack leverages the observation that models exhibit higher prediction confidence on training samples. Overfitted models are particularly vulnerable.

• Posterior Distribution: The full probability vector provides a rich signal for distinguishing members from non-members.
• Correct Classification Bias: Training samples are more likely to be classified correctly with high confidence than test samples.
• Defense Mechanism: Confidence score masking and prediction vector truncation directly reduce the information available to this attack vector.

Differential Privacy (DP) provides a formal mathematical guarantee against membership inference. By bounding the influence of any single data point, DP makes it statistically impossible to determine membership.

• Epsilon Budget: A lower privacy budget (ε) directly correlates with stronger resistance to membership inference.
• DP-SGD: Differentially Private Stochastic Gradient Descent clips gradients and adds calibrated Gaussian noise during training.
• Privacy-Utility Trade-off: Stronger DP guarantees inevitably degrade model accuracy, requiring careful calibration for production systems.

Overfitting is the primary enabler of membership inference. A model that memorizes rare features of training data creates a distinguishable gap between its behavior on training versus holdout data.

• Generalization Gap: The difference in loss between training and test sets serves as a direct leakage signal.
• Regularization: Techniques like dropout, weight decay, and early stopping reduce overfitting and incidentally improve privacy.
• Memorization: Large models can memorize rare sequences verbatim, making extraction and inference trivial without proper defenses.

Membership inference attacks are categorized by the level of access the adversary has to the target model.

• Black-box Attacks: Rely solely on output scores or labels. These are the most realistic threat vectors for cloud APIs.
• White-box Attacks: Exploit access to model weights, gradients, or intermediate layer activations to compute per-sample loss values.
• Label-only Attacks: A severe subset of black-box attacks that requires only the predicted class label, not confidence scores, making them robust against confidence masking defenses.

Evaluating vulnerability requires precise metrics beyond simple accuracy. The attack's success is measured by its ability to distinguish members from non-members.

• Attack AUC-ROC: The Area Under the Receiver Operating Characteristic curve measures the attack classifier's discriminative power.
• True Positive Rate at Low FPR: Measures the fraction of training members correctly identified while maintaining a very low false positive rate (e.g., 0.1%).
• MIA Advantage: A privacy metric quantifying the difference between the attack's true positive rate and false positive rate.