Membership Inference Attack (MIA)

The attacker trains multiple shadow models that mimic the target model's behavior. These shadow models are trained on datasets drawn from the same distribution as the target's training data.

• Synthetic Datasets: The attacker generates labeled data by querying the target model, using its predictions as ground truth.
• Known Membership: Since the attacker controls the shadow training, they know exactly which records were in the training set and which were held out.
• Attack Model: A binary classifier is trained on the shadow models' prediction vectors and membership labels to distinguish members from non-members.

The attack relies on the target model's prediction vector—the softmax output containing confidence scores for each class. Overfitted models exhibit distinct patterns in these vectors.

• Confidence Disparity: Training members typically receive higher confidence scores for their true class compared to non-members.
• Entropy Analysis: The entropy of the prediction vector is often lower for training members, indicating more decisive predictions.
• Loss-Based Signals: The model's cross-entropy loss on a record serves as a strong membership signal; lower loss correlates with training set inclusion.

MIAs are categorized by the level of access the adversary has to the target model, which dictates the features available for the attack.

• Black-Box (Label-Only): The attacker observes only the final predicted class label. Attacks in this setting exploit robustness to adversarial perturbations or distance to the decision boundary.
• Black-Box (Score-Based): The attacker receives the full prediction vector with confidence scores, enabling more precise statistical analysis.
• White-Box: The attacker has full access to model parameters and gradients, allowing for the most powerful attacks using gradient norms and internal activations.

The primary vulnerability enabling MIAs is model overfitting—when a model memorizes specific details of training examples rather than learning generalizable patterns.

• Memorization Gap: Overfitted models exhibit a measurable gap in prediction confidence between training and test data.
• Generalization vs. Privacy: There exists a fundamental tension; models that generalize well are inherently more resistant to membership inference.
• Regularization Impact: Techniques like dropout, weight decay, and early stopping reduce overfitting and thus mitigate MIA risk.

Modern MIAs often bypass the need for shadow models by using threshold-based metrics computed directly on the target model's outputs.

• Prediction Correctness: A simple baseline—if the model predicts the correct label, the record is more likely a member.
• Prediction Gap: The difference between the top two confidence scores; larger gaps indicate membership.
• Modified Entropy: Variants of Shannon entropy that amplify the signal from the top confidence scores are used as scalar membership scores.

Differential Privacy (DP) provides a formal mathematical guarantee that bounds an attacker's ability to infer membership. DP-SGD is the standard training algorithm.

• Privacy Budget (ε): A smaller epsilon value provides stronger protection against MIAs by limiting the influence of any single training example.
• Noise Calibration: Gaussian noise is added to clipped gradients, masking the individual contribution of each data point.
• Provable Guarantee: Unlike heuristic defenses, DP offers a quantifiable upper bound on the attacker's true positive rate and false positive rate.