Attack Model

The adversary trains multiple shadow models on datasets synthetically generated from the target model's output distribution. Each shadow model mimics the target's behavior, and its training data is fully known, creating a labeled dataset where membership status is ground truth. This process converts a black-box privacy attack into a supervised learning problem by generating synthetic member/non-member pairs for classifier training.

The attack model is fundamentally a binary classifier that distinguishes between two classes: IN (member of training set) and OUT (non-member). It learns to detect subtle statistical differences in the target model's behavior—such as higher confidence scores, lower entropy, or smaller loss values—that correlate with training data membership. The decision boundary is calibrated using known membership labels from shadow model outputs.

Attack features are engineered from the target model's prediction vector and include:

• Prediction confidence on the true class label
• Prediction entropy across all classes
• Prediction correctness (binary indicator)
• Loss value computed against the true label
• Gap metric: difference between top-1 and top-2 confidence scores These features exploit the observation that models are typically more confident and accurate on training data.

Attack models operate under different access assumptions:

• Black-box attacks: Only final prediction scores or labels are available
• White-box attacks: Full access to model parameters, gradients, and architecture
• Label-only attacks: Only the predicted class label is returned, exploiting robustness differences
• Partial white-box: Access to intermediate layer outputs or embeddings The attack model's feature set adapts to the available information level.

The attack model's output is a membership probability score that must be calibrated against a decision threshold. Adversaries tune this threshold to balance precision (fraction of flagged records that are true members) and recall (fraction of true members successfully identified). Receiver operating characteristic (ROC) curves and precision-recall curves are standard evaluation tools, with AUC serving as the primary aggregate metric for attack effectiveness.

Attack model effectiveness is directly degraded by differential privacy mechanisms. When the target model is trained with DP-SGD, the calibrated noise added to gradients reduces the memorization signal that attack models exploit. The attack AUC drops as the privacy budget epsilon decreases, with strong privacy guarantees (epsilon < 1) typically reducing attack performance to near-random guessing. This establishes a quantifiable privacy-utility-attack tradeoff.