Membership Inference Attack

A privacy attack where an adversary uses a model's outputs to reconstruct sensitive features of its training data. Unlike membership inference, which asks if a record was used, model inversion attempts to reveal what that record looked like.

• Objective: Reconstruct representative samples (e.g., a face from a facial recognition model's training set).
• Method: Often involves iterative querying and optimization to find an input that maximizes confidence for a target class.
• Defense: Techniques include differential privacy, which adds noise to training or outputs, and limiting the granularity of model confidence scores.

Also known as a model extraction attack, this technique aims to duplicate the functionality of a proprietary, black-box model by querying it and using the input-output pairs to train a surrogate model.

• Impact: Intellectual property theft, enabling cheaper local inference, and providing a local model for crafting more potent white-box attacks.
• Connection to MIA: A successfully stolen model can be analyzed locally to perform more accurate membership inference, as the attacker gains full white-box access to the surrogate.
• Defense: Query limiting, output obfuscation (e.g., returning only top-1 labels), and detecting anomalous query patterns.

An attack on the training pipeline where an adversary injects corrupted or malicious samples into the training dataset to compromise the model's future behavior. This is a causative attack, affecting the model from its foundation.

• Types: Includes backdoor attacks (trigger-specific misbehavior) and availability attacks (general performance degradation).
• Contrast with MIA: While MIA is an inference-time privacy leak, data poisoning is a training-time integrity attack. A poisoned model may have higher susceptibility to membership inference due to memorization of poisoned points.
• Defense: Data provenance tracking, robust statistics for outlier detection during training, and data sanitization.

A rigorous mathematical framework for quantifying and limiting privacy loss when releasing information about a dataset. It is the gold-standard defense against membership inference and model inversion attacks.

• Mechanism: Guarantees that the inclusion or exclusion of any single individual's data has a negligible statistical impact on the algorithm's output. This is typically achieved by adding calibrated noise.
• Application to ML: Can be applied during training (Differentially Private Stochastic Gradient Descent) or to the final model's outputs.
• Trade-off: Provides a provable privacy guarantee but often introduces a utility loss, requiring a careful privacy-accuracy trade-off.

A fundamental machine learning phenomenon where a model learns patterns specific to the training data (including noise) rather than generalizable features. This is the primary statistical vulnerability exploited by membership inference attacks.

• Why it enables MIA: An overfitted model exhibits a significant performance gap between its training and test data. It often assigns higher confidence or lower loss to memorized training points, which an attacker's shadow model can detect.
• Mitigation: Techniques like regularization (L1/L2), dropout, early stopping, and using more training data reduce overfitting, thereby increasing resistance to MIA.

The core reconnaissance technique used in many practical membership inference attacks. The attacker trains multiple 'shadow' models on datasets they control to mimic the behavior of the target model.

• Process: 1) Assemble datasets similar to the target's presumed training data. 2) Train many shadow models, recording each data point's loss/confidence and its membership status (in/out of that model's training set). 3) Use this labeled data to train a binary attack classifier that predicts membership based on loss/confidence.
• Significance: This method transforms the black-box MIA problem into a standard supervised learning task, enabling highly accurate attacks without internal model knowledge.