Differential Privacy Defense

The primary method for achieving (ε, δ)-differential privacy for numerical queries. It works by adding calibrated Gaussian noise to the true output of a function.

• Sensitivity (L2): The maximum change in the function's output when one record is added or removed.
• Scale: Noise is scaled proportionally to sensitivity / ε.
• This mechanism is fundamental to DP-SGD, the algorithm used to train private deep learning models.

The primary threat that differential privacy defends against. An attacker determines whether a specific data record was part of the model's training set by analyzing statistical differences in the model's confidence scores or loss values. Differential privacy provides a mathematical guarantee that the output distribution is nearly identical regardless of an individual's inclusion, directly limiting the attacker's advantage. Without formal privacy bounds, models can memorize and leak rare training examples.

A privacy attack that reconstructs representative features or entire training samples from a model's parameters or prediction API. For example, an attacker can generate a recognizable image of a person's face from a facial recognition model. Differential privacy bounds the information leakage from individual training points, making it statistically impossible to reconstruct specific inputs. This defense is critical for models trained on sensitive medical or financial data.

An integrity attack where adversaries inject malicious samples into the training dataset to implant backdoors or degrade model performance. While differential privacy primarily addresses confidentiality, its bounding mechanism can limit the influence of any single poisoned example, providing a secondary defense. However, strong poisoning attacks that corrupt many samples may still succeed, requiring dedicated defenses like robust aggregation and data sanitization.

A defensive technique that augments training data with adversarial examples to improve model robustness against evasion attacks. When combined with differential privacy, the training process becomes both robust and private. This dual objective is known as private robust learning. The challenge lies in balancing the privacy budget with the adversarial perturbation budget, as both mechanisms add noise that can degrade clean accuracy.

A formal guarantee that a model's prediction remains constant for all inputs within a mathematically defined perturbation bound. Randomized smoothing is a leading technique that constructs certifiably robust classifiers by adding Gaussian noise. This approach shares a deep connection with differential privacy: both rely on randomized mechanisms to provide provable guarantees—one for security against adversarial inputs, the other for privacy of training data.

An attack that steals model functionality by querying a prediction API to train a substitute replica. Differential privacy limits the information leaked per query, making extraction exponentially harder. By bounding the sensitivity of outputs to individual training points, an attacker cannot efficiently reconstruct the decision boundary. This defense is essential for protecting proprietary models deployed behind public APIs from intellectual property theft.