Output Perturbation

The core mechanism involves adding Gaussian or Laplacian noise to the model's output confidence scores (logits or softmax vectors). The noise scale is calibrated to the sensitivity of the output function—the maximum change in the prediction vector caused by adding or removing a single training record. This directly obfuscates the subtle overconfidence signals that black-box attacks rely on to distinguish members from non-members.

This defense operates on a fundamental spectrum controlled by a privacy budget (epsilon). A lower epsilon adds more noise, providing stronger membership privacy but degrading the model's prediction accuracy. The challenge is finding the optimal point where the model remains useful for its primary task while the output distributions for training and non-training data become statistically indistinguishable.

Unlike DP-SGD, which modifies the training process, output perturbation is applied after training is complete. It acts as a lightweight wrapper around the inference API. This makes it highly compatible with existing, already-trained models and complex architectures where retraining is prohibitively expensive, providing an immediate layer of defense against label-only and gap attacks.

A common implementation variant is to only release the top-K predicted classes and their perturbed scores, discarding the rest of the confidence vector. This directly counters attacks that analyze the distribution of low-probability classes. By limiting the information surface, the model reveals only the most necessary output, significantly reducing the signal available for a likelihood ratio attack.

When the noise is sampled from a distribution calibrated to the output's sensitivity, output perturbation can satisfy the mathematical definition of epsilon-differential privacy. This provides a provable guarantee that an adversary cannot confidently infer membership from the output, transforming the defense from a heuristic into a verifiable privacy property suitable for regulated environments.

The mathematical framework that provides provable privacy guarantees by injecting calibrated statistical noise into computations. It ensures the output distribution is nearly identical whether or not any single individual's data is included in the input.

• Parameterized by epsilon (ε) , the privacy loss parameter
• Lower epsilon values indicate stronger privacy but reduced utility
• Forms the theoretical basis for output perturbation mechanisms
• Provides composability: privacy loss accumulates predictably across multiple queries

A fundamental differential privacy mechanism that achieves privacy by adding noise drawn from a Gaussian (normal) distribution calibrated to the L2 sensitivity of the query function.

• Noise scale: σ = (Δ₂f · √(2·ln(1.25/δ))) / ε
• Requires the relaxation parameter δ for approximate differential privacy
• Commonly used in DP-SGD for gradient perturbation
• Provides tighter composition under Rényi DP accounting

A foundational differential privacy mechanism that adds noise drawn from a Laplace distribution to query outputs. It is the optimal mechanism for pure ε-differential privacy when protecting scalar-valued queries.

• Noise scale: b = Δ₁f / ε where Δ₁f is L1 sensitivity
• Provides pure DP without requiring the δ relaxation parameter
• Ideal for simple count queries and histogram releases
• Less efficient than Gaussian for high-dimensional vector outputs

A quantifiable limit on the total privacy loss permitted over a series of differentially private operations. Each query or training step consumes a portion of this budget, and once exhausted, no further access to sensitive statistics is allowed.

• Tracked via privacy accounting mechanisms like the Moments Accountant
• Sequential composition: total ε = ε₁ + ε₂ + ... + εₙ
• Parallel composition: total ε = max(εᵢ) for disjoint datasets
• Budget exhaustion triggers mandatory access denial

A lightweight defense that truncates or rounds the model's output confidence scores to limited precision. By revealing only the top-K predictions or coarsening probability values, it reduces the information leakage exploited by membership inference attacks.

• Top-K masking: reveal only the K most likely classes
• Precision reduction: round scores to 2-3 decimal places
• Lower computational overhead than full differential privacy
• Provides heuristic rather than provable protection

The process of determining the maximum change in a function's output caused by adding or removing a single record from the input dataset. This parameter is critical for calibrating the magnitude of noise required in any differential privacy mechanism.

• L1 sensitivity: maximum absolute difference in L1 norm
• L2 sensitivity: maximum Euclidean distance between outputs
• Functions with unbounded sensitivity cannot be made differentially private
• Clipping is used to bound sensitivity in practice