Privacy Budget (Epsilon)

Epsilon (ε) is a non-negative real number that bounds the privacy loss—the maximum factor by which an output's probability can shift due to the inclusion or exclusion of a single record. A smaller ε enforces stronger privacy because the output distributions become nearly indistinguishable. For example, ε = 0.1 provides a much stricter guarantee than ε = 10. In practice, ε values between 0.1 and 8 are common, with ε < 1 considered strong privacy and ε > 8 considered weak. The parameter directly controls the noise scale in mechanisms like the Gaussian or Laplace mechanism.

Delta (δ) is a relaxation parameter in (ε, δ)-differential privacy that allows a small probability of catastrophic privacy failure. While ε bounds the typical privacy loss, δ represents the probability that the guarantee is violated entirely. Typical values are δ < 1/n, where n is the dataset size, ensuring the failure probability is smaller than the chance of randomly selecting any individual record. This relaxation enables more efficient algorithms like the Gaussian mechanism and is essential for practical deployments like DP-SGD.

When multiple differentially private computations are performed on the same dataset, their privacy budgets add together. If you run two queries with budgets ε₁ and ε₂, the total privacy cost is ε₁ + ε₂. This property is fundamental to privacy accounting—tracking cumulative expenditure across training steps. For example, if DP-SGD consumes ε = 0.5 per epoch over 10 epochs, the total budget is ε = 5. This additive nature forces practitioners to carefully allocate their budget across queries or training iterations.

Any computation applied to the output of a differentially private mechanism cannot weaken the privacy guarantee. Once noise is injected to satisfy ε-differential privacy, subsequent transformations—sorting, filtering, visualization, or even training another model—do not consume additional budget or degrade privacy. This property is crucial because it allows data analysts to freely explore and process privatized outputs without fear of inadvertently leaking more information. The guarantee holds regardless of the complexity of the post-processing function.

Randomly sampling a subset of records before applying a differentially private mechanism amplifies the privacy guarantee. If each record is included with probability q, the effective ε is reduced by approximately a factor of q. This is why DP-SGD with Poisson sampling achieves much tighter privacy bounds than processing the full dataset. For instance, subsampling 1% of records can reduce the privacy cost by two orders of magnitude, enabling practical deep learning with strong formal guarantees.

The privacy budget directly controls the fundamental tension between privacy and accuracy. Reducing ε requires injecting more noise, which degrades model performance. This trade-off is not linear—utility often drops sharply below a problem-dependent threshold. Key considerations include:

• Dataset size: Larger datasets tolerate smaller ε with less utility loss
• Signal strength: Tasks with strong patterns survive higher noise
• Model capacity: Overparameterized models may memorize less, requiring less noise Practitioners must empirically evaluate this trade-off for their specific use case.

The mathematical framework that gives epsilon its meaning. DP provides a provable guarantee that the output of a computation is nearly indistinguishable whether or not any single individual's data is included.

• Pure ε-DP: The strictest form, bounded by a single epsilon value
• (ε, δ)-Approximate DP: Relaxes the guarantee by allowing a small failure probability δ
• The privacy loss random variable quantifies the actual leakage in a given run

A relaxation of pure DP based on Rényi divergence that provides significantly tighter composition bounds for iterative algorithms like DP-SGD.

• Parameterized by order α > 1, measuring divergence at different scales
• Converts to (ε, δ)-DP via a clean analytical formula
• Enables more efficient use of the privacy budget compared to naive composition

The workhorse algorithm that operationalizes epsilon in deep learning. Differentially Private Stochastic Gradient Descent clips per-sample gradients and injects calibrated Gaussian noise.

• Per-sample clipping: Bounds the sensitivity of each individual contribution
• Noise multiplier σ: Derived from the target (ε, δ) and the number of training steps
• The privacy budget is consumed incrementally with each noisy gradient update