Privacy Budget (Epsilon)

Epsilon (ε) defines the privacy loss bound—the maximum factor by which an output's probability can shift due to a single record's inclusion or exclusion. Formally, a mechanism M satisfies ε-differential privacy if for all neighboring datasets D and D' differing by one record, and all output sets S: P[M(D) ∈ S] ≤ e^ε × P[M(D') ∈ S]. This multiplicative bound ensures that no single individual's data substantially influences the published result. A typical ε range for production systems is 0.1 to 8.0, with ε < 1 considered strong privacy and ε > 10 considered weak.

Every differentially private query or training step consumes a portion of the total privacy budget. Under basic sequential composition, running k mechanisms each with privacy loss ε_i results in a total privacy loss of Σ ε_i. Advanced composition theorems provide tighter bounds: for k mechanisms each with ε, the total loss is approximately O(ε√k). This means a model trained with DP-SGD for 10,000 steps at a per-step ε of 0.01 may consume a total budget of ε ≈ 1.0–2.0. Privacy accountants track this cumulative expenditure to halt computation before the budget is exhausted.

Pure ε-differential privacy can be overly restrictive for iterative algorithms. (ε, δ)-differential privacy introduces a small failure probability δ, where the ε guarantee holds with probability at least 1 − δ. Formally: P[M(D) ∈ S] ≤ e^ε × P[M(D') ∈ S] + δ. The δ parameter is typically chosen to be cryptographically small—much smaller than 1/N where N is the dataset size—ensuring that catastrophic privacy failures are astronomically unlikely. Common values: δ = 10^-5 to 10^-8.

When training with differentially private stochastic gradient descent (DP-SGD), randomly sampling a mini-batch rather than using the full dataset amplifies the privacy guarantee. The uncertainty of whether a specific record was included in the sampled batch provides an additional layer of deniability. This amplification means the per-step epsilon can be significantly larger than the effective privacy loss, enabling practical training while maintaining a tight total budget. The amplification factor depends on the sampling rate q = B/N.

Traditional moment accountant methods can overestimate privacy consumption. Rényi Differential Privacy (RDP) provides tighter composition bounds by tracking the privacy loss under Rényi divergence of order α. The RDP accountant computes the exact privacy curve across multiple α values, then converts to an (ε, δ)-DP guarantee at the end of training. This approach, implemented in libraries like Opacus and TF Privacy, can reduce the reported ε by 20–40% compared to basic composition, enabling more training iterations under the same budget.

Standard differential privacy protects a single record. If an individual contributes k records to a dataset (e.g., multiple medical visits), the effective privacy guarantee degrades. Group privacy states that if a mechanism is ε-DP for single records, it is k·ε-DP for groups of size k. This linear scaling means that protecting correlated records or entire user profiles requires a proportionally smaller per-record epsilon. Data architects must account for maximum user contribution when setting the global privacy budget.