Differential Privacy Budget

The epsilon (ε) value is the primary metric defining the strength of the privacy guarantee. A lower epsilon indicates stronger privacy but higher noise.

• ε = 0: Perfect privacy, zero utility (pure noise).
• ε = 0.1 to 1: Strong privacy, suitable for highly sensitive census data.
• ε = 1 to 10: Standard privacy range for enterprise analytics.
• ε > 10: Weak privacy, minimal noise added.

Selecting epsilon is a policy decision, not just a technical one.

The total privacy loss accumulates linearly when multiple queries are performed on the same dataset. This is the fundamental rule of budget exhaustion.

• Rule: If you run query A with budget ε₁ and query B with budget ε₂, the total cost is ε₁ + ε₂.
• Implication: A data scientist cannot run infinite queries; every analysis has a cost.
• Management: A privacy accountant tracks this cumulative spend to prevent exceeding the global ceiling.

When queries operate on disjoint, non-overlapping subsets of data, the privacy cost does not accumulate. The total budget consumed is the maximum of the individual queries.

• Rule: If query A runs on users 1-1000 and query B on users 1001-2000, total cost is max(ε₁, ε₂).
• Benefit: This allows for highly parallelized analytics without draining the budget.
• Constraint: Requires strict data partitioning and isolation guarantees.

Once an output is generated under a differential privacy guarantee, any further computation on that output cannot weaken the privacy protection.

• Property: A differentially private statistic remains private regardless of downstream mapping, filtering, or transformation.
• Example: If a noisy mean salary is released with ε=0.5, an analyst can safely round it, graph it, or feed it into a public model without increasing privacy risk.
• Caveat: This only applies to the released output, not the underlying raw data.

A privacy accountant is the algorithmic mechanism that tracks cumulative privacy loss (ε) and the probability of catastrophic failure (δ) across all queries in a system.

• Moments Accountant: A state-of-the-art technique used in deep learning (e.g., DP-SGD) that provides tighter bounds on total privacy loss than simple sequential composition.
• δ (delta): The failure probability—the chance that the pure ε-differential privacy guarantee is violated. Typically set to less than 1/N, where N is the dataset size.
• Role: Acts as a hard gatekeeper, rejecting queries that would exceed the pre-defined budget.

A privacy budget is a finite, non-renewable resource. Once exhausted, the data must be locked down to prevent reconstruction attacks.

• Depletion Strategy: Organizations define a global ε ceiling (e.g., ε=5 per year). When cumulative spend hits this limit, all access is frozen.
• Reset Mechanisms: Budgets can be reset by ingesting a completely new, independent dataset or by introducing a new cohort of users.
• Risk: Without strict depletion enforcement, an attacker can use differencing attacks across many queries to reconstruct individual records.

The core mathematical parameter that quantifies the privacy loss in differential privacy. A smaller epsilon (e.g., 0.1) provides stronger privacy guarantees by ensuring that the output of an analysis is nearly indistinguishable whether or not any single individual's data is included. As epsilon approaches zero, the guarantee approaches perfect secrecy. In practice, epsilon values between 0.1 and 10 are common, with values above 10 generally considered to provide weak privacy protection.

A relaxation parameter that allows the privacy guarantee to fail with probability δ. In (ε, δ)-differential privacy, delta represents the failure probability—the chance that the privacy loss bound defined by epsilon is exceeded. A cryptographically negligible delta (e.g., 10⁻⁶ or smaller) is standard. Setting δ = 0 yields pure differential privacy, while δ > 0 defines approximate differential privacy, which enables more efficient mechanisms like the Gaussian mechanism.

Formal mathematical results that quantify how the privacy budget accumulates across multiple queries or analyses on the same dataset. Basic composition states that the total epsilon is the sum of individual epsilons. Advanced composition provides a tighter bound, showing that the total privacy loss grows sublinearly (roughly proportional to the square root of the number of queries). These theorems are critical for tracking cumulative privacy expenditure over a system's lifetime.

A measure of the maximum possible change in a query's output when a single record is added to or removed from the dataset. Global sensitivity (L1 or L2) determines the amount of noise required to achieve a given privacy guarantee. A query with high sensitivity (e.g., counting queries on sparse data) requires more noise, consuming more budget. Reducing sensitivity through techniques like clipping or truncation is a key optimization strategy.

A phenomenon where randomly sampling a subset of data before analysis amplifies the privacy guarantee. Because an individual's record may not be included in the sample, the effective privacy loss is reduced. This technique is foundational to differentially private stochastic gradient descent (DP-SGD), where each training batch is a random subsample. The amplification effect allows training deep learning models with meaningful privacy budgets.