Privacy Budget (Epsilon Budget)

The privacy budget is a strictly limited resource that is consumed with every differentially private query or model update. Each access to the data incurs a specific privacy loss (ε) , and the cumulative sum of these losses cannot exceed a pre-defined global threshold. This is analogous to a financial budget: once the allocated epsilon is spent, the data vault must be sealed to maintain the mathematical guarantee. In a federated learning context, this requires careful accounting across thousands of training rounds, as each round's model update consumes a fraction of the total budget.

A fundamental tension exists between the privacy budget (ε) and model accuracy. A smaller epsilon (e.g., ε = 0.1) injects more noise into the computation, providing a stronger privacy guarantee but degrading the utility of the resulting model or statistic. A larger epsilon (e.g., ε = 8) adds less noise, preserving utility but weakening the privacy protection. The art of differential privacy lies in finding an epsilon value that provides a meaningful guarantee while maintaining clinically acceptable performance. This trade-off must be evaluated using metrics like Federated AUC and Expected Calibration Error.

A privacy budget can be allocated using different strategies. A uniform allocation assigns an equal fraction of the total epsilon to each query. A more sophisticated adaptive allocation reserves budget for high-priority queries or later training rounds where gradients are more sensitive. In federated model evaluation, a data scientist might allocate a larger portion of the budget to computing a federated confusion matrix for a critical fairness audit, while using a smaller portion for routine performance monitoring. The privacy accountant module enforces these limits programmatically.

The standard definition of differential privacy is (ε, δ)-differential privacy. Epsilon (ε) is the privacy loss parameter, bounding the multiplicative difference in output probabilities when a single record is added or removed. Delta (δ) is a failure probability that allows for a small, negligible chance that the pure ε-guarantee is violated. A typical target for δ is to be much smaller than 1/N, where N is the dataset size, ensuring that the mechanism does not catastrophically fail by revealing a whole record. In practice, δ is often set to 10⁻⁵ or smaller.

When the cumulative privacy loss reaches the pre-defined global budget, the system must enforce a hard data lockout. No further queries or training rounds can be executed on the sensitive data partition. This is a critical architectural constraint for continuous model learning systems in healthcare. To mitigate this, techniques like privacy budget recycling (resetting the budget after a fixed time window) or privacy amplification by subsampling (where only a random subset of data is used per round) are employed to extend the useful life of the data access without weakening the overall guarantee.

The mathematical framework that provides a quantifiable guarantee that the output of a computation reveals no information about any single individual's participation. The privacy budget (ε) is the core parameter controlling the privacy-utility tradeoff: lower epsilon values enforce stronger privacy by adding more noise, while higher values preserve greater accuracy. DP is the foundational concept upon which the entire privacy budget paradigm is built.

Formal rules governing how privacy loss accumulates across multiple queries or training rounds. Basic composition states that the total epsilon is the sum of individual epsilons (ε_total = ε_1 + ε_2 + ... + ε_k). Advanced composition provides tighter bounds using Gaussian mechanisms, reducing total privacy loss to roughly O(√k · ε) for k queries. These theorems are essential for tracking cumulative expenditure against a fixed privacy budget over the entire lifecycle of a federated learning system.

A random variable quantifying the actual privacy loss incurred by a specific mechanism execution, beyond the worst-case epsilon guarantee. Privacy loss accounting tracks this distribution to provide tighter, empirically measured bounds rather than relying solely on theoretical upper limits. Techniques like Rényi Differential Privacy (RDP) and zero-concentrated DP (zCDP) use moments of the privacy loss distribution to enable more accurate composition accounting, allowing more queries before exhausting the budget.

The point at which the cumulative epsilon expenditure reaches the pre-defined threshold, after which no further queries or training rounds are permitted on the sensitive dataset. Strategies to manage depletion include:

• Privacy budget allocation: reserving portions for different query types
• Privacy amplification by subsampling: randomly selecting data subsets to reduce per-query epsilon cost
• Budget refreshing: introducing new data or rotating cohorts to reset the budget Depletion is irreversible without architectural changes.

A relaxation of standard differential privacy based on the Rényi divergence between output distributions on neighboring datasets. RDP provides tighter composition bounds than basic DP, enabling more accurate tracking of cumulative privacy loss across many training rounds. Key properties:

• Parameterized by order α > 1, with α → ∞ recovering pure ε-DP
• Supports tight sequential composition without advanced composition approximations
• Convertible to standard (ε, δ)-DP guarantees for final reporting RDP is widely used in production federated learning systems for precise budget accounting.