Differential Privacy (DP)

The fundamental parameter that quantifies the privacy loss in a differentially private mechanism. A smaller epsilon (e.g., ε = 0.1) provides stronger privacy guarantees by limiting the maximum divergence between outputs on neighboring datasets. In practice, healthcare deployments typically operate in the range of ε = 1 to ε = 8, balancing utility against the formal guarantee. The budget is composable—each query against the data consumes a fraction of the total budget, requiring careful accounting across the entire model training lifecycle.

Sensitivity measures the maximum impact a single record can have on a query's output. Global sensitivity (L1 or L2 norm) determines how much noise must be added to achieve a target epsilon. Key mechanisms include:

• Laplace Mechanism: Adds noise calibrated to L1 sensitivity for numerical queries
• Gaussian Mechanism: Uses L2 sensitivity, common in DP-SGD for deep learning
• Exponential Mechanism: For non-numeric outputs, selects results probabilistically based on a utility score The noise scale is directly proportional to sensitivity divided by epsilon—higher sensitivity demands more noise.

The workhorse algorithm for training deep learning models with differential privacy guarantees. DP-SGD modifies standard SGD through two operations at each training step:

• Gradient Clipping: Bounds the L2 norm of per-example gradients to a threshold C, enforcing a finite sensitivity
• Noise Addition: Injects calibrated Gaussian noise into the aggregated gradient before the model update This ensures the final model parameters do not memorize individual training examples. The moments accountant tracks cumulative privacy loss across iterations, providing a tight (ε, δ)-differential privacy guarantee.

Two deployment architectures with distinct trust models:

• Central DP (CDP): A trusted curator collects raw data, applies the privacy mechanism, and releases sanitized outputs. Provides higher utility at a given epsilon but requires trust in the central server
• Local DP (LDP): Each user randomizes their data before transmission. No trusted curator is needed, but utility degrades significantly—often requiring ε values 10-100x larger than CDP for equivalent accuracy In federated healthcare, cross-silo CDP is standard, as hospitals act as trusted curators for their own patient populations before sharing model updates.

A relaxation of pure ε-differential privacy based on Rényi divergence rather than max divergence. RDP provides tighter composition bounds than the standard strong composition theorem, making it the preferred accounting method for DP-SGD. Key properties:

• Parameterized by order α > 1, with α → ∞ recovering pure DP
• Supports subsampled mechanisms natively, critical for minibatch SGD
• Converts to standard (ε, δ)-DP via a tight conversion lemma RDP is implemented in major privacy libraries including Opacus and TF Privacy.

A phenomenon where randomly sampling a subset of data before applying a differentially private mechanism amplifies the privacy guarantee. In DP-SGD, each minibatch is a random sample from the full dataset. The probability that any specific record is included is q = B/N (batch size / dataset size). This sampling introduces uncertainty about whether a given record was even processed, effectively multiplying the privacy budget by q². This is why larger datasets and smaller batch sizes yield stronger privacy at the same noise level—a critical design consideration for federated hospital networks with varying cohort sizes.