Differential Privacy

The strength of the privacy guarantee is mathematically quantified by the privacy budget or epsilon (ε). A smaller ε provides stronger privacy but typically reduces the utility (accuracy) of the output. The guarantee states that the probability of any output is nearly the same whether any single individual's data is included or excluded from the dataset. This makes membership in the dataset statistically indistinguishable.

Differential privacy is achieved by introducing controlled randomness into the computation. A deterministic query (like a count or average) is transformed into a randomized algorithm (the mechanism). Common mechanisms include:

• Laplace Mechanism: Adds noise drawn from a Laplace distribution to numeric queries (e.g., counts, sums).
• Exponential Mechanism: Used for non-numeric outputs (e.g., selecting the best option from a set) by randomly choosing an outcome with probability proportional to its utility score.

Differential privacy guarantees compose, meaning the privacy cost of multiple analyses can be precisely tracked and bounded.

• Sequential Composition: If you run multiple differentially private mechanisms on the same data, their epsilon values add up. Running a mechanism with ε=0.1 ten times yields a total ε=1.0.
• Parallel Composition: If mechanisms are run on disjoint subsets of the data, the overall privacy cost is only the maximum ε used, not the sum. This is crucial for designing efficient, large-scale analyses.

Any computation performed on the output of a differentially private mechanism, without accessing the original raw data, cannot weaken the privacy guarantee. If an output is (ε)-differentially private, then any function of that output is also (ε)-differentially private. This allows for safe downstream analysis, aggregation, or publication of the privatized results without requiring new privacy budgets.

The differential privacy guarantee holds regardless of an adversary's prior knowledge or auxiliary information. Even if an attacker knows every other record in the dataset perfectly, the mechanism still prevents them from confidently determining if a specific individual's data is present. This makes it a strong defense against linkage attacks, where separate data sources are combined to re-identify individuals.

Differential privacy can be implemented in two primary architectures relevant to TinyML:

• Central Model: A trusted curator collects raw data, applies the DP mechanism, and releases the result. Used in federated learning aggregators.
• Local Model: Each individual device (e.g., a sensor or phone) randomizes their own data before sending it to an untrusted server. This provides a stronger, device-level guarantee but often requires more noise, reducing utility. It's foundational for privacy-preserving data collection in IoT.

In Differential Privacy, epsilon (ε) is the core privacy loss parameter that quantifies the strength of the privacy guarantee. A smaller ε provides stronger privacy (more noise), while a larger ε allows greater accuracy (less noise) at the cost of increased privacy risk.

• Interpretation: ε bounds the log-likelihood ratio of any output occurring with or without an individual's data. An ε of 0.1 is considered very strong privacy; an ε of 10 is relatively weak.
• Management: The privacy budget is consumed each time a query is answered on a dataset. Once depleted, no further queries can be answered without violating the guarantee, necessitating careful composition analysis.