K-Anonymity

A mathematical framework providing a provable privacy guarantee by injecting calibrated statistical noise into query responses or model outputs. Unlike k-anonymity, which focuses on syntactic grouping of records, differential privacy offers a semantic guarantee: the presence or absence of any single individual in a dataset is indistinguishable within a privacy budget parameterized by epsilon (ε).

• Mechanism: Laplace or Gaussian noise addition
• Key metric: Privacy loss parameter ε (lower = stronger privacy)
• Composition: Privacy budget consumed across multiple queries
• Advantage over k-anonymity: Resistant to linkage attacks using auxiliary data

A privacy attack where an adversary determines whether a specific data record was included in a model's training dataset. Attackers exploit subtle differences in model confidence scores, loss values, or output distributions between members and non-members. This attack directly threatens the re-identification prevention that k-anonymity aims to provide.

• Shadow model technique: Train surrogate models to mimic target behavior
• Threshold-based inference: Compare prediction confidence against calibrated thresholds
• Defense: Differential privacy, model regularization, and output perturbation
• Risk: Violates data minimization principles and can expose sensitive training data

A privacy breach where an attacker reconstructs representative samples of a target class by iteratively querying a model and optimizing inputs to maximize class confidence scores. While k-anonymity protects tabular data releases, model inversion targets the model itself as the leakage vector.

• Gradient-based inversion: Backpropagate through model to reconstruct inputs
• Confidence score exploitation: Use softmax outputs as optimization signal
• Face reconstruction: Classic attack recovers recognizable facial images from classifiers
• Mitigation: Output vector truncation, differential privacy during training, and limiting query access

A cryptographic scheme enabling computation directly on encrypted data without requiring decryption. While k-anonymity focuses on data release privacy, homomorphic encryption protects data during active processing. The result of encrypted computation, when decrypted, matches the output of operations performed on plaintext.

• Partially Homomorphic (PHE): Supports only addition or multiplication
• Somewhat Homomorphic (SHE): Limited number of operations
• Fully Homomorphic (FHE): Arbitrary computation on ciphertexts
• Trade-off: Massive computational overhead (10,000x+ slowdown) vs. strong privacy guarantees

A decentralized training paradigm where models learn across multiple edge devices or servers holding local data without centralizing raw data. Model updates (gradients or weights) are shared instead of training examples. This architectural approach complements k-anonymity by keeping data distributed.

• Federated Averaging (FedAvg): Aggregate local model updates via weighted averaging
• Secure Aggregation: Cryptographic protocol hiding individual updates from the central server
• Gradient Leakage risk: Raw updates can still reveal private information
• Differential Privacy integration: Add noise to updates before aggregation for stronger guarantees

An integrity attack where adversaries contaminate training data with malicious samples to corrupt model behavior. While k-anonymity addresses privacy, data poisoning targets model reliability. Attackers can implant backdoors triggered by specific patterns or degrade overall model accuracy.

• Label flipping: Mislabel training examples to confuse decision boundaries
• Backdoor injection: Insert trigger patterns that cause targeted misclassification
• Clean-label poisoning: Use correctly labeled but subtly perturbed samples
• Defense: Robust statistics, anomaly detection on training data, and certified defenses against bounded poisoning budgets