Model Inversion Attack

The adversary's goal is not to extract raw database records but to reconstruct a representative class prototype or specific sensitive attributes. In a genomic context, this could mean generating a consensus sequence that reveals the average genetic markers of a rare disease cohort, effectively exposing group-level private traits without identifying a single individual.

• White-Box Attack: The adversary has full access to model weights and gradients. They can analytically compute the input that maximizes a target class score, leading to high-fidelity reconstructions.
• Black-Box Attack: The adversary can only query the model and observe confidence scores. Reconstruction is performed iteratively using optimization heuristics, which is slower but more realistic for API-based genomic prediction services.

The attack is critically enabled by overly granular model outputs. When a genomic classifier returns precise probability vectors (e.g., 0.97 likelihood of a pathogenic variant), the adversary uses gradient descent on the input space to maximize this score. The resulting optimized input reveals the model's internal representation of that class. Mitigation requires truncating confidence scores or returning only hard labels.

Genomic models are uniquely susceptible because DNA sequences are low-entropy and highly correlated. Unlike natural images, a reconstructed genomic sequence that is 80% accurate may still contain clinically actionable information. An attack on a federated variant-calling model could reconstruct population-specific allele frequencies, violating the privacy guarantees of the federated consortium.

The primary mathematical defense involves clipping gradient norms and injecting calibrated Gaussian noise during stochastic gradient descent. This bounds the influence of any single genome on the final model, making it statistically impossible to reconstruct a specific training sample. The privacy budget (ε) quantifies the trade-off between model utility and reconstruction risk.

A defensive architectural strategy where a complex 'teacher' model trained on sensitive data is used to train a simpler 'student' model on public or synthetic data only. The student model learns the decision boundaries without direct access to the original training gradients, significantly increasing the difficulty of a successful inversion attack on the deployed model.

A privacy attack that determines whether a specific data record was used in the training set of a machine learning model. Unlike model inversion, which reconstructs features, membership inference answers a binary question: was this sample present? Attackers train shadow models to mimic the target model's behavior and analyze prediction confidence scores, exploiting the fact that models often behave differently on data they have seen before. This is particularly dangerous in genomic contexts where mere membership in a disease-specific training cohort can reveal sensitive health information.

A mathematical framework that provides a provable guarantee of privacy by injecting calibrated statistical noise into query results or model updates. The core principle is that the output of a computation should be nearly indistinguishable whether or not any single individual's data is included. The privacy budget, denoted by epsilon (ε), quantifies the strength of the guarantee—lower values mean stronger privacy. In federated genomic learning, differentially private stochastic gradient descent (DP-SGD) clips and noises gradients before aggregation, directly mitigating model inversion risks.

A security attack where an adversary manipulates the training data on compromised clients to corrupt the global federated model. While model inversion extracts information post-training, data poisoning is a causative attack that introduces backdoors or degrades performance during training. In genomic settings, an adversary might inject subtly altered DNA sequences with specific motifs that cause the model to misclassify pathogenic variants when a trigger pattern is present, creating a dangerous diagnostic backdoor.

A privacy-preserving protocol in federated learning that allows a central server to compute the sum of model updates from multiple clients without inspecting any individual client's contribution. Using cryptographic techniques like secret sharing, the server only sees the aggregated result, making model inversion attacks on individual client data significantly harder. However, secure aggregation alone does not prevent an honest-but-curious server from attempting inversion on the final aggregated model, necessitating layered defenses.

A specific class of privacy attack where an adversary reconstructs private training data directly from the gradient updates transmitted during distributed training. Unlike model inversion, which queries a final trained model, gradient leakage exploits the shared gradients themselves. In deep learning, gradients with respect to the input layer can be mathematically inverted to reveal pixel-perfect reconstructions of training images or token sequences. For genomic models, this could expose raw sequence fragments from participating institutions.

A privacy attack that infers sensitive attributes about individuals in the training data by exploiting correlations learned by the model. While model inversion reconstructs representative features of a class, attribute inference targets specific, often non-obvious characteristics—such as inferring a patient's ethnicity or disease status from a model trained on genomic and phenotypic data. This attack leverages the model's ability to learn latent correlations that may not be apparent to data custodians.