Machine Unlearning

The fundamental dichotomy in unlearning. Exact unlearning guarantees the resulting model distribution is identical to one trained without the target data, often achieved through retraining on sharded data (SISA). Approximate unlearning reduces data influence to a statistically negligible bound without perfect removal, using techniques like gradient ascent or weight perturbation. The choice hinges on regulatory requirements versus computational budget.

An acronym for Sharded, Isolated, Sliced, and Aggregated training. This architectural pattern partitions the training dataset into disjoint, non-overlapping shards, each training an independent constituent model. When a deletion request arrives, only the shard containing the target data requires retraining. This dramatically limits the scope of exact unlearning, reducing computational cost from O(N) to O(N/S) where S is the number of shards.

An approximate unlearning technique that reverses the standard gradient descent optimization process. Instead of minimizing the loss on target data, the algorithm adds the loss gradient of the forgotten data back to the model weights. This effectively erases the data's contribution by moving parameters in the opposite direction of the original learning trajectory. It is computationally cheap but can damage overall model utility if applied aggressively.

A statistical method that quantifies how removing or upweighting a single training point would change a model's learned parameters, without requiring any retraining. By computing the Hessian-vector product of the loss function, influence functions identify which training examples are most responsible for a given prediction. This serves as both a pre-unlearning diagnostic tool and a post-unlearning verification mechanism.

The primary empirical audit for unlearning efficacy. A membership inference attack determines whether a specific data record was part of a model's training set by analyzing prediction confidence, loss values, or shadow model behavior. Post-unlearning, a successful defense against these attacks indicates that the target data's influence has been sufficiently removed, providing a practical, non-cryptographic validation layer.

The gold standard of data removal that guarantees a model distribution identical to one trained without the target data from scratch. Unlike approximate methods, exact unlearning provides a mathematical proof of complete removal, often achieved through SISA training or retraining only the affected data shards. This approach eliminates any residual influence of deleted data points on model weights, satisfying the strictest regulatory requirements.

A formal guarantee, grounded in differential privacy, that an unlearning algorithm has bounded the influence of deleted data within a provable mathematical threshold. Certified removal leverages the epsilon budget to quantify maximum information leakage, providing auditors with verifiable proof that unlearning was effective. This framework transforms unlearning from a best-effort process into a cryptographically sound compliance mechanism.

Sharded, Isolated, Sliced, and Aggregated training partitions data into mutually exclusive shards, each training an independent constituent model. When a deletion request arrives, only the shard containing the target data requires retraining. This architecture reduces unlearning cost from O(N) to O(N/S) where S is the number of shards, making exact unlearning computationally feasible for large-scale production systems.

An approximate unlearning technique that reverses the standard gradient descent process by adding the loss gradient of the target data back to the model weights. This effectively erases the data's contribution without full retraining. While computationally efficient, gradient ascent risks catastrophic forgetting on retained knowledge and requires careful learning rate calibration to avoid degrading overall model performance.

A privacy audit technique that determines whether a specific record was used in training by analyzing model confidence scores and prediction patterns. In unlearning verification, MIAs serve as the primary empirical test: a successfully unlearned model should show no statistical difference in behavior on deleted versus genuinely unseen data. Shadow models trained on similar distributions enable black-box auditing without direct model access.

Extends machine unlearning to federated learning settings where data never leaves client devices. When a participant invokes the right to be forgotten, the global model must remove their contribution without compromising other clients' privacy. Techniques include gradient ascent on historical updates, model rollback with secure aggregation, and differential privacy-based noise injection to mask the removal process.