Amnesiac Unlearning

The primary algorithmic driver of amnesiac unlearning. Instead of minimizing the loss on target data (gradient descent), the model maximizes the loss. This effectively reverses the learning process for the forgotten data points.

• Mechanism: Adds the gradient of the loss on the 'forget set' to the model weights.
• Trade-off: Computationally cheap but can catastrophically damage model utility if the step size is too large.
• Use Case: Rapid, approximate scrubbing of specific documents.

A second-order statistical measure crucial for precise amnesiac unlearning. It quantifies how much information a data point carries about a model parameter.

• Function: Identifies which weights are most responsible for memorizing the target data.
• Application: Enables targeted perturbation—adding noise specifically to high-information weights rather than uniformly disturbing the entire network.
• Benefit: Minimizes collateral damage to performance on the retained data set.

A statistical tool that answers the counterfactual question: 'What would happen if this training point were removed?' without actually retraining the model.

• Mechanism: Uses gradient and Hessian-vector products to estimate parameter changes.
• Role in Unlearning: Identifies the exact weight delta required to simulate the absence of data.
• Limitation: Assumes a convex loss landscape, which is often violated in deep neural networks, making it an approximation rather than an exact solution.

The primary risk factor in amnesiac unlearning. This phenomenon occurs when a neural network abruptly loses previously acquired knowledge upon learning new information or, in this context, forcibly erasing old information.

• The Balancing Act: Amnesiac algorithms must induce selective forgetting without triggering a cascade of unintended memory loss.
• Mitigation: Elastic Weight Consolidation (EWC) and other continual learning strategies are often integrated to protect critical weights from being corrupted during the unlearning perturbation.

Provides the mathematical framework for certified removal. By training with differential privacy (DP-SGD), the model's weights become statistically indistinguishable from a model trained without any single data point.

• Connection: Amnesiac unlearning can be achieved by simply removing the 'privacy budget' (epsilon) allocated to a specific data point.
• Guarantee: Offers a formal proof that the deleted data's influence is bounded, moving beyond heuristic approximation to verifiable compliance.

The standard audit mechanism to verify amnesiac unlearning. An attacker (or auditor) attempts to determine if a specific record was part of the training set.

• Verification: If unlearning is successful, the attack's accuracy on the forgotten data should drop to random chance (50%).
• Methodology: Often uses shadow models trained on similar data distributions to train a binary classifier that detects subtle differences in model confidence scores between 'seen' and 'unseen' data.