Catastrophic Forgetting

The fundamental tension at the heart of catastrophic forgetting. Plasticity is the network's ability to adapt to new data by updating weights. Stability is the ability to retain previously learned mappings. When a gradient descent step minimizes loss on a new task, it often moves weights into a region of parameter space that is catastrophic for a prior task. The network exhibits excessive plasticity at the expense of stability, causing the loss landscape of the old task to spike sharply.

In dense neural networks, individual weights participate in representing multiple distinct concepts. When a weight crucial for recognizing Feature A (old task) is also the most convenient parameter to adjust for learning Feature B (new task), the update erases the representation of Feature A. This is not a gradual decay but a sudden, destructive interference pattern where the new gradient directly cancels out the protective configuration of the old weights.

In sequential learning, the internal feature representations in hidden layers shift to accommodate new data distributions. The network's embedding space reorganizes itself entirely around the new task, causing the decision boundaries for old tasks to become nonsensical. Even if output weights are frozen, the shared feature extractor drifts so far that the old classification logic no longer maps to the correct regions of the latent space.

Visualizing the parameter space as a landscape, a trained model sits in a low-loss basin for Task A. Learning Task B requires navigating to a new basin. Without constraints, standard SGD will follow the steepest descent path out of Task A's basin and into Task B's, with no mechanism to return. The old basin is effectively erased from the optimization trajectory, and the model suffers a complete performance collapse on Task A.

Biological brains avoid catastrophic forgetting partly through memory consolidation and replay during sleep. Standard neural networks lack this interleaved rehearsal. When training is strictly sequential, the model never revisits old data samples. Without explicit replay buffers or generative pseudo-rehearsal, the network has zero opportunity to maintain the joint distribution and will catastrophically forget the first task upon mastering the second.

In the context of machine unlearning, catastrophic forgetting is weaponized intentionally but must be surgically precise. A naive unlearning algorithm might apply gradient ascent on target data, but the destructive weight interference often spills over into adjacent, non-targeted concepts. The goal is to induce localized catastrophic forgetting of specific data points without triggering a global collapse of the model's utility on the retained distribution.

A continual learning algorithm that mitigates catastrophic forgetting by identifying and protecting parameters critical to previously learned tasks. It calculates the Fisher Information Matrix to approximate the importance of each weight, then applies a quadratic penalty to slow learning on those weights during new task training.

• Preserves old knowledge without storing past data
• Balances stability (retention) and plasticity (new learning)
• Foundational for controlled, non-catastrophic forgetting in sequential learning

An approximate unlearning technique that reverses the standard gradient descent process to erase specific data influence. Instead of minimizing loss on target data, the algorithm maximizes loss by adding the gradient, effectively scrubbing the data's contribution from model weights.

• Computationally cheaper than full retraining
• Risk of degrading overall model utility if over-applied
• Often paired with knowledge distillation to preserve general performance

Sharded, Isolated, Sliced, and Aggregated training partitions data into disjoint shards, each training an independent constituent model. When a deletion request arrives, only the shard containing the target data requires retraining.

• Limits the blast radius of unlearning operations
• Reduces retraining cost from O(N) to O(N/k) where k is shard count
• Enables exact unlearning guarantees within each shard

A statistical tool that quantifies how removing or upweighting a single training point would change model parameters and predictions, without retraining. Uses second-order gradient information to estimate counterfactual parameter shifts.

• Identifies which data points most affect specific predictions
• Enables targeted unlearning by subtracting estimated influence
• Computationally intensive for large models due to Hessian calculations

A structural unlearning mechanism that identifies and removes individual neurons or connections disproportionately activated by target data. By analyzing activation patterns on the data to be forgotten, specific network pathways are surgically ablated.

• Produces permanent, verifiable forgetting
• May create performance gaps requiring fine-tuning
• Complements weight-space methods like gradient ascent

A mathematical framework providing provable bounds on information leakage. In unlearning, certified removal leverages differential privacy to guarantee that the influence of deleted data points remains within a provable epsilon threshold.

• Formalizes the privacy-utility tradeoff via the epsilon budget
• Enables auditable proof of removal claims
• Often combined with SISA for practical deployment