Hard Attention to the Task (HAT)

HAT learns a hard, binary attention mask (values of 0 or 1) for each network unit (e.g., neuron, attention head) per task. A mask value of 1 allows the unit to be active and its parameters to be updated for that specific task. A value of 0 completely blocks gradient flow and activation for that unit, enforcing strict parameter isolation. This is in contrast to soft attention mechanisms that use continuous values, which can still allow small amounts of interference.

The collection of masks for a task defines a sparse, task-specific subnetwork within the larger, shared model architecture. Only the parameters under the 'active' mask positions are trainable for that task. This creates dedicated functional pathways for each task, preventing direct overwriting of weights important for other tasks. The model's capacity scales with the number of tasks, but the active parameter count per task remains a fraction of the total.

The core mechanism for preventing forgetting is gradient gating. During training on task k, the gradient for any parameter is multiplied by its binary mask for task k. If the mask is 0, the gradient is zeroed out.

• Key Result: Parameters allocated to previous tasks (mask=0 for current task) receive no gradient updates.
• This ensures their values remain frozen and unchanged, completely avoiding catastrophic interference at the parameter level. The loss function does not need complex regularization terms; isolation is enforced structurally.

To learn these binary masks, HAT employs a differentiable relaxation during training. A continuous, sigmoid-based mask is used, which is encouraged to become binary via a sparsity-inducing penalty term added to the task loss (e.g., L1 regularization on the mask values). This penalty pushes mask values toward 0 or 1. After training, masks are thresholded to exact binary values (e.g., >0.5 becomes 1) for inference, ensuring hard isolation.

At inference time, the task identity must be provided to select the correct pre-computed binary mask. The forward pass is then executed with only the subnetwork defined by that task's mask active. Units with a mask value of 0 have their outputs set to zero. This makes HAT a task-aware method, requiring a task ID or a separate task-inference mechanism during deployment. The computational cost is proportional to the sparsity of the active mask.

HAT is a fine-grained, neuron-level isolation method within a fixed network, contrasting with other architectural approaches:

• vs. Progressive Neural Networks: HAT expands capacity via masks within a single network, while Progressive Nets add entirely new columns, leading to faster parameter growth.
• vs. PackNet: Both perform parameter isolation, but PackNet uses iterative pruning and fine-tuning to free weights, whereas HAT learns masks concurrently with task training.
• vs. Regularization (EWC, SI): HAT provides absolute protection (zero gradient) for old tasks, while regularization methods only slow down parameter changes, potentially allowing gradual drift over many tasks.

Parameter Isolation is a family of architectural continual learning strategies that allocate distinct, non-overlapping subsets of a model's parameters to different tasks. HAT is a prominent example, using binary attention masks. This approach completely avoids interference and catastrophic forgetting by design, as gradients for a new task only flow to its assigned parameters.

• Core Mechanism: Dedicated capacity per task.
• Key Benefit: Zero forgetting by construction.
• Trade-off: Parameter efficiency decreases as tasks increase, potentially leading to model growth.

Architectural Methods are a category of continual learning strategies that dynamically modify the neural network's structure to accommodate new knowledge. This includes:

• Expansion: Adding new neurons or layers (e.g., Progressive Neural Networks).
• Masking: Sparsely activating sub-networks (e.g., HAT).
• Routing: Dynamically selecting pathways through the network.

The core principle is to allocate dedicated physical or logical capacity for new tasks, thereby preventing parameter interference with old tasks. These methods often trade off forgetting for increased model complexity.

Regularization-Based Methods mitigate catastrophic forgetting by adding penalty terms to the loss function that constrain how much important parameters for previous tasks are allowed to change. Unlike HAT's hard isolation, these are soft constraints.

• Elastic Weight Consolidation (EWC): Uses the Fisher Information Matrix to estimate parameter importance.
• Synaptic Intelligence (SI): Computes online importance weights based on cumulative parameter updates.
• Learning without Forgetting (LwF): Uses knowledge distillation as a regularization signal.

These methods aim for parameter efficiency but can struggle with long task sequences due to accumulating constraints.

Replay-Based Methods prevent forgetting by storing a subset of past data (or generating synthetic versions) and interleaving this 'replayed' data with new task data during training. This approximates the i.i.d. data assumption of standard offline learning.

• Experience Replay (ER): Stores raw examples in a fixed-size memory buffer.
• Generative Replay: Uses a generative model (e.g., GAN) to produce synthetic past data.
• Dark Experience Replay (DER): Stores past model logits ('dark knowledge') for a consistency loss.

These methods are often combined with other techniques but introduce memory overhead and potential privacy concerns.

The Stability-Plasticity Dilemma is the fundamental challenge in continual learning and adaptive systems. It describes the tension between two competing needs:

• Stability: The ability to retain consolidated knowledge from past experiences (resisting forgetting).
• Plasticity: The capacity to integrate new information and adapt flexibly to novel tasks.

All continual learning algorithms, including HAT, represent a specific engineering trade-off on this spectrum. HAT strongly favors stability for old tasks (via hard masking) while allowing full plasticity within the unmasked parameters for the new task.

Catastrophic Forgetting is the core problem addressed by continual learning. It is the tendency of a neural network to abruptly and drastically lose performance on previously learned tasks when it is trained on new, non-i.i.d. data. This occurs because gradient-based optimization overwrites weights that were critical for old tasks when minimizing loss for the new task.

• Cause: Non-stationary data distributions and overlapping parameter use.
• Contrast with HAT: HAT's binary masks create a hard boundary, preventing gradient flow to old task parameters and thus eliminating this interference by design.