Progressive Neural Networks

The core design principle is the instantiation of a new, task-specific neural network column for each new task. Each column is a complete, independent network (e.g., a multi-layer perceptron or CNN). Parameters from columns assigned to previous tasks are frozen permanently, providing an absolute guarantee against catastrophic forgetting for those tasks. This creates a growing, modular architecture where capacity scales with the number of tasks.

To enable forward transfer—where knowledge from past tasks improves learning on new ones—each new column receives lateral connections from all preceding columns. These are adaptor networks (typically small MLPs) that learn to transform and combine features from frozen columns. This mechanism allows the new task column to reuse useful representations without modifying the original, protected parameters, facilitating positive knowledge consolidation.

PNNs provide a structural solution to forgetting. By freezing the parameters of all columns associated with prior tasks, the model's performance on those tasks is mathematically locked and cannot degrade during subsequent training. This makes PNNs one of the most robust methods for zero forgetting in scenarios where retaining exact prior task performance is non-negotiable, such as in safety-critical systems.

The primary trade-off is in resource scaling. The model's total parameter count grows linearly with the number of tasks, as each new task adds a full column and a set of lateral adaptors. This leads to:

• Increased memory footprint for storing all frozen columns.
• Higher inference latency, as features from all previous columns may be routed via lateral connections.
• Inefficient parameter use, as columns are not shared or sparsely activated. This makes PNNs less suitable for long sequences of many tasks with limited hardware.

A key limitation of the vanilla PNN architecture is its inability to perform backward transfer. Since old columns are frozen, learning on a new task cannot refine or improve the model's performance on any previously learned task. The knowledge flow is strictly unidirectional (old → new). This contrasts with regularization or replay methods, where shared parameters can be subtly refined to improve performance on all tasks over time.

PNNs established a blueprint for parameter-isolation methods. They are well-suited for:

• Sequential multi-task learning with a known, finite task sequence.
• Lifelong learning benchmarks where forgetting must be eliminated.
• Robotics, where different skills (e.g., grasping, pushing) can be assigned to separate columns.

Modern variants like Progressive Nets with Compression address scalability by using techniques like knowledge distillation to compress old columns into a shared backbone, blending architectural and regularization approaches.

In reinforcement learning, PNNs enable an agent to master a series of Atari games (e.g., Pong, Breakout, Space Invaders). A new column is instantiated for each game's policy. Lateral connections from previous game columns provide a form of transfer learning, often accelerating mastery of the new game by reusing abstract features like ball tracking or enemy avoidance. The frozen parameters of old columns guarantee zero backward transfer, preserving perfect performance on earlier games.

A PNN can be deployed to diagnose a expanding set of medical conditions from imaging data (e.g., X-rays).

• Column 1: Trained to detect pneumonia.
• Column 2: Added and connected to Column 1 to learn COVID-19 detection, reusing general lung feature extractors.
• Column 3: Added to identify tuberculosis. This ensures the model's diagnostic accuracy for pneumonia remains clinically valid and auditable, as its original parameters are immutable. It allows for safe, regulatory-compliant model updates in healthcare.

A foundational language model column is pre-trained on a large English corpus. For each new language (e.g., Spanish, Mandarin, Arabic), a new column is added. Lateral connections allow the new language column to benefit from high-level semantic and syntactic features learned from English, improving learning efficiency. The English column remains frozen, so its translation or classification performance never degrades as the model becomes multilingual. This is a form of parameter isolation for languages.

The primary drawback of PNNs is linear parameter growth with each new task. For a network with N tasks, the total parameters scale with O(N²) due to lateral connections. This becomes computationally and memory prohibitive for long task sequences. This limitation spurred research into more parameter-efficient architectural methods like PackNet or Hard Attention to the Task (HAT). PNNs are therefore best suited for sequences of a few, high-stakes tasks where forgetting is unacceptable.

A core category of continual learning strategies that prevent catastrophic forgetting by dynamically modifying a neural network's structure. Unlike regularization or replay-based approaches, these methods allocate dedicated model capacity for new tasks. Key techniques include:

• Parameter Isolation: Assigning distinct, non-overlapping parameter subsets to different tasks.
• Dynamic Expansion: Adding new neurons, layers, or entire network columns (as in Progressive Neural Networks).
• Masking & Routing: Using learned attention masks or gating mechanisms to selectively activate pathways.

The primary advantage is near-zero interference, but the trade-off is linear growth in parameters and computational cost with each new task.

A family of architectural continual learning strategies designed to completely avoid parameter interference. The core principle is to allocate a unique, fixed subset of a model's parameters to each task. During inference, only the parameters associated with the identified task are activated.

Examples include:

• Progressive Neural Networks: Isolates parameters by instantiating a new column per task.
• Hard Attention to the Task (HAT): Learns binary attention masks over network units for each task, ensuring zero gradient flow to other tasks' parameters.
• PackNet: Iteratively prunes and freezes weights after learning a task, then uses remaining capacity for the next.

This approach provides strong forgetting prevention but can lead to inefficient parameter use and requires a task-ID at inference.

The core problem that continual learning and Progressive Neural Networks aim to solve. It is the tendency of a neural network to abruptly and drastically lose performance on previously learned tasks when trained on new, non-i.i.d. data. This occurs because standard gradient-based optimization overwrites weights that were critical for old tasks when minimizing loss for the new one.

Key characteristics:

• Caused by interference in shared parameters.
• Distinct from general overfitting or underfitting.
• Fundamentally linked to the stability-plasticity dilemma.

Progressive Neural Networks circumvent this by design, freezing old parameters and using lateral connections for knowledge transfer without modification.

The fundamental challenge in adaptive systems, including continual learning and neuroscience, of balancing two competing needs:

• Stability: The ability to retain consolidated knowledge and resist disruptive changes (preventing catastrophic forgetting).
• Plasticity: The capacity to integrate new information and adapt flexibly to novel tasks or environments.

All continual learning algorithms position themselves on this spectrum. Progressive Neural Networks strongly favor stability by freezing old parameters, ensuring perfect retention, while maintaining plasticity through the addition of new, trainable columns. Replay-based methods seek a balance by rehearsing old data, while pure regularization methods attempt to carefully constrain plasticity.

A critical quantitative metric for evaluating continual learning algorithms. It measures the influence of learning a new task on the performance of previously learned tasks. It is calculated as the average change in accuracy on all old tasks after training on the newest task.

Interpretation:

• Negative BWT: Indicates catastrophic forgetting; new learning has degraded past performance.
• Zero BWT: Indicates perfect stability; past performance is unchanged (a key goal of Progressive Neural Networks).
• Positive BWT: Indicates beneficial consolidation; new learning has improved understanding of past tasks (a rare and desirable outcome).

Progressive Neural Networks typically achieve BWT near zero by design, as old parameters are frozen.