Progressive Neural Networks

The core innovation of Progressive Neural Networks is the columnar architecture. For each new task (T_k), a completely new neural network column is instantiated. This column is trained from scratch on the data for T_k. Crucially, the parameters of all previously trained columns for tasks T_1...T_{k-1} are frozen, providing an absolute guarantee against catastrophic forgetting of those tasks. The architecture grows horizontally with each new task, creating a bank of specialized, non-interfering expert networks.

To enable knowledge transfer from old tasks to new ones, each new column receives lateral connections from all preceding columns. These are simple, trainable linear adapters (e.g., a single-layer perceptron) that project the activations from layer l of an old column into layer l of the new column. This allows the new column to leverage the feature representations learned by previous networks, accelerating learning and improving performance on the new task through forward transfer. The architecture creates a directed acyclic graph of information flow.

PNNs are a canonical example of a parameter isolation strategy. By dedicating a unique, non-overlapping subset of the model's total parameters (an entire column) to each task, interference is architecturally prevented. This is in contrast to regularization-based methods (like EWC) that attempt to share parameters while penalizing changes. The trade-off is a linear growth in parameters with the number of tasks, making PNNs computationally and memory intensive for long task sequences, but providing strong performance guarantees.

The primary benefit of PNNs is their strong forward transfer capability. When learning task T_k, the new column can query and incorporate features from all prior columns via lateral connections. For example, a column trained on image classification might provide useful edge detectors to a new column learning object detection. This transfer is learned end-to-end, as the lateral connection weights are trained alongside the new column's core weights. This often leads to faster convergence and higher final accuracy on the new task compared to learning in isolation.

At inference time, the architecture requires a mechanism to select the correct column(s). In the standard task-incremental learning setting, an explicit task ID is provided, and only the corresponding column is activated for prediction. For more challenging scenarios without a task ID, a task router must be implemented. This can be a small classifier trained to identify the task from the input, or a gating network that weights the outputs of all columns. The need for explicit routing is a key complexity introduced by the isolationist design.

The major limitation of PNNs is their scalability. Total parameters grow as O(k*n), where k is the number of tasks and n is the size of a single column. This leads to high memory footprint and slower inference as the task sequence lengthens. While lateral connections are relatively cheap, the core columns are not. This makes vanilla PNNs impractical for very long sequences (e.g., 100+ tasks) or for deploying extremely large base models. Subsequent research has focused on parameter-efficient columns using adapters or pruning to mitigate this cost.

PNNs are a natural fit for robotic systems that must master a sequence of motor skills. A robot can learn to grasp an object (Column 1), then learn to push it (Column 2) while leveraging the visual features from the first column via lateral connections. This prevents catastrophic forgetting of foundational skills while enabling complex skill chaining. This architecture is crucial for sim-to-real transfer learning, where a robot learns tasks progressively in simulation before physical deployment.

In complex video games or simulations, an AI agent must master diverse environments (e.g., forest, desert, urban) or game types (strategy, first-person shooter). A PNN can dedicate a column to each domain, allowing the agent to become an expert in new scenarios without degrading its performance in previously mastered ones. The lateral connections enable positive forward transfer, allowing tactics learned in one environment to bootstrap learning in another, leading to more robust and generalist game-playing agents.

In healthcare, diagnostic models must adapt to new medical imaging modalities (X-ray, MRI, ultrasound) or newly discovered disease markers without forgetting how to detect older, well-established conditions. A PNN column can be instantiated for a new imaging technique. Its lateral connections to columns trained on historical data allow it to leverage foundational anatomical knowledge, improving initial accuracy on the new task while preserving a reliable diagnostic baseline for all previous conditions—a critical requirement for clinical workflow automation.

For platforms serving diverse user cohorts (e.g., different geographic regions, age groups, or interest clusters), a monolithic model can struggle. A PNN can allocate a column to learn the deep preferences of a new user segment. This column can transfer broad patterns from a general recommendation column while specializing, ensuring the model for the new cohort doesn't negatively interfere with recommendations for existing users. This enables hyper-personalization at scale within a single, expanding model architecture.

A foundational language model (Column 0) can be progressively specialized for new, sensitive, or proprietary domains. A legal firm could add a column for contract analysis, a hospital for medical note summarization, and a finance team for earnings report generation. Each specialized column benefits from the base model's general linguistic knowledge via lateral connections but maintains isolated, domain-specific parameters. This provides a structured, auditable path for continuous model learning in enterprises with strict data segregation requirements.

The primary drawback of PNNs is linear parameter growth with each new task. If Task N requires a column with P parameters, the total model size grows as O(NP)*. This makes PNNs computationally and memory intensive for long task sequences. This limitation has spurred research into more parameter-efficient dynamic architectures, such as using hard attention masks or adapters, which aim to achieve similar isolation benefits without the full cost of a new network column per task.

A continual learning strategy that allocates distinct, non-overlapping subsets of a model's parameters to different tasks. This prevents catastrophic forgetting by eliminating parameter interference between tasks.

• Key Methods: Includes Progressive Neural Networks (new columns), Hard Attention to the Task (HAT) (binary masks), and PackNet (pruning and retraining).
• Trade-off: Provides strong stability but can lead to linear parameter growth and lacks forward transfer unless explicit lateral connections are added.

A regularization-based continual learning algorithm that slows down learning on parameters deemed important for previous tasks. It adds a quadratic penalty term to the loss function, anchored by the Fisher information matrix, which estimates each parameter's importance.

• Mechanism: Parameters with high Fisher importance are "consolidated" and change less during new task training.
• Contrast with PNNs: Unlike Progressive Neural Networks, EWC uses a single, shared network, making it more parameter-efficient but often less effective at preventing interference in complex task sequences.

A foundational technique that mitigates forgetting by storing a subset of past training examples in a replay buffer and interleaving them with new data during training.

• Core Function: Provides the model with direct exposure to old data distributions.
• Variants: Includes Generative Replay (using a generative model to produce synthetic past data) and Dark Experience Replay (DER) (which also stores past model logits for stronger consistency loss).
• Architectural Role: Often used in conjunction with other methods; a Progressive Neural Network could employ replay within its columns for additional robustness.

An algorithm that stores past examples in an episodic memory and uses them to constrain optimization. It projects the gradient for a new task so that it does not increase the loss on the memorized examples from previous tasks.

• Gradient Projection: This defines a feasible region in gradient space, ensuring updates are non-interfering.
• Comparison: While Progressive Neural Networks prevent interference via architecture, GEM prevents it via optimization constraints on a single network, avoiding parameter growth but requiring more complex optimization.

A challenging and realistic evaluation scenario for continual learning where new classes are introduced sequentially over time. The model must learn to discriminate between all seen classes without receiving an explicit task identifier at test time.

• Key Challenge: Requires the model to maintain a single, expanding output head.
• PNN Application: A Progressive Neural Network can be adapted to this setting by adding a new column per task and using a shared classifier, but managing the growing output space remains non-trivial. Algorithms like iCaRL are specifically designed for this setting.

The fundamental trade-off at the heart of all continual learning systems. Stability refers to the ability to retain knowledge from previous tasks, while plasticity is the capacity to learn new information efficiently.

• Algorithmic Spectrum: Methods like Elastic Weight Consolidation heavily favor stability. Progressive Neural Networks offer high stability for old tasks (via frozen columns) and high plasticity for new tasks (via new columns), at the cost of efficiency.
• Optimal Balance: The goal of continual learning research is to design algorithms that optimally navigate this dilemma for a given set of constraints.