Progressive Neural Networks

In artificial intelligence research, PNNs train agents across different Atari games or strategy game environments. Each game is a new task. The network retains mastery of Pong and Breakout while learning Space Invaders. Lateral connections allow the new game column to access useful abstractions (e.g., ball physics, scoring concepts) from prior columns, often improving forward transfer and learning speed for related games.

PNNs can learn to diagnose new diseases over time as medical knowledge expands. An initial column is trained to detect common conditions from X-rays. When a novel pathology emerges, a new column is added, freezing the original diagnostic capability. The new column uses lateral connections to build upon general radiological features learned previously, ensuring the model doesn't forget how to identify the original diseases while incorporating new knowledge.

For edge devices like smartphones, a base PNN column provides general services (e.g., next-word prediction). When a user develops a unique writing style or technical jargon, a new, small column can be added locally. This personalizes the model for that user without retraining the base model or affecting other users' experiences. The lateral connections allow the personal column to specialize effectively.

Applied to autonomous vehicles or surveillance systems, a PNN can learn new visual classes or environments sequentially. A base column trained for urban daytime driving can be frozen. New columns are added for night driving, adverse weather, or rural roads. Each new perceptual module benefits from the base features (edge detection, shape recognition) without causing catastrophic forgetting of the original driving conditions.

Catastrophic Forgetting is the phenomenon where a neural network abruptly and drastically loses previously learned information when trained on new data. It is the fundamental problem that continual learning methods like Progressive Neural Networks are designed to solve.

• Mechanism: Occurs due to parameter overwriting; as the model's weights are updated to minimize loss on new data, they drift from configurations optimal for old tasks.
• Analogy: Like learning Spanish and then completely forgetting French after starting to learn Italian.
• Contrast with PNNs: PNNs prevent this by design through parameter isolation, freezing old columns and adding new ones.

Architectural Methods are a family of continual learning strategies that dynamically modify the neural network's structure to accommodate new tasks. Progressive Neural Networks are a prime example of this approach.

• Core Principle: Parameter Isolation. Allocate dedicated, non-overlapping model capacity (e.g., new columns, masks, or sub-networks) for each new task.
• Key Techniques:
  • Progressive Neural Networks: Add laterally connected columns.
  • Hard Attention to the Task (HAT): Learn task-specific binary attention masks over shared neurons.
  • PackNet/Piggyback: Learn binary masks to freeze important weights.
• Trade-off: Provides strong forgetting prevention but leads to linear growth in parameters with tasks.

Elastic Weight Consolidation is a regularization-based continual learning method that slows down learning on parameters deemed important for previous tasks.

• Mechanism: Adds a quadratic penalty term to the loss function. The penalty is based on the Fisher information matrix, which estimates each parameter's importance to past tasks. Important parameters are "anchored" with high elasticity.
• Analogy: Treats important synaptic connections as if they are connected by elastic bands, resisting change.
• Contrast with PNNs: EWC is a parameter-sharing method (single network), while PNNs use parameter isolation. EWC is more parameter-efficient but can struggle with high task disparity.

Experience Replay is a rehearsal-based continual learning technique that stores a subset of past training data (or their representations) and interleaves them with new data during training.

• Core Component: The Replay Buffer, a memory of fixed or dynamic size that stores exemplars from previous tasks.
• Buffer Management Strategies:
  • Reservoir Sampling: Maintains a uniform random sample from a stream.
  • Core-Set Selection: Selects a representative subset that approximates the full data distribution.
  • Generative Replay: Uses a generative model to produce synthetic old data (pseudo-rehearsal).
• Contrast with PNNs: PNNs are a data-free method (no raw past data needed), while replay explicitly stores or generates past data.

The Stability-Plasticity Dilemma is the fundamental trade-off in continual learning and adaptive systems between retaining old knowledge (stability) and efficiently integrating new information (plasticity).

• Stability: The resistance to catastrophic forgetting. High stability means old knowledge is preserved.
• Plasticity: The ability to learn new patterns quickly and flexibly.
• Method Trade-offs:
  • Progressive Neural Networks: High stability (no forgetting), lower plasticity (fixed old columns, cannot refine past knowledge).
  • Regularization Methods (e.g., EWC): Moderate stability and plasticity, balancing both.
  • Replay Methods: Can tune the balance via buffer size and sampling strategy.
• All continual learning algorithms position themselves differently on this spectrum.

Federated Continual Learning combines the decentralized, privacy-preserving training of federated learning with the sequential, non-stationary data streams of continual learning.

• Challenge: Devices (clients) experience local, evolving data distributions (Edge-CL), and the global model must learn sequentially across all devices without forgetting.
• Interplay with PNNs: A Progressive Neural Network architecture could be deployed in a federated setting, where each client might manage its own column progression or contribute to a global column architecture. This presents complex challenges in column synchronization and personalization.
• Key Consideration: Must address both catastrophic forgetting and the statistical heterogeneity inherent in federated data.