Continual Learning

These methods add a penalty term to the loss function to constrain parameter updates for weights deemed important for previous tasks. They do not store raw data, making them memory-efficient but often less effective for complex task sequences.

• Elastic Weight Consolidation (EWC): Uses the Fisher information matrix to estimate parameter importance and applies a quadratic penalty.
• Synaptic Intelligence (SI): Computes an online, path-integral measure of each parameter's contribution to past task loss.
• Learning without Forgetting (LwF): Uses knowledge distillation, penalizing changes in the model's output logits for new data relative to its old parameters.

These methods retain a subset of data from previous tasks in a replay buffer and interleave it with new task data during training. This direct rehearsal is highly effective but introduces memory overhead.

• Experience Replay: Stores raw or compressed samples. Buffer management strategies like reservoir sampling are critical.
• Generative Replay: Uses a separately trained generative model to produce synthetic samples (pseudo-rehearsal), eliminating raw data storage.
• Gradient Episodic Memory (GEM): Stores past examples and projects new task gradients to avoid increasing loss on the memory, ensuring positive backward transfer.

These methods dynamically alter the network structure to allocate dedicated capacity for new tasks, preventing interference at its source. They are powerful but can lead to uncontrolled model growth.

• Progressive Neural Networks: Adds new, laterally connected columns for each task; previous columns are frozen.
• Hard Attention to the Task (HAT): Learns task-specific, hard binary attention masks over network neurons.
• Parameter Isolation: A general strategy of assigning non-overlapping parameter subsets to different tasks, completely avoiding interference.

Advanced approaches combine elements from multiple categories or introduce novel algorithmic mechanisms to balance the stability-plasticity dilemma more effectively.

• iCaRL: A class-incremental learning algorithm combining a nearest-mean-of-exemplars classifier with rehearsal and distillation.
• Meta-Continual Learning: Uses meta-learning to optimize a model's initial parameters for fast adaptation and reduced forgetting across a distribution of sequential tasks.
• Federated Continual Learning: Merges federated learning with continual learning, enabling decentralized sequential learning across edge devices with privacy guarantees.

The difficulty of continual learning is defined by the experimental protocol. These scenarios test different aspects of sequential adaptation.

• Task-Incremental Learning: Task identity is provided at test time. The simplest scenario, often serving as a baseline.
• Domain-Incremental Learning: The input distribution changes, but the output space is constant. Tests pure domain adaptation without forgetting.
• Class-Incremental Learning: New classes are added over time; the model must discriminate among all seen classes without task ID. The most challenging and practical scenario.

Deploying continual learning on resource-constrained devices (Edge-CL) imposes unique constraints that shape methodology selection.

• Memory Limits: Strict bounds on replay buffer size and model expansion.
• Compute & Energy: On-device training must be extremely efficient, favoring lightweight regularization or micro-replay.
• Data Privacy: Federated continual learning and generative replay become key for learning across devices without centralizing sensitive data.
• Connectivity: Algorithms must handle intermittent cloud connectivity, emphasizing robust, standalone on-device updates.

Edge devices have extremely limited RAM and persistent storage, creating a fundamental bottleneck for continual learning components.

• Replay Buffers: Storing even a small, representative subset of past data for rehearsal can exceed available memory.
• Model Growth: Architectural methods that expand the network (e.g., Progressive Neural Networks) are often infeasible.
• Parameter Overhead: Regularization methods like Elastic Weight Consolidation (EWC) require storing a Fisher information matrix per task, doubling memory for model parameters.
• Example: A microcontroller may have only 256KB of RAM, while a single 224x224 RGB image requires ~150KB uncompressed.

Inference-optimized edge hardware (e.g., NPUs, MCUs) often lacks the floating-point units or software stacks required for efficient on-device training.

• Low-Power CPUs: Training steps are computationally intensive and can drain batteries or exceed thermal budgets.
• Lack of Hardware Support: Many edge TPUs/NPUs are designed for fixed-model inference; backpropagation and gradient computation may not be accelerated or even supported.
• Heterogeneous Compute: Efficiently splitting the CL workload (e.g., replay on CPU, gradient computation on GPU) is complex on system-on-chip designs.
• Operational Impact: Sustained high compute usage can interfere with the device's primary function (e.g., a camera's frame processing).

Continual learning is an active learning process, consuming significantly more energy than static inference, which is often prohibitive for battery-powered or thermally constrained devices.

• Training vs. Inference: A single training step can be 10-100x more energy-intensive than a forward pass.
• Thermal Throttling: Sustained training can cause overheating, forcing the device to clock down or shut down, disrupting both learning and primary operations.
• Energy Harvesting Devices: For devices powered by solar or vibration, energy availability is sporadic, requiring CL algorithms to be pause-able and highly efficient per joule.
• This directly conflicts with the core CL requirement of frequent, incremental updates.

Edge data streams are non-i.i.d., unstructured, and often scarce, violating standard CL algorithm assumptions.

• Non-Stationary Distributions: Concept drift can be rapid and extreme (e.g., a security camera transitioning from day to night).
• Single-Pass/Online Setting: Data may be seen only once in a strict stream, ruling out multi-epoch training or easy buffer replenishment.
• Label Scarcity: True labels for new data may be unavailable or costly to obtain on-device, pushing requirements towards unsupervised or self-supervised CL.
• Unbalanced Tasks: New data may arrive in bursts of one class, leading to extreme task imbalance that exacerbates forgetting.

The edge paradigm emphasizes local processing, but this isolates devices, preventing solutions that rely on cloud offloading or data pooling.

• No Cloud Fallback: Algorithms cannot rely on uploading raw data or complex computations to a powerful server, a common trick in cloud-based CL research.
• Federated Continual Learning Complexity: While privacy-preserving, coordinating CL across a fleet of heterogeneous devices with non-i.i.d. streams and different learning stages is a massive systems challenge.
• On-Device Data Sovereignty: A core edge value proposition is that sensitive data never leaves the device. This rules out centralized replay buffers or cloud-based generative replay for pseudo-samples.

Edge models often operate in safety- or mission-critical contexts (e.g., medical devices, industrial sensors). CL updates must not introduce failures.

• Catastrophic Forgetting is Unacceptable: A security camera that forgets how to recognize a person after learning to detect cars is a system failure.
• Update Verification: There is often no human in the loop to validate model performance after an incremental update. The CL system must self-assess reliability.
• Algorithmic Stability: Methods must be numerically stable under low-precision arithmetic (e.g., INT8 quantization) common on edge hardware.
• Fault Tolerance: The learning process must survive sudden power loss or device resets without corrupting the operational model.

The core challenge continual learning aims to solve. Catastrophic Forgetting is the phenomenon where a neural network abruptly and drastically loses performance on previously learned tasks when trained on new data. This occurs due to unconstrained parameter overwriting.

• Mechanism: The gradient-based optimization for a new task shifts network weights away from configurations optimal for old tasks.
• Analogy: Like learning Spanish and then completely forgetting French after starting to learn Italian.
• Contrast: Continual learning algorithms are explicitly designed to mitigate this through regularization, rehearsal, or architectural isolation.

A foundational regularization-based continual learning method. EWC slows down learning on parameters deemed important for previous tasks by applying a quadratic penalty. Importance is estimated via the Fisher information matrix.

• Core Idea: Treats neural network parameters like springs; important parameters have high "elasticity" and resist change.
• Process: After learning Task A, compute the Fisher diagonal for each parameter. When learning Task B, the loss includes a penalty for changing parameters proportional to their Fisher importance from Task A.
• Use Case: Effective for task-incremental scenarios where task identity is known.

A rehearsal-based technique central to many continual learning systems. It involves storing a subset of past training data (or their latent representations) in a replay buffer and interleaving them with new data during training.

• Function: Provides direct exposure to old task data, preventing the model's decision boundaries from drifting.
• Buffer Management: Critical challenge. Strategies include:
  • Reservoir Sampling: Maintains a statistically uniform random sample from a stream.
  • Core-Set Selection: Chooses maximally representative exemplars to minimize buffer size.
• Edge Consideration: Storing raw data on devices raises privacy and storage concerns, leading to generative replay alternatives.

A strict, realistic variant of continual learning where the model receives a single, non-repeating pass through a stream of data, often one sample or a small mini-batch at a time.

• Constraint: No multiple epochs over data, and extremely limited memory for replay. Mirrors real-world data streams like sensor feeds or user interactions.
• Challenges: Amplifies the stability-plasticity dilemma. The model must adapt immediately (plasticity) while not overwriting crucial knowledge (stability) from a single exposure.
• Algorithms: Requires methods that make efficient, one-shot updates (e.g., Gradient Episodic Memory - GEM, A-GEM) or very fast generative replay.

The convergence of federated learning and continual learning. Enables a decentralized model to learn sequentially from non-stationary data streams across multiple edge devices (clients) without centralizing raw data.

• Process: Each device performs on-device training on its local, time-varying data stream using a continual learning algorithm. Only model updates (or important meta-data like parameter importance) are sent to a server for secure aggregation.
• Key Challenge: Managing client drift where devices experience different task sequences (non-i.i.d. and non-stationary data), complicating global aggregation.
• Privacy Benefit: Addresses both data privacy (via federated learning) and catastrophic forgetting (via continual learning) for lifelong edge AI.

Applies meta-learning (learning to learn) principles to the continual learning problem. The goal is to train a model's initial parameters or an outer-loop algorithm such that it can learn new tasks rapidly with minimal forgetting.

• Objective: Discover parameter initializations or learning rules that are inherently resilient to catastrophic forgetting and facilitate positive forward transfer.
• Common Approach: Use an episodic training protocol where the model is exposed to many simulated continual learning sequences (a sequence of tasks) during a meta-training phase.
• Outcome: The meta-learned model or optimizer exhibits improved performance on unseen task sequences, aiming for a general solution to continual learning.