Continual Learning

Catastrophic Forgetting is the tendency of a neural network to abruptly and drastically lose performance on previously learned tasks when trained on new data. This occurs because gradient-based optimization overwrites the weights critical for old knowledge as it minimizes loss for the new task.

• Mechanism: The plasticity-stability dilemma. Network parameters must be plastic enough to learn new tasks but stable enough to retain old ones.
• Example: A microcontroller-based wildlife classifier that learns to recognize a new bird species might suddenly fail to identify species it previously knew.
• Mitigation: Techniques include elastic weight consolidation (EWC), which estimates parameter importance to penalize changes to critical weights, and replay buffers, which store a subset of old data for rehearsal.

In real-world deployment, a model does not receive explicit signals about which "task" it is performing. Task Ambiguity refers to the challenge of determining the current data distribution or objective from the input stream alone.

• Problem: An on-device sensor must infer whether new vibration patterns belong to a known failure mode (requiring recall) or a novel one (requiring learning).
• Task Inference: The model must autonomously decide when to recall, when to learn anew, and when to create a new internal representation. This is often approached with unsupervised or self-supervised methods to cluster data or detect distribution shifts.
• Consequence: Incorrect inference leads to interference, where new knowledge corrupts unrelated old knowledge.

Concept Drift occurs when the statistical properties of the target variable a model is trying to predict change over time in unforeseen ways. This is distinct from simply learning new tasks; it's the evolution of an existing task.

• Types: Sudden (abrupt change), Gradual (slow shift), Recurring (seasonal patterns).
• On-Device Impact: A microphone's keyword detector may degrade as ambient noise profiles change with seasons, or a user's voice characteristics slowly change.
• Challenge: The system must differentiate drift (requiring model update) from anomalous noise (which should be ignored). Requires robust online change-point detection algorithms that operate within tight memory constraints.

Continual learning on microcontrollers must contend with severe, non-negotiable hardware limits, making many cloud-based solutions infeasible.

• Memory Overhead: Replay buffers, generative models for pseudo-rehearsal, or additional network parameters for regularization (like in EWC) consume precious SRAM/Flash.
• Compute Budget: Forward/backward passes for rehearsal or regularization increase latency and energy consumption, conflicting with real-time and battery-life requirements.
• Key Trade-off: The memory-compute-accuracy trade-off. Solutions must be asymmetric, favoring extremely low memory overhead even if it requires slightly more compute. Techniques like hyperdimensional computing or sparse synaptic updates are explored for this environment.

The goal of continual learning is not just to avoid forgetting, but to enable positive knowledge transfer across tasks.

• Forward Transfer (FWT): The ability for learning on Task A to improve performance on future, related Task B before Task B is seen. Indicates useful generalization.
• Backward Transfer (BWT): The ability for learning Task B to improve performance on the previously learned Task A (positive BWT). Negative BWT is synonymous with catastrophic forgetting.
• Measurement: A continual learning algorithm is evaluated by its Average Accuracy and its deliberate balance of FWT and BWT. On-device, positive transfer is crucial for efficient learning; discovering shared features across sensor modalities (e.g., temporal patterns in audio and vibration) can reduce the need for task-specific parameters.

Robustly evaluating continual learning systems is a complex meta-challenge. Naive metrics like final average accuracy can mask critical failure modes.

• Standard Protocols: Class-Incremental Learning (CIL), Task-Incremental Learning (TIL), and Domain-Incremental Learning (DIL), each with varying levels of task identity provided at inference.
• Key Metrics:
  • Average Accuracy (ACC): Performance averaged across all tasks after training is complete.
  • Forgetting Measure (FM): The average drop in performance for each task from its peak accuracy to its final accuracy.
• On-Device Specifics: Benchmarks must also track peak memory usage, energy per update, and inference latency throughout the entire lifelong sequence. Real-world benchmarks involve data streams from sensors with natural temporal correlations and drifts.

Catastrophic Forgetting is the primary challenge in continual learning, where a neural network abruptly loses previously learned information when trained on new data. On-device, this is exacerbated by limited memory for storing old data. Mitigation strategies include:

• Elastic Weight Consolidation (EWC): Penalizes changes to weights deemed important for previous tasks.
• Gradient Episodic Memory (GEM): Stores a subset of past examples in a fixed-size buffer to constrain new gradients.
• Synaptic Intelligence: Dynamically estimates parameter importance to protect critical connections. Without these techniques, a sensor model that learns a new sound pattern could completely forget how to recognize the original one.

A Replay Buffer is a constrained memory store for a subset of past training data, used to interleave old examples with new data during training, directly combating forgetting. In TinyML, this buffer is extremely small (e.g., 100-1000 samples). Generative Replay is a more advanced technique where a small generative model (like a Variational Autoencoder) is trained to produce synthetic examples of past data, eliminating the need to store raw data. The key trade-off is between the fidelity of the replay and the computational/memory overhead on the microcontroller.

Parameter-Efficient Fine-Tuning methods are essential for on-device continual learning as they minimize the number of trainable parameters, reducing memory, compute, and energy costs. Key techniques include:

• Adapter Layers: Small, trainable modules inserted between frozen pre-trained layers.
• Low-Rank Adaptation (LoRA): Injects trainable low-rank matrices into attention layers, updating a tiny fraction of weights.
• Prompt Tuning: Learns a small set of continuous embedding vectors (soft prompts) that condition the frozen model on a new task. These methods allow a microcontroller to adapt a vision model to a new object class by training less than 1% of the total parameters.

For a device to perform multiple learned tasks, it needs mechanisms to select the correct behavior at inference time. Task-Aware Inference often uses a task identifier (from a classifier or user input) to activate specific model components, like a task-specific output head or adapter. Dynamic Architectures, such as Progressive Neural Networks, expand the network by adding new columns for new tasks, preventing interference but increasing size. A more TinyML-friendly approach is PackNet, which iteratively prunes and freezes weights for old tasks, then uses the freed capacity to learn new ones, all within a fixed parameter budget.

Continual learning on microcontrollers operates under severe, non-negotiable constraints that define the algorithm design space:

• RAM (<< 512 KB): Limits model size, batch size, and replay buffer capacity.
• Flash (1-2 MB): Stores the model parameters, with limited space for multiple task checkpoints.
• Compute (MHz-range CPU, no GPU): Makes backpropagation slow and energy-intensive.
• Power (µW-mW active): Dictates that training must be infrequent, short, or triggered by specific events. Algorithms must be designed for incremental learning in a single pass over tiny data batches, with minimal overhead.

Federated Continual Learning merges two paradigms: learning sequentially from local data streams on devices (continual) and aggregating knowledge across a fleet without sharing raw data (federated). This creates a "lifelong learning network" of devices. Challenges are compounded: devices face non-IID data streams and catastrophic forgetting. Solutions involve federated aggregation of consolidated parameters (like EWC's Fisher information matrix) or generative models for replay. It enables a fleet of industrial sensors to each adapt to local machine wear patterns while contributing to a globally improved failure prediction model.

Catastrophic Forgetting is the tendency of a neural network to abruptly and drastically lose previously learned information when trained on new data. This is the core problem continual learning aims to solve. Mitigation strategies include:

• Elastic Weight Consolidation (EWC): Penalizes changes to weights deemed important for previous tasks.
• Replay Buffers: Storing a subset of old data to retrain alongside new data.
• Architectural Expansion: Dynamically adding new network capacity for new tasks.

On-Device Fine-Tuning is the process of adapting a pre-trained model using local data directly on an edge device (e.g., a microcontroller). It is a practical application of continual learning under severe hardware constraints. Key enabling techniques are:

• Parameter-Efficient Fine-Tuning (PEFT): Methods like LoRA or Adapter Layers that update only a tiny fraction of the model's parameters.
• Federated Fine-Tuning: A variant where fine-tuning occurs across a fleet of devices, with updates aggregated privately.

Federated Learning (FL) is a decentralized training paradigm where a global model is learned across many edge devices without centralizing raw data. It shares continual learning's sequential, data-stream nature but focuses on cross-device collaboration. Core challenges include:

• Statistical Heterogeneity: Devices have Non-IID data, similar to a continual learner facing shifting data distributions.
• Communication Efficiency: Minimizing the cost of sending model updates.
• Privacy Preservation: Using techniques like Differential Privacy and Secure Aggregation.

Parameter-Efficient Fine-Tuning encompasses methods that adapt large pre-trained models by training only a small number of extra parameters, making continual learning on edge devices feasible. Primary methods include:

• Low-Rank Adaptation (LoRA): Injects trainable low-rank matrices into transformer layers.
• Adapter Layers: Inserts small, trainable modules between frozen pre-trained layers.
• Prompt Tuning: Learns continuous task-specific vectors (soft prompts) prepended to the input. These methods enable personalization and task adaptation with minimal memory and compute overhead.

Rehearsal-Based Methods are a class of continual learning techniques that mitigate catastrophic forgetting by retaining and replaying examples from previous tasks during training on new data. Key implementations are:

• Experience Replay: Maintaining a fixed-size buffer of past data samples.
• Generative Replay: Using a generative model (e.g., a GAN) to produce synthetic examples of past data.
• Core-Set Selection: Intelligently selecting a representative subset of old data to store. The major trade-off is between rehearsal buffer size and preservation performance.

Meta-Learning (learning to learn) frameworks are applied to continual learning to discover optimization algorithms or model initializations that are inherently resilient to forgetting. Approaches include:

• Model-Agnostic Meta-Learning (MAML): Finds an initial set of parameters that can rapidly adapt to new tasks with few gradient steps, which can ease sequential adaptation.
• Meta-Experience Replay: Meta-learns a strategy for selecting which past experiences to replay.
• Online Meta-Learning: Adapts the learning algorithm itself in an online, continual manner.