Rehearsal-Based Methods

Experience Replay is the foundational rehearsal technique where a subset of raw data from previous tasks is stored in a replay buffer and interleaved with new task data during training. This direct rehearsal of old examples provides strong constraints against forgetting.

• Core Mechanism: The loss function becomes a weighted sum of the loss on new data and the loss on buffered old data.
• Edge Consideration: Storing raw data can be memory-intensive, making efficient buffer management critical for edge deployment.

Generative Replay (or Pseudo-Rehearsal) uses a separately trained generative model to produce synthetic data that mimics the distribution of past tasks. The main model rehearses on these generated samples instead of storing raw data.

• Core Mechanism: A Generative Adversarial Network (GAN) or Variational Autoencoder (VAE) is trained on each task's data and retained to produce samples for future rehearsal.
• Advantage: Drastically reduces memory footprint, as only model weights are stored, not data. This is highly relevant for edge-CL.
• Challenge: Requires training and maintaining a generative model, adding computational overhead.

Gradient Episodic Memory (GEM) is an optimization-centric rehearsal method. It stores past data in an episodic memory and uses it to constrain the gradient updates for new tasks, ensuring they do not increase the loss on old tasks.

• Core Mechanism: Solves a quadratic programming problem to find a gradient direction for the new task that satisfies inequality constraints derived from the memory data.
• Outcome: Provides a formal guarantee of non-negative backward transfer, meaning new learning does not harm old performance.
• Use Case: Suitable for scenarios with strict performance guarantees on all previous tasks.

iCaRL (Incremental Classifier and Representation Learning) is a seminal algorithm for class-incremental learning. It combines rehearsal with a nearest-mean-of-exemplars classification rule and a distillation loss.

• Core Components:
  • Bounded Replay Buffer: Stores a fixed number of exemplars per class using a herding selection algorithm.
  • Distillation Loss: Preserves knowledge in the network's representations.
  • Nearest-Mean Classification: At inference, a sample is classified based on the smallest distance to the mean feature vector of each class's exemplars.
• Significance: Established a strong benchmark for learning new classes over time without task identity.

Buffer Management is the algorithmic strategy for selecting and maintaining which data points are stored in a fixed-size replay buffer, directly impacting rehearsal quality and efficiency.

• Reservoir Sampling: A probabilistic algorithm that maintains a uniformly random sample from an infinite or large data stream. It gives each incoming sample an equal probability of being included in the buffer.
• Core-Set Selection: Aims to select a subset of data that best represents the entire dataset, often by solving a k-center or diversity-maximization problem. This maximizes coverage of the data distribution with minimal samples.
• Ring Buffer: A simple FIFO (First-In-First-Out) buffer that overwrites the oldest entry when full. It's computationally cheap but may not retain the most informative samples.

Learning without Forgetting (LwF) is a rehearsal-adjacent method that uses knowledge distillation instead of storing raw data. It encourages the model's new parameters to produce similar outputs (soft labels) for new data as the old parameters did.

• Core Mechanism: A distillation loss term is added to the new task loss, measuring the KL divergence between the current and old model's output distributions for the new task data.
• Advantage: Zero memory overhead for past data, as only the previous model's parameters are needed.
• Limitation: Performance is highly dependent on the similarity between new and old tasks; it can struggle with disjoint tasks. Often used in combination with a small replay buffer for robustness.

Edge devices have severely limited RAM and persistent storage. Maintaining a replay buffer of raw data or generated samples directly competes with the memory needed for the model itself and its runtime operations.

• Buffer Size vs. Performance: A small buffer may not adequately represent past tasks, leading to forgetting. A large buffer is infeasible.
• Data Representation: Storing compressed representations or embeddings instead of raw data is a common trade-off, but this adds complexity to the training pipeline.
• Example: A smart camera with 512MB RAM cannot store thousands of high-resolution images for rehearsal while also running a vision model.

The core rehearsal operation—interleaving old and new data during training—significantly increases computational cost, which is prohibitive for on-device learning.

• Training Cost: Each training step involves forward/backward passes on mixed batches, multiplying the compute required compared to standard inference.
• Energy Drain: Continuous computation rapidly depletes battery-powered devices.
• Latency Impact: The device cannot perform its primary function (e.g., object detection) if compute is monopolized by rehearsal training. Techniques like sparse rehearsal or performing updates only during idle periods are necessary.

Selecting what to store in the fixed-size replay buffer is a critical algorithmic challenge that must operate efficiently on-device.

• Core-Set Selection: Finding a minimal subset that best represents the data distribution is computationally expensive (e.g., requires solving a k-center problem).
• Online Strategies: Algorithms like reservoir sampling provide a uniform random sample but do not optimize for representativeness.
• Dynamic Prioritization: Should the buffer prioritize difficult, rare, or recent examples? Implementing these heuristics adds logic and overhead.

Rehearsal inherently involves retaining user data. On edge devices, this creates significant privacy risks and compliance challenges.

• Local Data Persistence: Sensitive data (e.g., personal photos, location traces) stored in the buffer is vulnerable if the device is compromised.
• Regulatory Compliance: Regulations like GDPR may treat the replay buffer as a data store, requiring mechanisms for right-to-erasure.
• Mitigations: Using generative replay (synthetic data) or federated continual learning where only model updates are shared can reduce privacy exposure, but they introduce other technical hurdles.

The very problem rehearsal aims to solve—catastrophic forgetting—can be exacerbated by the stringent limits of edge deployment.

• Insufficient Rehearsal: Due to memory limits, the model may only rehearse on a tiny, non-representative subset of past data, leading to partial forgetting.
• Interference from Non-IID Data: Edge data streams are highly non-independent and identically distributed (Non-IID). Rapidly shifting data distributions can overwhelm a small buffer's ability to stabilize learning.
• Example: A wearable health monitor learning new user patterns might forget older, seasonal patterns because the buffer cannot hold enough long-term history.

Deploying a rehearsal-based continual learning system requires co-design with the device's operating system, scheduler, and other applications.

• Resource Arbitration: The system must manage conflicts when the model needs compute for rehearsal versus other processes.
• Checkpointing and Recovery: Model and buffer state must be reliably saved to persistent storage to survive reboots or crashes, adding I/O complexity.
• Lifecycle Management: Over-the-air updates must handle versioning for both the model architecture and the structure/content of the replay buffer.

A foundational rehearsal technique where a subset of past training data or their learned representations are stored in a replay buffer and interleaved with new task data during training. This forces the model to rehearse old tasks concurrently with learning new ones. It is the most direct implementation of rehearsal, often using strategies like reservoir sampling for buffer management.

A rehearsal variant that uses a generative model (e.g., a Generative Adversarial Network or Variational Autoencoder) trained on past data to produce synthetic samples (pseudo-rehearsal). These generated examples mimic old experiences, allowing the main model to rehearse without storing raw data. This is critical for privacy-preserving or memory-constrained edge scenarios where storing real data is infeasible.

The fixed or dynamic memory storage component used in experience replay. Effective continual learning depends on buffer management strategies to select which past examples to retain. Common approaches include:

• Reservoir Sampling: Maintains a uniform random sample from a stream.
• Core-Set Selection: Chooses a representative subset that approximates the full data distribution.
• Herding: Selects prototypes closest to the class mean.

The core problem rehearsal methods aim to solve. It is the phenomenon where a neural network abruptly and drastically loses performance on previously learned tasks when trained on new data. This occurs due to unconstrained parameter overwriting. Rehearsal directly combats this by providing interleaved data that regularizes the loss landscape for old tasks.

The fundamental trade-off in all continual learning. Stability refers to a model's ability to retain old knowledge (resisting forgetting). Plasticity is its capacity to learn new information efficiently. Rehearsal-based methods explicitly manage this trade-off: the replay of old data promotes stability, while training on new data maintains plasticity. The buffer size is a direct knob for tuning this balance.

A strict, realistic variant of continual learning where the model receives a single, non-repeating pass through a stream of data, often one sample or a small batch at a time. This imposes severe constraints on rehearsal, as the model cannot loop over old data. Efficient buffer management and sample-efficient rehearsal strategies are paramount for success in this challenging scenario, especially on edge devices.