Buffer Management

A Ring Buffer or First-In-First-Out (FIFO) strategy maintains a fixed memory capacity by overwriting the oldest sample when a new one is added. This is computationally trivial and memory-efficient, making it suitable for strict Edge-CL deployments. However, it can lead to rapid forgetting of foundational early tasks if the data distribution shifts significantly. Its performance is highly dependent on task ordering and is often used as a simple baseline for comparing more sophisticated buffer management techniques.

Core-Set Selection strategies aim to store the most informative or representative subset of past data. Instead of random sampling, they use metrics to maximize coverage of the past data distribution within the buffer's constraints.

Common approaches include:

• K-Center Greedy: Selects points to minimize the maximum distance from any data point to its nearest buffer point.
• Herding: Selects samples that best approximate the class-wise mean feature vector.
• Gradient-Based Scoring: Ranks samples by their expected contribution to the loss if forgotten.

This method prioritizes buffer efficiency but adds computational overhead for sample selection.

This strategy prioritizes storing data samples that the model finds uncertain or challenging, under the hypothesis that rehearsing hard examples better preserves decision boundaries. Samples are scored using metrics like predictive entropy, Bayesian uncertainty, or loss value. High-scoring samples are retained or replace low-scoring ones in the buffer. This approach can be more effective than random sampling per stored example but may increase the risk of overfitting to noisy or outlier data if not carefully regularized.

Instead of storing raw input data, this strategy stores compressed representations (e.g., features from an intermediate model layer) or prototypes (e.g., class-mean feature vectors). This drastically reduces memory footprint, a key concern for On-Device Training. During rehearsal, the model uses these stored features to compute distillation or classification losses. A major challenge is the representation drift—as the model's feature extractor updates for new tasks, the old stored features may become inconsistent, reducing rehearsal effectiveness.

Dynamic Buffer Allocation strategies adjust the amount of memory dedicated to each past task based on its perceived importance, complexity, or performance degradation. This moves beyond a fixed buffer per task. Techniques include:

• Performance-Gated Allocation: Allocate more slots to tasks where Backward Transfer is most negative.
• Data-Distribution-Aware Allocation: Allocate buffer space proportional to the diversity or spread of a task's data.
• Learnable Allocation: Use a meta-controller to learn an optimal allocation policy. This is a sophisticated approach aimed at maximizing overall Forward and Backward Transfer.

A foundational continual learning technique where a subset of past training data or their representations are stored and interleaved with new data during training. This rehearsal of old tasks is the primary mechanism that Buffer Management strategies support. The effectiveness of the overall system is directly tied to the quality and representativeness of the data selected for the replay buffer.

The core problem that Buffer Management aims to mitigate. This is the phenomenon where a neural network abruptly and drastically loses previously learned information when trained on new data. Without strategies like rehearsal from a well-managed buffer, models suffer significant performance degradation on past tasks, rendering continual learning impossible.

A canonical probabilistic algorithm for maintaining a uniformly random sample of fixed size from a potentially infinite data stream. It is a fundamental Buffer Management policy where each new sample has a probability of k/n (where k is buffer size, n is total samples seen) of replacing a randomly selected existing buffer sample. This ensures the buffer remains a statistically representative snapshot of the entire stream.

An advanced buffer management strategy that moves beyond random sampling. It aims to select a minimal subset of data points (a core-set) that best approximates the properties (e.g., the gradient space or feature distribution) of the entire dataset. Methods include:

• K-Center Greedy: Selects points to minimize the maximum distance from any point to its nearest center.
• Gradient-Based Matching: Selects samples whose gradients are most representative of the full set. This maximizes the informational value of each buffer slot.

The fundamental trade-off that Buffer Management directly addresses. Stability refers to a model's ability to retain old knowledge (resisting forgetting), while Plasticity is its capacity to learn new information efficiently. A large, diverse buffer favors stability but may limit plasticity by saturating training with old data. Management strategies must balance buffer composition to optimize this trade-off for the given data stream.

The most challenging and realistic setting for Buffer Management. In this strict variant, the model receives a single, non-repeating pass through a stream of data, often one sample or a small mini-batch at a time. Buffer management policies must make immediate, irrevocable decisions about what to store or discard without future access to the data, placing a premium on efficient and robust selection algorithms like reservoir sampling or herding.