iCaRL (Incremental Classifier and Representation Learning)

Instead of using a standard linear output layer, iCaRL classifies based on the nearest class mean in the feature space. For each class, a prototype vector is computed as the mean feature vector of its stored exemplars. At inference, an input is assigned to the class whose prototype is closest in L2 distance. This non-parametric rule allows the classifier to scale to an increasing number of classes without retraining the final layer and inherently mitigates the bias toward newly learned classes common in incremental learning.

To preserve knowledge of old classes, iCaRL employs a knowledge distillation penalty. When learning new classes, the model is trained not only with the standard cross-entropy loss for new data but also with a distillation loss that encourages the new model's outputs (logits) for old classes to match the outputs of the previous model. This technique, adapted from Learning without Forgetting (LwF), helps maintain the relative relationships between old classes in the output space, effectively combating catastrophic forgetting without requiring access to the original old data during the new training phase.

iCaRL maintains a fixed-size memory buffer of real data samples (exemplars) from past classes. Exemplars are selected using an iterative herding algorithm, which aims to approximate the average feature vector of all training samples for a class. The process:

• Computes the mean feature vector (prototype) for the class using the feature extractor.
• Iteratively selects the sample whose feature vector brings the current subset's mean closest to the full class mean. This results in a representative, diverse subset that best preserves the class distribution in feature space, enabling effective rehearsal.

The algorithm operates under a strict constant memory budget K, which is the total number of exemplars stored across all learned classes. When new classes are added, the budget is redistributed. The standard policy is to allocate an equal number of exemplars per class (m = K / total_classes). As the number of classes grows, the exemplars per class shrinks, simulating a realistic memory-constrained scenario. During training on a new task, the model rehearses by interleaving new class data with a balanced mix of exemplars from all old classes, a form of experience replay that directly counteracts forgetting.

Unlike methods that freeze the feature extractor, iCaRL jointly updates the entire neural network (feature extractor and classifier) when learning new classes. The update is carefully regularized by the combination of distillation loss and rehearsal with exemplars. This allows the representation space to evolve incrementally to accommodate new classes while being anchored by old class exemplars. The evolving representation is crucial for the nearest-mean classifier, as both old and new class prototypes must reside in a coherent, updated feature space.

iCaRL is explicitly designed for the challenging class-incremental learning (CIL) setting. Key protocol attributes:

• No task identifier at test time: The model must distinguish between all classes seen so far without knowing which incremental "task" the input belongs to.
• Uniform evaluation: Accuracy is measured over all classes encountered, reflecting real-world deployment where data from all periods can appear.
• Exemplar-only access to past data: After a training phase for a set of classes concludes, only the selected exemplars are retained; the original training data is discarded. This defines iCaRL's approach to the stability-plasticity dilemma.

Class-incremental learning is the specific, challenging continual learning scenario that iCaRL was designed to address. In this setting, a model learns new classes sequentially over time and must, at test time, discriminate between all classes seen so far without being provided an explicit task identifier. This contrasts with task-incremental learning, where the task ID is known. The core difficulty is the stability-plasticity dilemma: balancing the retention of old knowledge with the acquisition of new knowledge, all while the output space expands. Benchmarks like Split CIFAR are commonly used to evaluate algorithms under this protocol.

Knowledge distillation is a model compression technique where a smaller "student" model is trained to mimic the output distribution of a larger "teacher" model. In continual learning, particularly in methods like Learning without Forgetting (LwF) and iCaRL, it is repurposed for knowledge retention. The previous version of the model acts as the teacher for the old tasks, and the new model being trained acts as the student. A distillation loss (e.g., Kullback-Leibler divergence) penalizes the new model for deviating from its old predictions on new data, thereby helping to preserve the decision boundaries for previous classes without storing the original data.

Exemplar management is the algorithmic strategy for selecting and maintaining a fixed-size memory buffer of representative past examples. iCaRL's approach is a core innovation. Its strategy involves:

• Selection: After learning a new set of classes, iCaRL constructs the exemplar set for those classes using a herding algorithm, which selects instances whose feature vectors approximate the class mean.
• Pruning: To maintain a fixed total memory budget, exemplars from previous classes are reduced via random selection.
• Usage: These stored exemplars are used for both rehearsal during training and for the nearest-mean-of-exemplars classification rule at inference. Effective management is critical for balancing memory constraints with classification accuracy.

This is the novel classification rule introduced by iCaRL for the class-incremental learning scenario. Instead of using a traditional linear classification layer (which is prone to bias toward newly learned classes), iCaRL classifies based on prototype similarity in the feature space. For each class, it computes a prototype vector as the mean of the feature vectors of all stored exemplars for that class. At inference, an input is passed through the feature extractor, and its resulting feature vector is compared to all class prototypes using L2 distance. The input is assigned to the class with the nearest prototype. This non-parametric rule naturally accommodates an expanding number of classes.

Experience Replay is a foundational continual learning technique where a model interleaves training on new data with rehearsal on a small, stored subset of past data from a replay buffer. iCaRL is a specific, sophisticated instantiation of this idea. The core mechanism mitigates catastrophic forgetting by periodically reminding the model of old patterns. Key design choices include:

• Buffer Sampling Strategy: How to select which old examples to store (e.g., iCaRL's herding, random selection, reservoir sampling).
• Rehearsal Ratio: The proportion of training batches that contain replayed data versus new data.
• Loss Function: Simply replaying with a standard cross-entropy loss is Vanilla ER. iCaRL enhances this by combining replay with a distillation loss applied to the new data.

The stability-plasticity dilemma is the fundamental, competing-objective challenge at the heart of all continual learning. Stability refers to a system's ability to retain knowledge and performance on previously learned tasks. Plasticity refers to its ability to rapidly learn new information from novel data or tasks. Naive neural networks are highly plastic but lack stability, leading to catastrophic forgetting. iCaRL explicitly navigates this trade-off:

• For Stability: It uses exemplar replay and knowledge distillation to anchor the model to past knowledge.
• For Plasticity: It allows the entire feature representation and classifier to update with new data. The algorithm's success is measured by its ability to maximize both backward transfer (retaining old performance) and forward transfer (accelerating new learning).