Task-Incremental Learning

The defining characteristic of this scenario. During both the training and evaluation/inference phases, the model is provided with an explicit, discrete identifier (e.g., task_id=2) specifying which task in the sequence the current data belongs to.

• Mechanism: This identifier acts as a conditional input or a selector, often used to activate a dedicated task-specific output head or to modulate network activations.
• Implication: This strong supervisory signal simplifies the learning problem compared to class-incremental learning, as the model does not need to autonomously infer the task context.

Each sequential task typically has its own, separate set of output classes or prediction targets. Learning is the process of accumulating these distinct output modules over time.

• Example: Task 1 might classify cats vs. dogs, Task 2 might classify cars vs. trucks, and Task 3 might classify planes vs. helicopters. The model learns a 2-way classifier for each.
• Architectural Consequence: This often leads to the use of a multi-head output layer, where a shared backbone feature extractor is connected to separate, task-specific output heads. The task_id determines which head is active for a given input.

A core goal is to learn a common, reusable feature extractor (the backbone network) that generalizes across all tasks. This promotes positive forward transfer and efficient parameter use.

• Benefit: Knowledge from early tasks (e.g., edge detection, shape recognition) can accelerate learning on later, related tasks.
• Challenge: The stability-plasticity dilemma is central. The backbone must be plastic enough to learn new features for novel tasks, but stable enough to preserve features critical for old tasks. Algorithms must protect this shared representation from catastrophic interference.

Performance is measured per-task, and crucially, the evaluator provides the correct task_id to the model during testing. This is a key differentiator from more challenging scenarios.

• Protocol: Given a test sample from Task k, the evaluator tells the model "use your Task k head." The model is only evaluated on its ability to discriminate within that task's output space.
• Contrast with Class-Incremental: In class-incremental learning, the model must infer the task from the input data alone, a significantly harder problem. Task-incremental learning isolates the challenge of representation stability from the challenge of task inference.

Solutions focus on managing updates to the shared backbone to prevent forgetting. The explicit task_id simplifies the isolation of task-specific parameters.

• Regularization Methods: Techniques like Elastic Weight Consolidation (EWC) or Synaptic Intelligence (SI) add penalties to important weights for previous tasks, slowing their change during new learning.
• Architectural Methods: Progressive Neural Networks instantiate new columns, while Hard Attention to the Task (HAT) learns binary masks over activations. The task_id selects the correct column or mask.
• Replay Methods: Experience Replay (ER) stores old examples in a buffer. During training on a new task, old examples are replayed, and the task_id ensures their loss is computed with the correct output head.

Task-incremental learning is one point on a spectrum of continual learning difficulty, defined by the information provided at test time.

• Easier than Class-Incremental: Class-incremental provides no task ID at test time, forcing a single, expanding output head.
• Easier than Domain-Incremental: Domain-incremental keeps the same output classes (e.g., always cats vs. dogs) but changes the input distribution (e.g., photos to sketches). The model must adapt without a task ID to signal the shift.
• Foundation for Harder Settings: It serves as a critical testbed for developing core anti-forgetting techniques for the shared backbone, which are also essential for more challenging scenarios.

A core strategy in task-incremental learning where distinct, non-overlapping subsets of a model's parameters are allocated to different tasks. This prevents catastrophic forgetting by eliminating interference between tasks.

• Hard Attention to the Task (HAT): Learns binary attention masks over network activations to selectively route information and protect task-specific parameters.
• Progressive Neural Networks: Instantiates a new, separate neural network column for each new task, with lateral connections to previous columns for knowledge transfer without overwriting.
• Key Benefit: Provides strong stability by design, as old task parameters are frozen or masked.

Techniques that add a penalty term to the loss function to constrain how much important parameters for previous tasks can change when learning a new one.

• Elastic Weight Consolidation (EWC): Uses the Fisher information matrix to estimate parameter importance and applies a quadratic penalty to slow learning on critical weights.
• Synaptic Intelligence (SI): Online method that accumulates the contribution of each parameter's change to the total loss change over time to estimate importance.
• Memory-Aware Synapses (MAS): Estimates importance based on the sensitivity of the learned function's output to parameter perturbations.
• Trade-off: Balances stability-plasticity dilemma but can limit plasticity if regularization is too strong.

Methods that retain a subset of data from previous tasks in a replay buffer and interleave it with new task data during training to rehearse old knowledge.

• Experience Replay (ER): Stores raw input-label pairs. A simple, highly effective baseline.
• Dark Experience Replay (DER): Stores both input-label pairs and the model's logit outputs, using them for a stronger consistency loss via knowledge distillation.
• Generative Replay: Uses a generative model (e.g., a GAN or VAE) trained on past data to produce synthetic samples for rehearsal, eliminating the need to store raw data.
• Core Challenge: Managing buffer size and sampling strategies to maximize retention with minimal memory.

Dynamic approaches where the model's capacity grows to accommodate new tasks, often combined with mechanisms to leverage prior knowledge.

• Progressive Neural Networks: As mentioned, adds new columns, ensuring no forgetting but leading to linear parameter growth.
• Adapter Layers: Adds small, task-specific adapters (e.g., LoRA modules) to a frozen backbone model. New tasks get new adapters, leaving the core model unchanged.
• Expert Networks (Mixture-of-Experts): Routes inputs to different, sparse sub-networks (experts). New tasks can be assigned to new or existing experts.
• Advantage: Naturally avoids forgetting; disadvantage is potential for unbounded model growth.

Uses the principle of knowledge distillation to transfer knowledge from a previous model version (the teacher) to the current model (the student) while learning a new task.

• Learning without Forgetting (LwF): A seminal method. For old tasks, it uses the model's outputs from before new training as soft targets, adding a distillation loss to preserve responses.
• iCaRL: Combines distillation with an exemplar management strategy and a nearest-mean-of-exemplars classifier for class-incremental learning.
• Mechanism: The Kullback-Leibler divergence loss encourages the new model's output distribution for old tasks to match the old model's, preserving decision boundaries.

Algorithms that directly manipulate the gradients computed on new task data to prevent updates that would harm performance on previous tasks.

• Gradient Episodic Memory (GEM): Stores past examples in an episodic memory. It projects the new task's gradient to the closest direction (in cosine similarity) that does not increase the loss on the memory examples.
• Averaged GEM (A-GEM): A more efficient variant that only requires the new gradient's projection to not increase the average loss on the memory, rather than for each past task individually.
• Objective: Enforces positive backward transfer or minimizes negative transfer by solving a constrained optimization problem during each update.

A more challenging continual learning scenario where new classes are introduced sequentially, and the model must learn to discriminate between all seen classes without being provided an explicit task identifier during evaluation. This requires the model to solve the task-identity inference problem. Common benchmarks include Split CIFAR and Split ImageNet.

A continual learning scenario where the input data distribution (or domain) changes across tasks, but the underlying output classes and the core task remain the same. For example, a model might learn to classify digits first from the MNIST dataset, then from the SVHN dataset. The challenge is to adapt to new visual styles without forgetting the original classification function.

A strict, non-i.i.d. setting where the model receives a single, continuous, and non-repeating pass over a potentially infinite stream of data. This imposes severe constraints: efficient single-epoch learning, minimal memory footprint, and the inability to revisit past data without a dedicated buffer. It closely mimics real-world data streams like social media feeds or sensor telemetry.

A family of architectural strategies that allocate distinct, non-overlapping subsets of a model's parameters to different tasks. This prevents catastrophic forgetting by design, as parameters for old tasks are frozen or masked. Key methods include:

• Progressive Neural Networks: Instantiates new network columns per task.
• Hard Attention to the Task (HAT): Learns binary attention masks over activations. The trade-off is potential parameter inefficiency and lack of positive transfer between tasks.

A foundational technique that mitigates forgetting by storing a subset of past training examples in a fixed-size replay buffer. During training on new data, the model interleaves rehearsal on these stored examples. This helps approximate the original data distribution. Advanced variants include:

• Generative Replay: Uses a generative model to produce synthetic past data.
• Dark Experience Replay (DER): Stores past model logits for stronger consistency regularization.

The fundamental, opposing objectives at the core of all continual learning. A model must maintain stability to retain knowledge of previous tasks, while also possessing plasticity to adapt efficiently to new information. Catastrophic forgetting occurs when plasticity overwhelms stability. Every continual learning algorithm is essentially a proposed solution to balancing this trade-off, whether through regularization, architectural expansion, or rehearsal.