Domain-Incremental Learning

The core characteristic is a fixed output space with a non-stationary input distribution. For example, a model trained to classify digits must perform the same 10-class classification task, but the data source changes sequentially (e.g., MNIST digits → SVHN street-view house numbers → handwritten digits on different paper textures). The task definition (class labels, prediction head) remains constant, but the visual features, styles, and statistical properties of the input data evolve.

Unlike task-incremental learning, explicit task identifiers are not provided at inference time. The model must correctly handle inputs from any previously seen domain without being told which domain the current sample belongs to. This creates a significant challenge, as the model cannot simply activate a domain-specific sub-network; it must develop a unified, robust representation that generalizes across all encountered distributions.

The dominant risk is catastrophic forgetting, where adapting the model's parameters to a new data distribution causes a severe performance drop on previous domains. This occurs due to interference in the shared network weights. Mitigation strategies are central to domain-incremental learning and include:

• Regularization Methods: Penalizing changes to parameters important for old domains (e.g., Elastic Weight Consolidation).
• Replay-Based Methods: Storing and retraining on exemplars from past domains (Experience Replay) or generating synthetic data (Generative Replay).
• Architectural Methods: Allocating new, domain-specific parameters (e.g., Adapters) while freezing the core model.

PEFT methods are exceptionally well-suited for domain-incremental learning. Instead of fully fine-tuning all model weights for each new domain—which maximizes interference—PEFT techniques like Low-Rank Adaptation (LoRA) or Adapter modules learn a small set of new parameters per domain. This approach provides several advantages:

• Minimized Interference: The frozen base model acts as a stable feature extractor, preserving knowledge.
• Efficiency: Only a tiny fraction of parameters are trained and stored per domain.
• Modularity: Domain-specific adapters can be swapped or composed, enabling flexible multi-domain inference.

Performance is measured across all encountered domains after sequential training. Key metrics include:

• Average Accuracy: The final accuracy averaged over all domains.
• Forgetting Measure: Quantifies the performance drop on a domain after training on subsequent domains. A low forgetting measure indicates good knowledge retention.
• Forward Transfer: Measures how learning earlier domains improves initial performance on a new domain. These metrics evaluate the model's stability (retaining old knowledge) and plasticity (learning new domains).

Domain-incremental learning mirrors real-world deployment where data streams evolve. Examples include:

• Autonomous Vehicles: A perception model must adapt to new geographic locations, weather conditions, and sensor hardware over time.
• Medical Imaging: A diagnostic model deployed across multiple hospitals must learn from each institution's unique imaging devices and protocols without forgetting others.
• Content Moderation: A classifier for harmful content must adapt to new social media platforms, slang, and cultural contexts as they emerge. These scenarios require models that continuously adapt without expensive full retraining on all historical data.

These methods store or generate examples from past domains to rehearse during training on new data, directly combating catastrophic forgetting. Experience Replay maintains a fixed-size replay buffer of real data samples. Generative Replay uses a separately trained generative model (e.g., a GAN or VAE) to produce synthetic data from previous domains. The core trade-off is between storage costs (for a buffer) and the quality/faithfulness of generated samples.

• Key Mechanism: Interleaving old and new domain data during training.
• Primary Challenge: Buffer management and generative model fidelity.
• PEFT Integration: Replay is often combined with a PEFT backbone (e.g., training only LoRA modules) to make the rehearsal process more parameter-efficient.

These techniques add a penalty term to the loss function to protect parameters deemed important for previous domains. They estimate parameter importance and slow down learning on those weights.

• Elastic Weight Consolidation (EWC): Approximates the Fisher information matrix to identify parameters critical to past performance and penalizes their deviation.
• Synaptic Intelligence (SI): Online method that accumulates parameter importance as the sum of weight changes multiplied by the loss gradient over training.

These methods are task-agnostic and do not require storing raw data, but they can struggle with severe distribution shifts as the constraint may become too restrictive.

This family of methods allocates distinct model parameters for each domain, preventing inter-task interference at the architectural level.

• Task-Specific Adapters: A unique set of adapter modules (e.g., LoRA, AdapterFusion) is learned and activated for each domain. The base model remains frozen.
• Progressive Networks: Expand the architecture by adding new columns of parameters for each new domain, with lateral connections to previous columns to enable forward transfer.
• Supermasks / Sparse Subnetworks: Discover and freeze a sparse, winning subnetwork for each domain within a larger model.

These approaches excel at knowledge retention but can lead to linear parameter growth with the number of domains.

These methods aim to learn an update rule or model initialization that is inherently conducive to sequential domain adaptation.

• Model-Agnostic Meta-Learning (MAML): Learns an initial set of parameters that can be rapidly adapted to a new domain with few gradient steps, promoting efficient forward transfer.
• Gradient Episodic Memory (GEM) & Averaged GEM (A-GEM): Store a small episodic memory of past domain data. When updating on a new domain, they project the gradient to a direction that does not increase the loss on the memory, ensuring positive backward transfer or minimizing negative transfer.

These strategies are computationally intensive but provide strong theoretical foundations for continual learning.

Low-Rank Adaptation (LoRA) is a particularly effective PEFT method for domain-incremental learning. For each new domain, a new pair of low-rank matrices (A and B) is trained and stored. During inference, the correct LoRA matrices are merged with the base model weights for the target domain.

• Advantages: Extremely parameter-efficient; only the low-rank matrices are stored per domain. Easy to deploy and switch between domains.
• Challenges: Requires a mechanism to select the correct LoRA weights at inference if the domain is not explicitly provided (task-agnostic inference).
• Common Practice: Often combined with a replay buffer for rehearsal or a regularization term to protect shared base model features.

These methods prepend a set of continuous, trainable vectors (prompts or prefixes) to the model's input or hidden states to steer its behavior for a specific domain.

• Domain-Specific Prompts: A unique set of prompt embeddings is learned for each domain. Only these embeddings are updated during training.
• Mechanism: The prompts act as a contextual bias, shifting the model's activation patterns to suit the new data distribution without altering core weights.
• Benefits: Highly modular and lightweight. Easy to add/remove domains by swapping prompt sets.
• Limitations: Performance can be sensitive to prompt length and initialization, and may be less effective for drastically different domains compared to adapter methods.

Continual learning is a machine learning paradigm where a model learns a sequence of tasks over time, aiming to accumulate knowledge without catastrophic forgetting. The core challenge is the stability-plasticity dilemma: balancing the retention of old knowledge with the acquisition of new information. It encompasses several scenarios, including domain-incremental, task-incremental, and class-incremental learning.

Catastrophic forgetting is the tendency of a neural network to abruptly and drastically lose performance on previously learned tasks when trained on new data. It is the primary failure mode in continual learning and is driven by inter-task interference during gradient-based optimization. Mitigation strategies include experience replay, regularization (e.g., EWC), and parameter-isolation methods like task-specific adapters.

Experience replay is a core continual learning technique where a model is periodically retrained on a stored subset of data from previous tasks to mitigate forgetting. A replay buffer holds exemplars from past data distributions. Variants include:

• Generative Replay: Using a generative model to produce synthetic past data.
• Constraint-based Replay: As used in Gradient Episodic Memory (GEM), which constrains new gradients to not increase loss on replayed data. This method directly combats the non-stationarity of sequential data streams.

Elastic Weight Consolidation is a regularization-based continual learning method. It estimates the importance of each model parameter to previous tasks using an approximation of the Fisher information matrix. During training on a new task, EWC applies a quadratic penalty that slows down learning on parameters deemed important for old tasks. This approach, along with Synaptic Intelligence (SI), enforces stability by making important parameters "elastic."

Task-specific adapters are small, trainable neural network modules (e.g., bottleneck feed-forward networks) inserted into a frozen pre-trained model. A unique adapter is learned for each task or domain. During inference, the correct adapter is activated, providing a parameter-isolation solution to catastrophic forgetting. This makes them a natural Parameter-Efficient Fine-Tuning (PEFT) method for continual learning, as only the tiny adapters are updated per task, leaving the shared backbone intact.

The stability-plasticity dilemma is the fundamental challenge in continual learning and adaptive systems. Stability refers to the system's ability to retain previously acquired knowledge. Plasticity is its capacity to integrate new information. Excessive stability leads to an inability to learn new tasks (intransigence), while excessive plasticity causes catastrophic forgetting. All continual learning algorithms, including those for domain-incremental learning, explicitly or implicitly navigate this trade-off.