Multi-Task PEFT

AdapterFusion is a two-stage, knowledge-composing architecture for multi-task learning. First, multiple task-specific adapters are trained independently on a frozen base model. In the second stage, a new fusion layer is trained on top of the frozen adapters. This layer uses an attention mechanism to dynamically combine the outputs of the relevant adapters for a given input, enabling cross-task knowledge transfer without catastrophic forgetting.

• Key Mechanism: Attention-based composition of frozen adapter outputs.
• Advantage: Prevents negative interference between tasks by isolating initial training.
• Use Case: Building a unified model for related NLP tasks like sentiment analysis, named entity recognition, and natural language inference.

Mixture-of-Adaptors (MoA) is a conditional computation architecture that integrates multiple adapter modules with a learned routing network. For each input, a gating mechanism (e.g., a softmax layer) calculates a weighted combination of expert adapters, and only the selected adapters' parameters are activated. This creates a sparse, parameter-efficient pathway through the model tailored to the input's inferred task.

• Key Mechanism: Learned gating for sparse activation of specialized adapters.
• Advantage: Increases model capacity without a proportional increase in compute per forward pass.
• Use Case: Multi-domain chatbots that dynamically route queries to domain-specific reasoning modules (e.g., support, sales, technical).

Task Arithmetic is a post-training method for creating a multi-task model by mathematically combining task vectors. A task vector is the difference between a model fine-tuned on a specific task and the original base model (θ_task - θ_base). Competencies are merged by performing arithmetic on these vectors, such as adding them (θ_base + τ_A + τ_B) or applying weighted combinations.

• Key Mechanism: Linear arithmetic operations on weight-space task vectors.
• Advantage: Enables zero-shot merging of independently trained PEFT modules (e.g., LoRA matrices) without additional training.
• Use Case: Combining a model fine-tuned for code generation with another fine-tuned for text summarization into a single, multi-skilled model.

This architecture uses a hypernetwork—a small neural network that generates the parameters for task-specific adapters—conditioned on a task embedding. Instead of storing a separate adapter for every task, the hypernetwork learns to produce the necessary adapter weights on-demand. The base model remains frozen, and only the hypernetwork and task embeddings are trained.

• Key Mechanism: A meta-network that dynamically generates adapter parameters.
• Advantage: Drastically reduces storage overhead for a large number of tasks; enables generalization to unseen tasks via the task embedding space.
• Use Case: Personalization at scale, where a unique task embedding represents each user, and a shared hypernetwork generates their personalized adapter.

Inspired by multi-task learning fundamentals, this design uses two types of adapters within each transformer layer: shared adapters and task-private adapters. The shared adapters capture common, transferable features across all tasks, while the private adapters learn nuances specific to individual tasks. The output is typically a combined activation from both pathways.

• Key Mechanism: Explicit separation of shared and task-specific parameters within the adapter structure.
• Advantage: Provides an inductive bias for positive transfer while minimizing negative interference.
• Use Case: Multi-lingual NLP models where shared adapters capture cross-lingual semantics and private adapters handle language-specific syntax.

This strategy addresses continual learning by adding a new, small PEFT module (e.g., a LoRA rank or an adapter) for each sequential task while keeping previous modules frozen. To manage growth, techniques like adapter pruning or consolidation may be applied. The system routes inputs through a growing library of task-specific modules, often using a task identifier.

• Key Mechanism: Progressive expansion of the model with immutable task-specific parameter blocks.
• Advantage: Mitigates catastrophic forgetting by design, as old task parameters are never overwritten.
• Use Case: A model that learns new product classification categories or updated legal compliance rules over time without retraining from scratch.

A customer service chatbot uses AdapterFusion to handle queries across different departments. First, isolated adapters are trained for:

• Finance (billing, refunds)
• Technical Support (troubleshooting)
• Sales (product inquiries) A secondary fusion layer then dynamically combines these expert adapters based on the user's query intent, enabling a single model to provide specialized, accurate responses without maintaining separate models for each domain.

An internal enterprise search engine employs a Mixture-of-Adaptors (MoA) architecture. A learned gating network routes a user's query to the most relevant specialized adapter:

• Legal Document Adapter: Trained on contracts and compliance texts.
• Engineering Adapter: Fine-tuned on code repositories and technical RFCs.
• HR Policy Adapter: Adapted on employee handbooks and policy documents. This allows a unified search interface to understand the distinct jargon and context of each department, retrieving precise information from a massive, heterogeneous document corpus.

A financial services firm uses task arithmetic to create a multi-competency analyst model. Starting from a base LLM (e.g., Llama 3), they compute task vectors for:

• Sentiment Analysis of financial news.
• Numerical Reasoning for earnings reports.
• SEC Filing Summarization. These vectors are scaled and added to the base model's weights, creating a single model proficient in all three financial analysis tasks. This avoids the latency and cost of running an ensemble of separate models.

A software company uses Continual PEFT to keep its code-generation assistant updated. As new programming frameworks are released, they sequentially train small LoRA modules:

1. Adapter A for Framework v1.0.
2. Adapter B for Framework v2.0, while keeping Adapter A frozen.
3. Adapter C for a new cloud SDK. A task identifier at inference time selects the correct adapter, allowing the assistant to support multiple framework versions without catastrophic forgetting of previous knowledge. This is far more efficient than retraining the entire model for each update.

A healthcare AI platform uses a hierarchical adapter setup for processing clinical notes. A shared adapter is trained on general medical language understanding. Downstream, lightweight task-specific adapters are added for:

• Named Entity Recognition (extracting medications, conditions).
• Relation Extraction (linking a dosage to a drug).
• ICD-10 Code Prediction. The shared adapter provides a strong biomedical foundation, while the task-specific adapters enable precise, modular functionality. This design allows new tasks (e.g., clinical trial eligibility screening) to be added by training only a new small adapter.

A smartphone keyboard app uses multi-task LoRA for on-device personalization. A base language model is deployed on the device. Users can optionally enable personalization modules that are tiny LoRA matrices trained locally on their data for:

• Personal Writing Style (email vs. text message tone).
• Technical Jargon (for developers or doctors).
• Frequently Used Phrases & Slang. The device can store and switch between multiple LoRA modules, allowing highly personalized predictions without sending private data to the cloud or storing a massive, fully fine-tuned model for each user.

An adapter is a small, trainable neural network module (typically a two-layer feed-forward network with a bottleneck) inserted into the layers of a frozen pre-trained model. In multi-task PEFT, adapters provide the fundamental building blocks for task-specific adaptation.

• Function: Learns a compressed, task-specific representation.
• Multi-Task Role: Multiple independent adapters can be trained for different tasks on the same base model.
• Example: Training separate adapters for sentiment analysis, named entity recognition, and text summarization on a frozen BERT model.

AdapterFusion is a two-stage, knowledge-composition technique designed for multi-task learning. It first trains multiple task-specific adapters in isolation, then learns a secondary, shared fusion layer that dynamically combines them.

• Stage 1: Train adapters independently on different tasks (prevents negative interference).
• Stage 2: Freeze adapters and base model; train a lightweight attention-based mechanism to weigh and combine adapter outputs for a new task or for improved multi-task inference.
• Key Benefit: Enables knowledge transfer between tasks without catastrophic forgetting, as the original adapter knowledge is preserved.

Mixture-of-Adaptors is a conditional computation architecture that routes each input through a subset of multiple, specialized adapter modules using a learned gating network.

• Mechanism: A router network computes probabilities for each adapter. The final output is a weighted sum of the selected adapter outputs.
• Multi-Task Advantage: Allows a single model to develop a "library" of expert adapters. The gating mechanism can learn to invoke the correct expert(s) based on the input task, enabling efficient multi-task capacity within one model.
• Efficiency: More parameters than a single adapter, but only a fraction are activated per input, keeping inference costs manageable.

A task vector is the arithmetic difference between the weights of a model fine-tuned on a specific task and its original pre-trained weights (Δ = W_finetuned - W_base). It encapsulates the directional change needed for adaptation.

• Representation: Captures the "task concept" as a direction in parameter space.
• Multi-Task Application: Task vectors from different fine-tuned models (often created via PEFT) can be manipulated through task arithmetic (e.g., addition, scaling, negation) to merge competencies, create multi-task models, or mitigate biases.
• Example: Adding a summarization task vector and a translation task vector to a base model can yield a model with blended capabilities.

Model merging is the technique of combining the parameters from multiple fine-tuned models into a single, unified model. In the context of PEFT, merging typically involves combining lightweight adapters or task vectors rather than full model weights.

• Methods: Includes simple weight averaging, task arithmetic, and more sophisticated linear mode connectivity approaches.
• PEFT Advantage: Merging small adapters or LoRA matrices is computationally trivial and avoids the interference often seen when merging fully fine-tuned models.
• Outcome: Creates a multi-task capable model without additional training, though performance can be task-dependent.

Continual PEFT addresses the sequential learning of multiple tasks over time, a core challenge in multi-task scenarios where tasks arrive in a stream. It uses parameter-efficient methods to mitigate catastrophic forgetting.

• Strategy: Assign a new, small adapter (or LoRA module) to each incoming task. The base model and previous adapters are frozen.
• Memory Management: Requires a growing repository of task-specific modules. Techniques like adapter pruning or consolidation can manage this growth.
• Inference: For task identification, an external classifier or a learned router (like in MoA) selects the appropriate historical adapter to use.