Task Arithmetic

The core unit of task arithmetic is the task vector, defined as the element-wise difference between a fine-tuned model's weights and the original pre-trained model's weights: τ = θ_fine-tuned - θ_base. This vector encodes the directional change in parameter space required for a specific task. It represents the 'knowledge' added by fine-tuning, isolated from the base model's general capabilities.

Task arithmetic operates on the empirical observation that, in overparameterized models like LLMs, task vectors often reside in approximately linear subspaces. This means adding multiple task vectors to a base model can confer multiple competencies. For example: θ_merged = θ_base + α * τ_task1 + β * τ_task2. The scaling coefficients (α, β) control the influence of each task, allowing for balancing and interpolation.

A critical property is that subtracting a task vector can remove or reduce a specific capability. If θ_toxic = θ_base + τ_toxicity, then θ_base ≈ θ_toxic - τ_toxicity. This principle is used for 'model editing' and safety unlearning, where undesirable behaviors learned during fine-tuning are arithmetically subtracted. It provides a computationally cheap alternative to retraining from scratch.

Performance depends on the interference between task vectors. High-performing merges occur when vectors are relatively orthogonal, meaning they modify different parameter subspaces. Significant overlap leads to catastrophic interference, where one task degrades another. Research focuses on measuring vector similarity and developing re-scaling or projection methods to minimize negative interference during arithmetic operations.

The scaling coefficients applied to task vectors are not always 1.0. Optimal coefficients often differ per task and model. A common finding is that slightly dampening coefficients (e.g., using 0.3-0.8) improves multi-task robustness. Finding these coefficients can be done via:

• Grid search on a small validation set
• Using the Task Arithmetic paper's proposed normalization based on weight magnitudes
• Learning coefficients via a small hypernetwork or gradient-based search

Task arithmetic is a direct application of the delta tuning paradigm, where adaptation is conceptualized as learning a parameter change (delta). It provides a mathematical foundation for model merging techniques like:

• Model Soups: Averaging multiple fine-tuned checkpoints.
• Adapter/Gating Composition: Arithmetically combining small adapter modules.
• Task Arithmetic demonstrates that complex behaviors can be composed through simple, memory-efficient weight operations, advancing the vision of modular, composable AI systems.

Task arithmetic enables the creation of a single, unified model from multiple specialized checkpoints. By adding scaled task vectors—calculated as (fine-tuned weights) - (base weights)—from models fine-tuned on distinct tasks (e.g., sentiment analysis, named entity recognition, code generation), a composite model can perform all tasks without a proportional increase in parameters or inference latency. This is critical for production deployments where serving multiple separate models is infeasible.

• Key Benefit: Eliminates the need to maintain and load-switch between multiple large model instances.
• Example: Merging a legal contract analysis adapter with a financial report summarization adapter into a single model for a banking AI assistant.

A core challenge in multi-task learning is catastrophic forgetting, where learning a new task degrades performance on previously learned ones. Task arithmetic provides a mechanism to computationally "subtract" undesired behaviors. If a model develops a harmful bias or unwanted stylistic trait during fine-tuning, the corresponding negative task vector can be subtracted from the weights to revert the change, preserving core capabilities.

• Process: Corrected Weights = Current Weights - (Bias-Infused Weights - Original Weights)
• Use Case: Removing a tendency for a customer service model to generate overly formal or verbose responses after fine-tuning on a specific dataset.

Instead of costly retraining or reinforcement learning, task arithmetic allows for precise, surgical updates to model knowledge and behavior. Engineers can create small edit vectors that represent factual updates (e.g., a new company CEO) or stylistic adjustments (e.g., a more concise tone) and add them to the base model. This is far more parameter-efficient than full fine-tuning and allows for modular, auditable changes.

• Advantage: Enables rapid A/B testing of model behaviors by adding or subtracting different behavioral vectors.
• Application: Updating a retrieval-augmented generation (RAG) model's internal knowledge base without retraining the entire architecture.

Organizations can maintain one large, general-purpose base model (e.g., Llama 3) and a library of lightweight task vectors for different departments. At inference time, the system dynamically combines the base model with the relevant task vector(s). This architecture centralizes core model maintenance while allowing decentralized, agile adaptation for various use cases like marketing copy generation, SQL query creation, or technical support.

• Infrastructure Model: Central Base Model + Distributed Library of Task Deltas.
• Cost Benefit: Dramatically reduces storage and GPU memory requirements compared to storing dozens of fully fine-tuned 70B parameter models.

Research indicates that averaging multiple task vectors from related tasks can create a more robust and generalizable merged vector. This technique, sometimes called model souping, smooths out task-specific noise and captures the shared underlying structure between domains. The resulting model often outperforms individual task-specific models on held-out or related evaluation sets.

• Mechanism: Generalized Vector = (Task_Vector_A + Task_Vector_B + Task_Vector_C) / 3
• Practical Example: Averaging vectors from models fine-tuned on different programming languages (Python, JavaScript, SQL) to create a general-purpose code assistant with stronger cross-language understanding.

Task arithmetic is not exclusive to full-model fine-tuning. It can be applied to the small adapters created by methods like LoRA or Adapter modules. Engineers can perform arithmetic (addition, scaling, interpolation) on the low-rank matrices of multiple LoRA adapters to merge their functionalities. This creates a highly efficient, multi-talented model where the base weights remain completely frozen, and only merged adapter weights are stored and loaded.

• Workflow: 1. Train separate LoRA adapters for Task A and Task B. 2. Perform task arithmetic on the LoRA delta matrices. 3. Load base model + merged LoRA for inference.
• Result: Achieves multi-task capability with the ultra-low parameter footprint characteristic of LoRA.

A task vector is the foundational mathematical object in task arithmetic. It is defined as the element-wise difference between the weights of a model fine-tuned on a specific task and the weights of the original pre-trained base model: Δ = W_finetuned - W_base. This vector represents the precise directional change in parameter space required for task adaptation. In task arithmetic, these vectors are treated as first-class citizens that can be added, subtracted, or scaled to combine competencies.

Model merging is the overarching technique of combining multiple neural networks into a single, unified model. Task arithmetic is a specific, weight-space method within this category. Other merging approaches include:

• Surgical merging: Combining only specific layers or modules from different models.
• Averaging: Simple interpolation of weights from models trained on similar tasks.
• Fisher merging: Weighting parameter updates by their estimated importance (Fisher Information). The goal is to create a multi-competency model without additional joint training.

Delta tuning is the general paradigm of adapting a pre-trained model by learning a small set of parameter changes (the 'delta'), rather than updating all weights. Task arithmetic directly manipulates these deltas (task vectors). This paradigm includes all Parameter-Efficient Fine-Tuning (PEFT) methods, such as:

• LoRA: Learns low-rank matrices that approximate the delta.
• Adapters: Inject small, trainable modules whose parameters constitute the delta.
• Prefix/Prompt Tuning: Learns input embeddings that act as a virtual delta. Delta tuning makes task vector extraction and arithmetic feasible.

Multi-task learning (MTL) is the traditional approach of training a single model on multiple tasks simultaneously, sharing representations to improve generalization. Task arithmetic offers a post-hoc alternative. Key comparisons:

• Traditional MTL: Requires concurrent access to all task datasets and joint optimization. Can suffer from negative interference between tasks.
• Task Arithmetic: Enables 'multi-task' capability by merging independently fine-tuned models after training. This is often called multi-task model merging and avoids interference during training, but may face capacity limits during merging.

Weight interpolation is a simple form of model merging where the parameters of two or more models are combined via a weighted linear combination: W_merged = α * W_A + (1-α) * W_B. Task arithmetic generalizes this concept. While interpolation typically blends models trained on similar distributions (e.g., different checkpoints), task arithmetic uses vector addition (W_base + Δ_A + Δ_B) to explicitly combine distinct task adaptations. Scaling task vectors (e.g., λ * Δ) is a direct form of interpolation along a specific task direction.

Catastrophic forgetting is the tendency of a neural network to abruptly lose previously learned information when trained on a new task. Task arithmetic and related merging techniques provide a potential mitigation strategy within continual learning. Instead of sequentially overwriting weights, each task is learned in isolation (creating a task vector), and models are merged post-hoc. However, naive addition of multiple task vectors can lead to interference or performance degradation, analogous to forgetting, which advanced arithmetic (like task negation or magnitude scaling) aims to address.