Task Arithmetic

The core operation is computing the task vector, defined as the difference between a model fine-tuned on a specific task and the original pre-trained base model: τ = θ_fine-tuned - θ_base. This vector represents the directional change in parameter space required for the model to perform the new task. The arithmetic is performed on these deltas, not the raw weights, isolating the learned adaptation.

Multi-task models are created via a weighted sum of task vectors. For tasks A and B, the merged model parameters are: θ_merged = θ_base + ατ_A + βτ_B. The coefficients α and β control the influence of each task. This linearity is surprisingly effective for combining diverse capabilities like translation and question answering, assuming tasks are not in direct conflict within the model's parameter space.

A key advantage over sequential fine-tuning is the mitigation of catastrophic forgetting. Since the base model's parameters remain a central anchor and task-specific changes are added independently, the merged model retains a strong connection to the original, general-purpose knowledge. This makes task arithmetic a form of multi-task learning performed after the individual single-task training phases are complete.

The method is highly efficient post-training. The expensive step is the initial fine-tuning of individual models, which can be done in parallel. The merging itself is a simple, fast linear operation with no gradient computations, requiring minimal compute compared to training a multi-task model from scratch or using adapter composition. This makes it viable for rapid prototyping of model capabilities.

Arithmetic operations extend beyond addition. Task negation (subtracting a task vector) can be used to reduce or remove a capability, which is useful for safety editing or unlearning. Scaling a task vector (multiplying by a factor >1 or <1) can amplify or dampen its influence. This provides a simple, interpretable knob for adjusting model behavior post-hoc.

Performance depends on the orthogonality of task vectors. When tasks conflict (their gradients point in opposing directions), simple addition leads to interference and performance degradation. The method assumes the linear mode connectivity of loss landscapes—that fine-tuned models lie in a basin where linear interpolation remains low-loss. It works best for tasks that are sufficiently distinct from the base model's knowledge and from each other.

Model merging is the overarching technique of combining the parameters of multiple neural networks into a single unified model. Unlike task arithmetic, which specifically manipulates fine-tuning deltas, merging can involve various strategies:

• Linear Interpolation: Directly averaging the weights of two or more models.
• Task Vectors: Extracting the difference between a fine-tuned model and its base pre-trained model, which is the foundational concept for task arithmetic.
• Slerp (Spherical Linear Interpolation): Interpolating weights on a hypersphere, often producing smoother blends than linear interpolation. The goal is to consolidate capabilities—such as coding and reasoning—without the compute cost of multi-task training.

A task vector is the mathematical representation of the changes a model undergoes during fine-tuning for a specific task. It is calculated as the element-wise difference between the fine-tuned model's parameters and the original pre-trained model's parameters: τ = θ_fine-tuned - θ_base.

• Core Component: Task vectors are the fundamental operands in task arithmetic; adding or subtracting them performs the 'arithmetic'.
• Directional Semantics: The vector's direction in parameter space encodes the learned task. Adding vectors combines tasks, while subtracting can negate or remove a behavior.
• Scaling: Task vectors can be scaled (e.g., α * τ) to control the strength of the task's influence in the merged model.

Parameter-Efficient Fine-Tuning (PEFT) methods, such as LoRA and Adapters, are highly compatible with task arithmetic. Instead of updating all model weights (θ), PEFT methods learn a small set of additive parameters (Δ).

• Efficient Vectors: The PEFT Δ parameters act as a natural, compact task vector, making arithmetic operations more memory-efficient.
• Isolation: Since the base model remains frozen, PEFT deltas cleanly represent the task-specific knowledge without interference from other changes.
• Merging PEFT Modules: Techniques like Merging PEFT Adapters are a direct application of task arithmetic, where LoRA matrices from different tasks are summed before being reintegrated into the base model.

Catastrophic forgetting is the tendency of a neural network to abruptly lose previously learned information when trained on a new task. Task arithmetic offers a potential mitigation strategy.

• Problem: Traditional sequential fine-tuning on task B often degrades performance on task A.
• Arithmetic as a Solution: By treating tasks as independent vectors (τ_A, τ_B), they can be combined (θ_base + τ_A + τ_B) without the destructive interference typical of sequential training. The base model's general knowledge is preserved, and tasks are combined post-hoc.
• Limitation: Simple addition may not perfectly preserve both tasks if their parameter-space directions conflict, leading to a need for more advanced merging techniques.

Multi-Task Learning (MTL) is the conventional approach of training a single model on multiple tasks simultaneously from scratch, sharing representations across tasks. Task arithmetic provides a post-training alternative.

• MTL Process: Requires a curated, balanced multi-task dataset and joint training, which is computationally expensive.
• Arithmetic Contrast: Task arithmetic creates a multi-task model after individual single-task models are fine-tuned. It bypasses the need for complex joint training data and dynamic loss balancing.
• Trade-off: MTL often leads to better shared representations through co-training, while arithmetic can be more flexible and modular but may not achieve the same level of synergistic optimization.

Model souping is a simple model merging technique where the weights of multiple models fine-tuned from the same base (often with different hyperparameters or data orders) are averaged.

• Relation to Arithmetic: Task arithmetic can be seen as a generalized, more principled form of souping. While souping averages entire parameter sets (θ_soup = mean(θ_1, θ_2, ...)), task arithmetic averages the deltas (θ_merged = θ_base + mean(τ_1, τ_2, ...)).
• Use Case: Souping is primarily used for ensembling to improve robustness and generalization on a single task. Task arithmetic is explicitly designed for combining different tasks.
• Performance: Research indicates that averaging fine-tuned checkpoints (souping) often outperforms the best individual checkpoint, providing a strong baseline for merging techniques.