Task Arithmetic

A task vector is the core mathematical object in task arithmetic. It is defined as the parameter difference between a model fine-tuned on a specific task and the original pre-trained base model: τ = θ_fine-tuned - θ_base. This vector τ encodes the directional change in weight space required for the model to perform the new task. In PEFT methods like LoRA, this vector is inherently low-rank and compact, making arithmetic operations computationally efficient.

The fundamental operation is the element-wise linear combination of multiple task vectors. For tasks A and B, a combined model can be created as: θ_combined = θ_base + ατ_A + βτ_B. The coefficients α and β control the influence of each task. This linearity often holds surprisingly well in the weight space of over-parameterized models, allowing for the composition of learned behaviors without retraining.

Task arithmetic supports the negation of a task vector to remove a learned capability. Applying θ_base - τ_A can effectively "unlearn" or de-emphasize task A, reverting the model closer to its base state or mitigating unwanted behaviors learned during fine-tuning. This is a form of targeted model editing and is foundational for task forgetting and bias mitigation.

By scaling a task vector with a coefficient λ, practitioners can interpolate the strength of a task's influence. For example, θ_base + λτ_A where 0 < λ < 1 applies a weaker adaptation, potentially useful for balancing task performance or for progressive model merging. This allows fine-grained control over how much the base model's behavior is altered.

The success of linear arithmetic depends on the degree of orthogonality between task vectors. When vectors for different tasks point in similar directions in weight space, they interfere minimally, enabling clean composition. Significant interference or conflict occurs when vectors are negatively correlated, leading to performance degradation on one or both tasks when combined. Analyzing vector similarity is key to predicting composability.

Task arithmetic is naturally suited to LoRA. Instead of combining full model weights, you combine the small low-rank adapter matrices (ΔW = BA). The merged model for inference is created by summing the base weights with the combined LoRA deltas: W' = W + Σ(λ_i * B_i A_i). This is extremely efficient, as the base model remains static and multiple lightweight adapters can be stored, combined, and applied on-the-fly.

The primary application of task arithmetic is to combine specialized adapters (e.g., LoRA modules) into a single model that performs multiple tasks without further training. This is achieved by linearly combining task vectors—the difference between a fine-tuned model's weights and the base model's weights. For example, adding a translation task vector to a summarization task vector can yield a model capable of both operations. This approach bypasses the need for costly multi-task joint training on a combined dataset.

Task arithmetic allows for precise model editing by adding or subtracting specific capabilities. Developers can:

• Add a skill: Model_New = Base + Vector_TaskA
• Remove an unwanted behavior: Model_Clean = Model_FineTuned - Vector_Bias
• Interpolate between traits: Model_Blended = Base + αVector_Task1 + βVector_Task2 This enables the modular assembly of model behaviors, treating fine-tuned adaptations as lego blocks that can be attached or detached. It's foundational for creating customized AI agents with tailored skill sets.

In continual learning scenarios, where a model is sequentially trained on new tasks, task arithmetic provides a mechanism to preserve prior knowledge. Instead of fine-tuning the base model directly on a new task (which overwrites weights), a separate adapter is trained. All previous task adapters remain intact. The model can then perform inference by dynamically switching or merging the relevant adapters. This maintains a library of task vectors, effectively eliminating catastrophic forgetting by design, as the base model parameters are never updated.

Task arithmetic enables rapid personalization at scale. A single, large base model can serve multiple users or domains by storing only small, user-specific task vectors. For instance:

• A financial services firm can maintain a base legal document model and add client-specific jargon adapters.
• A healthcare provider can combine a base clinical language model with vectors for different specialties (radiology, cardiology). Deployment involves simply loading the base model and the relevant additive vectors, drastically reducing storage and serving costs compared to maintaining dozens of fully fine-tuned model copies.

Beyond adding skills, task arithmetic can subtract undesirable attributes. By treating problematic model behaviors—such as generating toxic language or exhibiting demographic bias—as a "task," a negative task vector can be computed. Applying this vector via subtraction (Base - Vector_Bias) can reduce the unwanted behavior. This is a form of targeted unlearning that is more efficient and precise than retraining from scratch or applying broad post-hoc filters. It allows for surgical correction of model outputs.

Task arithmetic is the mathematical precursor to advanced model merging techniques. Methods like Task Arithmetic, TIES-Merging, and DARE use linear combinations of task vectors as a core operation. These techniques go beyond simple addition by resolving sign conflicts and pruning redundant parameters across vectors. The result is a single, consolidated model that retains high performance across all source tasks. This makes task arithmetic a critical component in the scalable development of generalist models from collections of specialists.

Delta weights (ΔW) represent the learned parameter change applied to a frozen pre-trained model. In PEFT, ΔW is constrained to be highly efficient—such as a low-rank matrix in LoRA—capturing the essential task-specific adaptation without modifying the original weights.

• Core Concept: The mathematical difference between the fine-tuned and base model weights: ΔW = W_finetuned - W_base.
• Storage: Stored separately, enabling modular reuse and combination.
• Foundation for Arithmetic: Task arithmetic linearly combines these ΔW from different tasks to create multi-task models.

Model merging is the process of combining parameters from multiple neural networks into a single unified model. It is a broader technique that encompasses task arithmetic.

• Methods: Includes weight averaging, task vector addition (task arithmetic), and more sophisticated linear mode connectivity approaches.
• Goal: To create a model that exhibits blended capabilities from its source models without additional multi-task training.
• Contrast with Task Arithmetic: General merging may involve full model weights, while task arithmetic specifically manipulates the efficient delta weights (ΔW) from PEFT runs.

A task vector is the parameter delta (ΔW) obtained from fine-tuning a model on a specific task, conceptualized as a direction in weight space. Task arithmetic treats these vectors as composable units.

• Representation: τ = θ_task - θ_base, where θ are model parameters.
• Properties: Vectors can be added, subtracted, and scaled. For example, τ_multi = τ_taskA + τ_taskB.
• Interpretation: Moving the base model's weights along the task vector direction imbues it with that task's capability.

Multi-task fine-tuning trains a single model on multiple objectives simultaneously using a combined dataset. This contrasts with task arithmetic, which combines already-trained single-task adapters.

• Traditional Approach: Requires a curated, balanced multi-task dataset and joint training, which is computationally expensive.
• Data Mixing Challenge: Performance can be sensitive to the dataset mixture ratios and task conflicts.
• Advantage of Arithmetic: Task arithmetic bypasses the need for simultaneous data access and training, composing existing efficient adapters post-hoc.

Modular adaptation is a design paradigm where a model is extended with small, self-contained, and often interchangeable neural modules (like LoRA adapters) for each new task or skill.

• Core Idea: The base model acts as a shared, frozen backbone. Task-specific modules are plugged in and out dynamically.
• Infrastructure Enabler: Enables efficient storage, versioning, and deployment of many specialized model variants.
• Relation to Task Arithmetic: Provides the modular components (adapters/deltas) that are the operands for arithmetic operations, treating adaptation as a modular algebra.

Catastrophic forgetting is the tendency of a neural network to lose performance on previously learned tasks when trained on new data. Task arithmetic and PEFT are strategies to mitigate this.

• Cause in Full Fine-Tuning: Large, unconstrained updates to network weights overwrite representations critical for old tasks.
• PEFT Mitigation: By constraining updates to a small set of parameters (e.g., low-rank adapters), the core representations in the base model remain largely intact.
• Arithmetic's Role: Allows for the preservation of single-task adapters. A base model can be adapted to Task B without destroying the separately stored adapter for Task A.