Task Arithmetic

The foundational operation of task arithmetic is the linear combination of task vectors. A task vector (τ) is defined as the difference between a model fine-tuned on a task and the base model: τ = θ_fine-tuned - θ_base. Core operations include:

• Addition: τ_A + τ_B adapts the model to perform both tasks A and B.
• Negation: θ_base - τ_A can "forget" or negate a learned task.
• Interpolation: ατ_A + (1-α)τ_B creates a blended model behavior. This linearity assumes task vectors occupy roughly orthogonal directions in parameter space, allowing for relatively clean composition.

Task arithmetic operates directly in the high-dimensional parameter space of the neural network, not in the output or activation space. This makes it a form of model surgery at the weight level. The technique edits the model's fundamental knowledge representation by:

• Applying vector arithmetic to the weight matrices of the model.
• Assuming the loss landscape is locally linear around the pre-trained weights for related tasks.
• Enabling edits that persist across all possible inputs, unlike prompt-based methods which are input-dependent.

A primary advantage is the composability of skills. By adding task vectors, one can theoretically create a model capable of a new, combined task without training on it directly. For example:

• Adding a sentiment analysis vector to a summarization model could yield a model that summarizes with emotional tone.
• This facilitates rapid prototyping of multi-task models and exploration of the model capability space. The success of composition depends on the interference or compatibility between the constituent tasks.

Task arithmetic enables controlled knowledge forgetting or behavior negation through vector subtraction. This is critical for:

• Bias Mitigation: Subtracting a vector representing a biased association (e.g., gender stereotypes) from the model weights.
• Task Removal: De-adapting a model from a specific fine-tuned task to revert towards base model behavior.
• Safety Editing: Reducing the propensity for harmful outputs by subtracting a "harmful behavior" vector. Negation is not always perfect, as task vectors may not be perfectly reversible, but it provides a direct editing mechanism.

The efficacy of task arithmetic relies on the empirical observation that task vectors for distinct tasks are often approximately orthogonal in parameter space. This orthogonality minimizes catastrophic interference when vectors are combined. Key implications:

• Independent Learning: Tasks fine-tuned from the same base model tend to lie in nearly orthogonal directions.
• Interference Metric: The cosine similarity between task vectors predicts the success of their combination; low similarity (high orthogonality) yields better results.
• Limitation: For very similar or conflicting tasks, vectors may not be orthogonal, leading to performance degradation upon combination.

It is a highly parameter-efficient and compute-efficient post-training method. Once task vectors are created (via standard fine-tuning), new model configurations can be generated instantly without gradient updates. This enables:

• Zero-Shot Task Composition: Creating models for new task combinations at inference time by simply adding stored vectors.
• Lightweight Deployment: Storing and swapping small delta vectors (<1% of full model size) instead of multiple full model checkpoints.
• Rapid Experimentation: Testing hundreds of potential model behaviors through simple arithmetic, bypassing costly training runs.

A primary application is bias removal. If a model exhibits a gender bias in a profession classification task, a bias task vector can be created by fine-tuning on a biased dataset. Applying task vector negation (subtracting this vector from the base model) reduces the undesired behavior.

• Process: ΔW_bias = W_biased - W_base; then W_debiased ≈ W_base - α * ΔW_bias.
• Result: The model retains its core linguistic capabilities while exhibiting reduced stereotypical associations, a form of post-hoc model editing.

Task arithmetic can combine distinct capabilities. For instance, a base language model can be adapted for both code generation (task A) and text summarization (task B).

• Process: Create independent task vectors: ΔW_code and ΔW_summarize. A new model for a "documentation generation" task can be formed via linear combination: W_new = W_base + α * ΔW_code + β * ΔW_summarize.
• Key Insight: This performs multi-task adaptation without multi-task training, merging skills by adding their corresponding deltas. The scaling factors (α, β) control the influence of each skill.

Interpolation between task vectors allows for smooth control over model behavior. This is useful for creating models with blended expertise or adjusting style.

• Example: Interpolate between a formal writing vector and a casual writing vector. W_blended = W_base + λ * ΔW_formal + (1-λ) * ΔW_casual.
• Application: By varying λ from 0 to 1, the model's output tone transitions smoothly from casual to formal. This demonstrates how task arithmetic enables continuous control over model attributes, facilitating fine-grained behavioral tuning.

An anti-task vector represents a direction in weight space that, when added, reduces proficiency in an undesirable task (e.g., generating harmful content).

• Method: An anti-task vector can be approximated as the negation of a safety-alignment vector: ΔW_anti-harm ≈ -(W_aligned - W_base).
• Use Case: In a multi-vector safety setup, a model's weights could be composed as W_safe = W_base + ΔW_capability + ΔW_anti-harm. This modular approach separates skill acquisition from safety enforcement, a principle in modular adaptation.

In a sequential learning setting, task arithmetic can help mitigate catastrophic forgetting. After learning Task B, the model's knowledge of previous Task A can be partially restored.

• Process: Store the task vector for Task A: ΔW_A. After fine-tuning on Task B to get W_B, an approximation of a multi-task model is: W_AB ≈ W_B + γ * ΔW_A.
• Limitation: This is an approximation, as tasks may not be perfectly linearly separable in weight space. However, it provides a computationally cheap method for task rehearsal without accessing old data.

The magnitude of a task vector matters. Empirical studies show that applying a task vector with a scaling coefficient (α ≠ 1) is often necessary for optimal performance.

• Key Finding: Simple addition (α=1) often underperforms. A common practice is to search for an optimal α (e.g., 0.8, 1.2) on a validation set.
• Renormalization: Some implementations apply weight normalization after vector addition to stabilize the merged model. This step counteracts potential drift in the model's output distribution caused by the arithmetic operation.

The overarching paradigm where a model is adapted by learning a small, task-specific parameter change (a delta) applied to a subset of the pre-trained model's weights. The core principle is additive parameterization, where adapted weights (W') are expressed as W' = W₀ + ΔW, with the base weights W₀ frozen.

• Key Insight: Enables efficient adaptation by isolating and optimizing only the necessary change.
• Contrast with Full Fine-Tuning: Updates <1% of parameters versus 100%, drastically reducing memory and storage costs.
• Foundation for Task Arithmetic: Provides the mathematical framework (the delta ΔW) that task vectors manipulate.

The mathematical object central to task arithmetic. A task vector is typically defined as the difference between a fine-tuned model's weights and the base model's weights: τ = θ_task - θ_base. This vector encodes the direction and magnitude of change needed for a specific task.

• Representation: Can be a full weight delta or a compressed representation (e.g., from a PEFT method like LoRA).
• Properties: Enables linear operations; adding a task vector to base weights instills a task, while negation can remove or invert a behavior.
• Use Case: The fundamental unit for model editing and multi-task composition via arithmetic operations.

A class of techniques for making precise, localized updates to a neural network's knowledge or behavior after its initial training. The goal is to correct errors, update facts, or adjust capabilities without costly retraining.

• Scope: Ranges from modifying individual neurons or layers to injecting modular components.
• Connection to Task Arithmetic: Task arithmetic is a form of parametric model editing, where edits are performed by directly manipulating the model's weight space through vector operations.
• Example: Using task vector negation to 'unlearn' a biased association learned during fine-tuning.

A foundational mathematical strategy for delta tuning where the adapted model's weights are represented as the sum of the original frozen weights and a learned delta matrix. This explicit separation is what makes delta manipulation possible.

• Formula: W_adapted = W_base + ΔW.
• Advantage: The delta ΔW is often highly structured (e.g., low-rank, sparse), making it efficient to store and combine.
• Implementation: Directly employed by LoRA (ΔW = BA), (IA)³ (ΔW as scaling vectors), and is the implicit assumption behind task vector arithmetic.

A design philosophy for PEFT that extends a base model with small, self-contained, and often composable neural modules (like adapters or expert layers). Each module encapsulates a specific skill or task adaptation.

• Core Idea: Create a library of reusable, plug-and-play components.
• Contrast with Blending: Unlike task arithmetic which blends weight deltas, modular adaptation often routes inputs to distinct modules (conditional computation).
• Related Techniques: AdapterFusion (learns to combine adapters), Mixture-of-Experts (dynamic routing to expert modules). Task arithmetic offers a weight-space alternative to this modularity.

A secondary neural network that generates the weights or parameters for a primary (main) model. In PEFT, a hypernetwork can dynamically produce the task-specific delta (e.g., adapter weights) conditioned on a task descriptor.

• Mechanism: Takes a low-dimensional intrinsic task vector as input and outputs the full set of adaptation parameters ΔW for the main model.
• Benefit: Decouples the size of the task representation from the size of the main model, offering extreme parameter efficiency.
• Relation to Task Arithmetic: Provides a learned, non-linear function for generating deltas, whereas task arithmetic uses direct, linear operations on pre-existing deltas.