Task Vectors

A task vector is formally defined as the arithmetic difference between the weight tensors of a fine-tuned model and its original pre-trained base model: τ = W_finetuned - W_base. This delta (Δ) represents the precise directional update learned during adaptation. It is a directional quantity, indicating how the model's function space was shifted, not just the magnitude of change.

A key empirical finding is that task vectors often exhibit approximate linearity. This means the vector for a combined task (e.g., 'creative writing in French') can be approximated by adding the vectors for its constituent tasks ('creative writing' + 'French translation'). This property enables:

• Model Merging: Combining multiple fine-tuned models by adding their task vectors to a base model.
• Task Interpolation: Creating models with blended capabilities by scaling and adding vectors (e.g., W_base + α*τ_task1 + β*τ_task2).
• Negation: Applying a negative task vector can 'forget' or reduce a specific skill.

While a task vector is theoretically the same size as the full model (billions of parameters), the effective information it contains is often highly sparse. Most weight changes are near zero, meaning the adaptation is concentrated in a small subset of parameters. This sparsity is the theoretical foundation for parameter-efficient fine-tuning (PEFT) methods like LoRA and Adapters, which explicitly learn a low-dimensional representation of this delta. The task vector concept validates that full fine-tuning is often computationally wasteful.

Task vectors encode task-specific knowledge. When added to the base model, they specialize it. However, their utility depends on the orthogonality of the tasks. Adding vectors for conflicting tasks (e.g., sentiment classification and machine translation) can lead to interference and performance degradation. Techniques like task arithmetic often require careful scaling or projection to manage this. The vector's direction in weight space is more critical than its magnitude for preserving the model's general world knowledge.

Task vectors are not exclusive to full fine-tuning. They can be derived from PEFT modules. For example:

• A LoRA adapter's matrices (A and B) represent a low-rank factorization of a task vector: ΔW ≈ B*A.
• An Adapter module's weights directly constitute a task-specific delta for its layer. This allows for efficient storage, sharing, and composition of adaptations. A library of PEFT modules is essentially a library of compressed task vectors.

Beyond multi-task merging, task vectors enable precise model editing. A vector can be engineered to correct factual errors or update knowledge without retraining. Conversely, they facilitate machine unlearning: applying a negative vector derived from a 'forget' dataset can reduce a model's performance on specific data, aiding compliance with data removal requests. This treats the model as a weight space that can be navigated with vector operations.

Task vectors enable the creation of a single model proficient in multiple tasks by arithmetically combining vectors. This is achieved through task arithmetic, where vectors from different fine-tuned models are added to the base model's weights. For example, adding a translation vector and a summarization vector can yield a model capable of both tasks. This approach is far more parameter-efficient than training separate expert models or a monolithic multi-task model.

• Key Benefit: Achieves positive task transfer where combined skills can outperform individually fine-tuned models.
• Mechanism: W_combined = W_base + α * V_taskA + β * V_taskB
• Use Case: Building a unified assistant from specialized models for translation, summarization, and sentiment analysis.

Task vectors can be used to remove or dampen undesired behaviors learned during fine-tuning. By subtracting a task vector, you can effectively "unlearn" a capability. This is crucial for:

• Safety and Alignment: Negating vectors associated with harmful outputs or biased behavior.
• Domain De-specialization: Reverting an over-specialized model back towards its more general, robust base state.
• Mechanism: W_safer ≈ W_fine_tuned - γ * V_unsafe_behavior

This use case provides a computationally cheap method for model editing and corrective adaptation post-deployment, acting as a form of counterfactual fine-tuning.

By scaling a task vector, engineers can control the strength of a learned adaptation. This allows for smooth interpolation between model behaviors.

• Behavioral Knob: A scaling factor λ acts as a dial, where λ=1 applies the full task adaptation and λ=0 reverts to the base model. Values between 0 and 1 create intermediate behaviors.
• Use Case: Adjusting the formality or creativity of a text generation model dynamically at inference time.
• Application in Ensembles: Creating a continuous spectrum of models from a single base and task vector, enabling efficient behavioral search without retraining.

This makes task vectors a powerful tool for controllable generation and A/B testing of model personalities.

In continual learning, a model must learn sequential tasks without forgetting previous ones (catastrophic forgetting). Task vectors offer an elegant solution:

• Isolated Storage: Each new task is learned as a separate, compact task vector, while the base model remains frozen.
• Composition at Inference: To perform a previous task, the corresponding vector is added to the base weights: W_for_taskN = W_base + V_taskN.
• Benefit: Completely avoids parameter interference, the root cause of forgetting, as each task's update is stored independently.

This approach, sometimes called modular continual learning, allows for an unbounded number of tasks with constant base model memory footprint, storing only small deltas per task.

Task vectors serve as a diagnostic tool for understanding what and how a model learns during fine-tuning. By analyzing the vector's properties, researchers can:

• Measure Task Similarity: Compute the cosine similarity between different task vectors. High similarity suggests the model uses overlapping mechanisms for both tasks.
• Identify Critical Parameters: Locate weights with the largest magnitude changes in the vector, revealing network components most crucial for the task.
• Study Interference: Analyze the dot product between vectors to predict negative transfer (if vectors point in opposing directions).

This turns task vectors into a lens for mechanistic interpretability, moving beyond black-box adaptation.

Task vectors are the fundamental building block for advanced model merging techniques like Task Arithmetic, TIES-Merging, and DARE. These methods go beyond simple averaging:

• TIES-Merging: 1) Trims each task vector to retain only the most significant changes, 2) Elects the sign (direction) of conflicting updates via majority vote, 3) Scales and merges them.
• DARE: Drops random elements of task vectors to zero (sparsification) and rescales them before merging, improving stability.
• Outcome: Creates a single, robust model that retains high performance across all source tasks, enabling the consolidation of multiple specialized fine-tunes into a unified, generalist model for efficient deployment.

Delta tuning is the overarching paradigm for parameter-efficient fine-tuning (PEFT) that focuses on learning a small set of parameter changes—the 'delta'—to apply to a frozen base model. A task vector is the canonical example of a delta: the arithmetic difference between fine-tuned and pre-trained weights. This paradigm includes all methods that learn a compact representation of the necessary adaptation, such as Low-Rank Adaptation (LoRA), adapters, and prefix tuning, rather than updating the full parameter set.

Model merging is the technique of combining the weights or adapter modules from multiple fine-tuned models into a single, unified model. This is often done to create a multi-task model or to improve generalization. Task arithmetic is a specific model merging method that operates directly on task vectors. For example, to create a model proficient in both translation and summarization, one can add the respective task vectors to the base model's weights: W_merged = W_base + α * V_translation + β * V_summarization, where α and β are scaling coefficients.

Low-Rank Adaptation (LoRA) is a specific, highly popular delta tuning method that provides a parameter-efficient way to create a task vector. Instead of computing the full dense update ∆W, LoRA constrains it to a low-rank decomposition: ∆W = B * A, where A and B are small, trainable low-rank matrices. The final adapted weights are W = W_base + B*A. This makes LoRA an efficient implementation for generating a compact task vector, drastically reducing the number of trainable parameters compared to full fine-tuning.

An adapter is a small, trainable neural network module (typically a two-layer feed-forward network with a bottleneck) that is inserted into the layers of a frozen pre-trained model. During fine-tuning, only the adapter parameters are updated. The functional change introduced by the adapter module to the model's forward pass can be conceptually viewed as a structured, modular form of a task vector. Instead of a diffuse change across all weights, the adaptation is localized to specific, injected components.

Weight-Decomposed Low-Rank Adaptation (DoRA) is an advanced PEFT method that refines the LoRA approach. It first decomposes a pre-trained weight matrix W into a magnitude vector m and a directional matrix V. DoRA then applies LoRA only to the directional component V and learns a separate scaling factor for m. This decomposition allows for more stable and effective fine-tuning. The DoRA update can be seen as a more sophisticated, structured task vector that separately modifies the magnitude and direction of pre-trained weights.

Sparse fine-tuning is a parameter-efficient strategy where updates are applied to only a selected, sparse subset of the model's original parameters. Methods like BitFit, which updates only bias terms, are a form of sparse tuning. This creates a task vector that is inherently sparse—most of its elements are zero. The key difference from a dense task vector (from full fine-tuning) or a low-rank one (from LoRA) is that the non-zero changes are applied directly to a small set of the original weights, rather than being distributed across all weights via a low-rank structure.