Sparse TIES-Merging

The first step, Trimming, addresses the problem of parameter interference by removing spurious, task-specific changes that may harm performance on other tasks. It calculates the task vector (the delta between a fine-tuned model's weights and the base model's weights) and prunes a percentage (e.g., 20-30%) of parameters with the smallest absolute magnitude in this vector. This is based on the hypothesis that small-magnitude changes are more likely to be noise or overfitting artifacts that do not contribute to the core task knowledge.

The second step, Electing, resolves sign conflicts where different task vectors propose updates to the same parameter in opposite directions. For each parameter, it examines the signs (+/-) of the trimmed task vectors. It then elects a single, dominant sign based on a majority vote or a weighted sum. Parameters where no clear consensus exists (e.g., a 50/50 split) are often reset to zero, preventing contradictory updates from canceling each other out and degrading the merged model's performance.

The final step performs a Sparse Disjoint Merge. Instead of a dense, weighted average of all parameters, it creates a sparse merged mask. This mask identifies, for each parameter, which task vector (if any) provides the update. Typically, the task vector with the largest magnitude (after trimming and sign election) 'wins' that parameter. The result is a final merged model where each parameter is updated by at most one source task, minimizing interference and creating a more parameter-efficient multi-task model.

Sparse TIES-Merging is a direct successor to Task Arithmetic, a simple method that merges models by adding their task vectors. While Task Arithmetic suffers from performance degradation due to interference, TIES introduces the Trimming and Electing stages as corrective filters. The sparse merging stage further refines the approach. This evolution demonstrates a principled engineering response to the limitations of naive model averaging, prioritizing robustness in multi-task fusion.

The core value proposition is extreme parameter efficiency in the merged model. By using sparse, disjoint updates:

• It avoids catastrophic interference where updates from one task overwrite and erase knowledge from another.
• It often outperforms dense merging methods like Model Soups or simple averaging.
• The resulting model has an effective parameter count for any given task that is far lower than the full model size, as only a sparse subset of parameters are active for that task's functionality. This can lead to inference-time efficiencies.

Sparse TIES-Merging is primarily used to build multi-task models from a collection of single-task expert models. This is valuable when:

• Training a joint multi-task model from scratch is computationally prohibitive.
• Tasks arrive sequentially, and you want to merge a new expert into an existing multi-task model (continual learning).
• You want to create a unified model for deployment that retains capabilities across several domains (e.g., translation, summarization, coding) without maintaining multiple separate models.

The foundational technique where only a small, strategically selected subset of a pre-trained model's weights are updated during training. This contrasts with full fine-tuning, which updates all parameters.

• Core Principle: Achieves parameter efficiency by identifying and training only the most task-relevant parameters.
• Methods: Includes parameter masking, gradient masking, and learning sparse masks.
• Benefit: Dramatically reduces memory footprint and computational cost while often preserving performance.

A model composition framework where fine-tuned models are represented as the sum of the pre-trained base model and a task vector. This vector is the element-wise difference between the fine-tuned and base weights.

• Key Insight: Enables operations like Model_MultiTask ≈ Base + Σ(Task_Vector_i).
• Limitation: Naive addition of task vectors leads to interference and performance degradation.
• Relation to TIES: Sparse TIES-Merging directly extends Task Arithmetic by introducing trimming, sign consensus, and disjoint merging to resolve these interference issues.

A core implementation technique for sparse fine-tuning where a binary mask is applied to the model's weights or gradients to selectively enable or disable updates.

• Function: The mask M, with values in {0,1}, is element-wise multiplied with the weight update: ΔW_sparse = M ⊙ ΔW.
• Types: Can be static (pre-defined based on heuristics) or learned (optimized during training).
• Use Case: Directly enables the creation of sparse task vectors, which are essential for techniques like TIES-Merging.

The general process of combining multiple neural networks, trained on different tasks or datasets, into a single unified model without additional training.

• Goal: Achieve multi-task capability or improved generalization.
• Challenges: Includes weight interference (negative transfer) and loss of plasticity.
• Sparse TIES-Merging's Role: It is a specific, advanced merging algorithm designed for merging models created via sparse fine-tuning, using sparsity to minimize interference.

A class of optimization algorithms designed to efficiently handle models where gradients or parameters are predominantly zero. These are essential for training sparse networks.

• Examples: Sparse variants of SGD and Adam that skip computations for zero-valued gradients.
• Mechanism: Optimizers exploit the sparsity pattern to reduce memory movement and FLOPs during the backward pass.
• Application: Critical for making the training of sparsely masked models computationally viable.

A continual learning technique that penalizes changes to parameters deemed important for previous tasks. A sparse variant is relevant for sequential sparse fine-tuning.

• Core Idea: Adds a regularization term Σ_i F_i (θ_i - θ*_i)^2, where F_i is the Fisher information (importance) for parameter i.
• Sparse Adaptation: Sparse EWC approximates the Fisher matrix to be sparse, only protecting a subset of critical weights.
• Connection: Informs how to selectively consolidate knowledge when merging models adapted for sequential tasks.