Model Merging

The simplest merging technique that computes a weighted average of the parameters from two or more fine-tuned models. Each model's weights are multiplied by a scaling factor and summed together.

• Formula: θ_merged = α · θ_modelA + (1 - α) · θ_modelB
• Use case: Blending a general instruction-tuned model with a domain-specific manufacturing model
• Key parameter: The interpolation weight α controls the balance between model behaviors
• Limitation: Assumes linear compatibility between parameter spaces, which may not hold for models fine-tuned on vastly different tasks

An advanced interpolation method that blends model weights along the geodesic path on a hypersphere rather than a straight line. This preserves the magnitude and directional properties of the weight vectors.

• Advantage: Maintains the normalization characteristics of each model's weight space
• Formula: Computes the shortest path between two points on a unit sphere
• Best for: Merging models with similar architectures but divergent fine-tuning trajectories
• Practical result: Often produces more coherent outputs than simple linear interpolation when combining models trained on related but distinct manufacturing tasks

A method that treats fine-tuning as producing a task vector—the element-wise difference between the fine-tuned weights and the original pre-trained weights. These task vectors can be added, subtracted, or scaled independently.

• Task vector: τ = θ_finetuned - θ_pretrained
• Merging formula: θ_merged = θ_pretrained + λ_A · τ_A + λ_B · τ_B
• Key capability: Enables task subtraction to selectively remove unwanted behaviors
• Manufacturing example: Add a defect-detection task vector while subtracting a general conversational vector to create a focused quality inspection model

Trim, Elect Sign, and Merge—a technique that resolves interference between task vectors from different fine-tuned models before combining them. It addresses the problem where parameter updates from different tasks conflict.

• Trim: Zeroes out small-magnitude changes that likely represent noise rather than signal
• Elect Sign: Resolves sign conflicts by selecting the dominant direction of change across models
• Merge: Averages only the agreed-upon parameter updates
• Benefit: Significantly reduces performance degradation when merging multiple specialized manufacturing models simultaneously

A pre-processing step applied to task vectors before merging that randomly drops a large percentage of delta parameters and rescales the remaining ones. This sparsification reduces redundancy and interference.

• Drop rate: Typically 90-99% of delta parameters are randomly pruned
• Rescaling: Remaining parameters are multiplied by 1/(1 - drop_rate) to preserve expected magnitude
• Synergy: Often combined with TIES-Merging for superior results
• Efficiency: Dramatically reduces the effective parameter count being merged, enabling the combination of multiple large industrial models without overwhelming interference

A simple yet effective technique that averages the weights of multiple models fine-tuned with different hyperparameters on the same task. Unlike other merging methods, model soups do not require separate specialized models.

• Uniform soup: Simple equal-weight averaging of all fine-tuned checkpoints
• Greedy soup: Sequentially adds models only if they improve validation performance
• Key insight: Fine-tuned models with different random seeds or learning rates often lie in the same error basin
• Manufacturing application: Average multiple checkpoints from a defect classifier training run to create a more robust model without additional training cost

The upstream process that creates the specialized models destined for merging. PEFT methods like LoRA produce compact weight deltas rather than full model copies, making them the ideal atomic units for model merging operations.

• A LoRA adapter trained on visual defect detection can be arithmetically merged with an instruction-following adapter
• PEFT's small footprint (often <1% of base model parameters) enables rapid experimentation with merge compositions
• Adapter swapping becomes trivial when each skill is encapsulated in its own parameter-efficient module

The theoretical property that makes weight interpolation viable. Two fine-tuned models exhibit linear mode connectivity if the straight-line path between their weights in parameter space contains no significant loss barriers.

• Models fine-tuned from the same pre-trained checkpoint often reside in a single low-loss basin
• This shared basin enables simple averaging operations like Linear Interpolation to produce functional merged models
• Loss landscape analysis tools can verify connectivity before attempting a merge, predicting success probability

A mathematical framework that treats fine-tuning as a directional vector in weight space. A task vector is computed by subtracting the pre-trained weights from the fine-tuned weights, isolating the learned skill.

• Task vectors can be added to inject a skill into a model
• Task vectors can be negated to selectively forget or suppress unwanted behaviors
• TIES-Merging resolves interference between task vectors by trimming redundant parameters and resolving sign conflicts before summation

A pre-processing technique that sparsifies task vectors before merging to reduce interference. DARE randomly drops a high percentage of delta parameters and rescales the survivors to preserve expected magnitude.

• Dropping 90-99% of delta parameters often maintains or improves merge quality
• The extreme sparsity reduces the probability of destructive parameter interference between models
• Originally developed for language models, DARE has proven effective for vision and multimodal industrial models

An automated search paradigm that treats merge configurations as a genome to be optimized. Genetic algorithms evolve populations of merged models by mutating layer-wise mixing ratios and selecting for performance on a multi-objective fitness function.

• Enables discovery of non-intuitive merge recipes that outperform manually designed compositions
• CMA-ES and other evolution strategies can optimize over continuous merge parameters
• Particularly valuable when the target task has no single fine-tuned expert, requiring novel skill recombination

The systematic benchmarking framework required to validate merged models. Without original training data, merged models must be evaluated on held-out task suites to verify that composed skills remain intact and no catastrophic interference has occurred.

• LM Evaluation Harness and similar frameworks provide standardized benchmarks for language capabilities
• Manufacturing-specific evaluation requires curated test sets of defect images, sensor anomalies, and procedural queries
• Multi-objective evaluation tracks skill retention across all source tasks simultaneously, not just aggregate performance