Frozen Backbone

A frozen backbone's weights are locked and do not receive gradient updates during fine-tuning. This is enforced by setting requires_grad = False or an equivalent framework-specific flag for all parameters in the base model. The state is typically loaded from a pre-trained checkpoint (e.g., meta-llama/Llama-3.1-8B).

• Primary Benefit: Preserves the model's generalized knowledge and prevents catastrophic forgetting of its original capabilities.
• Core Mechanism: The computational graph for the backbone is truncated during backpropagation, drastically reducing memory footprint.

Freezing the backbone is the principal driver of efficiency in PEFT. It reduces the number of trainable parameters by over 99% in models like GPT-3 or Llama 2, converting a multi-terabyte GPU memory requirement into one manageable on a single consumer-grade GPU.

• Optimizer State: Only the small set of injected parameters (e.g., LoRA matrices, adapters) requires optimizer states (like Adam moments), leading to massive memory savings.
• Gradient Computation: Backpropagation is limited to the new modules, slashing computational cost.

The frozen backbone provides the stable base weight matrix W₀ to which a parameter delta ΔW is applied. All delta tuning methods, including LoRA and (IA)³, rely on this fixed reference point. The adapted weights are computed as W = W₀ + ΔW, where only ΔW is learned.

• Task Vectors: The learned delta can be extracted as a task vector, enabling model editing and task arithmetic (e.g., Vector_{TaskA} + Vector_{TaskB}).
• Additive Parameterization: This is the core mathematical principle enabled by a frozen backbone.

By fixing the core model, adaptation is forced into small, modular components that are injected into the architecture. These modules—such as adapters, prefixes, or scaling vectors—become the sole trainable components.

• Composability: Multiple task-specific modules (e.g., Adapter for sentiment, Adapter for NER) can be attached to the same frozen backbone and switched contextually.
• AdapterFusion: Techniques like AdapterFusion learn to combine knowledge from multiple frozen adapter modules without altering the backbone.

The frozen backbone acts as a fixed feature extractor. Its internal representations—learned from vast corpora during pre-training—remain stable. Injected modules learn to re-purpose these general features for downstream tasks.

• Transfer Learning Guarantee: Mitigates the risk of overfitting to small downstream datasets, as the model cannot drastically deviate from its pre-trained manifold.
• Multi-Task Serving: A single frozen backbone instance can serve numerous downstream tasks by swapping modular components, simplifying production deployment.

In production, the frozen backbone is a static artifact. Only the small delta weights or adapter files need to be versioned, loaded, and managed per task. This decouples base model infrastructure from task-specific logic.

• Storage: Storing hundreds of task-specific adaptations requires megabytes instead of gigabytes per task.
• Hot-Swapping: Task-specific behavior can be changed by loading a new delta checkpoint without restarting the model service, enabling dynamic multi-tenant inference.

Delta tuning is the overarching paradigm for parameter-efficient fine-tuning (PEFT) where adaptation is achieved by learning a small, task-specific parameter change (a delta) to apply to a subset of the pre-trained model's weights. The core model, or frozen backbone, remains unchanged.

• Core Principle: Learn ΔW, not W.
• Methods: Encompasses LoRA, (IA)^3, and Adapter-based approaches.
• Goal: Achieve performance close to full fine-tuning while updating <1% of parameters.

Delta weights (ΔW) are the learned parameter updates that represent the difference between the final adapted weights and the original frozen weights of the pre-trained model. They are the tangible output of delta tuning.

• Mathematical Representation: W_adapted = W_frozen + ΔW.
• Storage: Only ΔW needs to be saved per task, enabling efficient multi-task serving.
• Example: In LoRA, ΔW is factorized into low-rank matrices A and B.

Additive parameterization is the foundational mathematical framework for delta tuning, where the adapted model's weights are represented as the sum of the original frozen weights and a learned delta matrix.

• Formula: W' = W₀ + ΔW.
• Advantage: Explicitly separates foundational knowledge (W₀) from task-specific learning (ΔW).
• Implementation: Directly enables techniques like LoRA and forms the basis for task arithmetic.

A task vector is a mathematical construct, often derived from fine-tuned model weights, that encodes the direction and magnitude of change needed to adapt a base model to a specific task. It is a conceptual and practical instantiation of delta weights.

• Creation: τ = θ_fine-tuned - θ_base.
• Use Case: Enables task arithmetic (e.g., τ_sentiment + τ_formality).
• Property: Task vectors for different tasks often reside in a low-dimensional subspace, explaining the efficacy of low-rank adaptation.

Modular adaptation is a PEFT approach that extends a frozen backbone with small, self-contained, and often composable neural modules that are tuned for specific tasks or skills. Adapters are the canonical example.

• Philosophy: Add specialized "skill blocks" to a generalist base.
• Benefits: Modules are reusable, composable (see AdapterFusion), and easily swapped.
• Deployment: Enables a single base model to serve multiple downstream tasks by activating different module sets.

Trainable components are the small, added modules or selected parameter subsets within a PEFT setup that are updated during training, while the frozen backbone remains static. These are the "delta" in delta tuning.

• Examples: Adapter layers, LoRA matrices, prefix embeddings, (IA)^3 scaling vectors.
• Key Metric: The number of trainable components is typically 0.1% to 5% of the full model size.
• Design Choice: The selection and architecture of these components define specific PEFT methods (e.g., Parallel Adapter vs. Residual Adapter).