Hyperparameter Tuning

The learning rate is a scalar multiplier that controls the size of the step taken during gradient descent optimization. It is arguably the most critical hyperparameter.

• Too high: The model may overshoot minima, diverge, or oscillate, failing to converge.
• Too low: Training becomes prohibitively slow and may get stuck in poor local minima.
• Common Values: Typically ranges from 1e-5 to 1e-1. Adaptive optimizers like Adam have separate rates for weights and biases.
• In RAG Fine-Tuning: A lower learning rate (e.g., 1e-5) is standard for fine-tuning pre-trained encoders to avoid catastrophic forgetting of general knowledge.

Batch size determines the number of training examples processed before the model's internal parameters are updated.

• Small Batch (Stochastic): Provides a regularizing, noisy gradient signal that can help escape sharp minima and generalize better, but is computationally inefficient per epoch.
• Large Batch: Leads to more stable, precise gradient estimates and faster training per epoch but may converge to sharp minima that generalize poorly.
• Memory Constraint: The primary practical limit is GPU/TPU memory. Techniques like gradient accumulation simulate larger batches.
• Rule of Thumb: Powers of two (32, 64, 128, 256) are used for hardware optimization.

An epoch is one complete pass through the entire training dataset. The number of epochs controls training duration and is a key guard against overfitting.

• Too Few: Results in underfitting; the model fails to learn the underlying patterns in the data.
• Too Many: Leads to overfitting; the model memorizes the training data, including noise, and performs poorly on unseen data.
• Early Stopping: A universal best practice. Training is halted when performance on a validation set stops improving, automatically determining the effective number of epochs.
• Fine-Tuning Context: For adapting pre-trained models, only a few epochs (1-10) are typically required.

Weight decay is a regularization technique that adds a penalty proportional to the squared magnitude of the model's weights to the loss function.

• Purpose: It discourages the model from relying too heavily on any single feature or developing excessively large weights, which improves generalization.
• Mechanism: The loss function becomes: Loss = Original_Loss + λ * Σ(weights²). The hyperparameter λ controls the strength of the penalty.
• Distinction: In adaptive optimizers like Adam, weight decay is decoupled from the gradient-based update (AdamW) for correct implementation.
• Typical Values: Often set between 0.01 and 0.1 for large models, or as low as 0.0 for some fine-tuning tasks.

A learning rate scheduler defines how the learning rate changes over time. Warmup is a specific initial phase of this schedule.

• Warmup: Gradually increases the learning rate from a very small value (e.g., 0) to the target maximum over a set number of steps or epochs. This stabilizes training early on when gradients are volatile.
• Common Schedules:
  • Linear Decay: Decreases the rate linearly to zero.
  • Cosine Annealing: Decreases the rate following a cosine curve, often with restarts.
  • ReduceLROnPlateau: Dynamically reduces the rate when validation loss plateaus.
• Impact: The right schedule can significantly improve final model accuracy and convergence speed.

Specific to fine-tuning retrievers in RAG systems, these hyperparameters govern the contrastive loss objective used to train dense embedding models.

• Temperature (τ): A scaling factor in the softmax of the contrastive loss (e.g., InfoNCE). It controls how "hard" or "soft" the model distinguishes between positive and negative samples. Lower temperature makes the probability distribution sharper.
• Margin (in Triplet Loss): The enforced minimum distance between the anchor-positive and anchor-negative pairs in the embedding space.
• In-Batch Negatives: The batch size itself becomes a hyperparameter, as all other examples in the batch are typically used as negatives. Larger batches provide more, and often harder, negative samples.
• Hard Negative Mining Ratio: The proportion of challenging negatives to include in the batch, which is crucial for teaching the model fine-grained discrimination.

A family of techniques that adapt large pre-trained models by training only a small subset of parameters. This is critical for tuning retrievers without the prohibitive cost of full retraining.

• Core Methods: Includes LoRA (Low-Rank Adaptation) and Adapter modules.
• Key Benefit: Reduces memory footprint and computational cost by orders of magnitude, enabling fine-tuning on consumer-grade hardware.
• Use Case: Adapting a general-purpose embedding model like BERT to a specialized enterprise vocabulary for improved semantic search.

A training paradigm that teaches a model to distinguish between similar (positive) and dissimilar (negative) data pairs by manipulating their embeddings in a shared latent space.

• Objective: Pull embeddings of relevant query-document pairs closer together while pushing irrelevant pairs apart.
• Foundation for Retrievers: Forms the basis for training dual-encoder architectures used in dense retrieval.
• Key Loss Functions: Implemented via Triplet Loss or InfoNCE (Noise-Contrastive Estimation) Loss.

A critical training strategy that involves identifying and using challenging, semantically similar but irrelevant documents as negative examples.

• Purpose: Improves a retriever's discrimination ability by forcing it to learn fine-grained differences.
• Process: Often involves using a first-pass retriever or a cross-encoder to find documents that are topically related but not correct answers.
• Impact: Dramatically improves retrieval precision compared to using only random or in-batch negatives.

An algorithm that dynamically adjusts the learning rate during training according to a predefined policy, which is a hyperparameter itself.

• Function: Manages the speed of model weight updates to improve convergence and final performance.
• Common Schedules: Linear decay, step decay, and cosine annealing (often with warm restarts).
• Hyperparameter Tuning Aspect: The scheduler type, warm-up steps, and decay rate are all tuned to optimize training stability and model accuracy.

A fundamental information retrieval metric used to evaluate and tune retriever performance.

• Definition: Measures the proportion of all relevant documents for a query that are successfully retrieved within the top K results.
• Tuning Target: A primary optimization goal during retriever fine-tuning is to maximize Recall@K (e.g., Recall@5, Recall@100) on a validation set.
• Trade-off: Must be balanced with precision; optimizing purely for recall can lead to noisy context for the generator.

An advanced optimization paradigm where the retriever and generator components of a RAG system are jointly trained.

• Mechanism: Allows gradients from the generator's language modeling loss to flow back through the retrieval mechanism, directly optimizing the retriever for the final answer quality.
• Technical Challenge: Requires solving backpropagation through retrieval, often using approximations like the straight-through estimator.
• Outcome: Aligns the retriever's notion of relevance directly with what helps the generator produce the best answer.