Teacher Forcing

During training, the model receives the ground truth token from the previous time step as input, rather than its own potentially incorrect prediction. This conditions the model on the correct sequence history, providing a stable learning signal.

• Mechanism: At time step t, the input is the true token y_{t-1} from the training dataset.
• Impact: This drastically reduces exposure to compounding errors early in training, accelerating initial convergence.

The primary limitation of teacher forcing is exposure bias. The model is trained on a distribution of inputs (ground truth) that differs from the distribution it encounters during inference (its own predictions).

• Training Distribution: Inputs are drawn from the clean, correct data distribution.
• Inference Distribution: Inputs are drawn from the model's own, often imperfect, generative distribution.
• Consequence: This mismatch can lead to error accumulation and degraded performance at inference time, as the model is less robust to its own mistakes.

To mitigate exposure bias, scheduled sampling introduces a curriculum. The probability of using the ground truth token versus the model's prediction is annealed over training.

• Early Epochs: High probability (e.g., 1.0) of using ground truth for stable learning.
• Later Epochs: Probability is gradually reduced, exposing the model to its own predictions.
• Result: The model learns to recover from its own errors, bridging the train-inference gap.

Because the input sequence for the entire training example is known in advance (the ground truth), teacher forcing enables fully parallel computation across time steps during the forward pass. This is a key efficiency advantage.

• Contrast with Free-Running: Methods that use model predictions are inherently sequential.
• Framework Use: This parallelism is leveraged by frameworks like PyTorch and TensorFlow for fast batch processing, making it the default for most sequence-to-sequence training.

Teacher forcing is intrinsically linked to the standard cross-entropy loss objective for next-token prediction. The loss at each time step is calculated independently between the predicted distribution and the single ground truth token.

• Objective: Minimize -log P(y_t | y_1, ..., y_{t-1}, x) for all t.
• Implication: The model is optimized for local, stepwise accuracy. It does not directly optimize for global sequence-level metrics like BLEU or ROUGE, which can require reinforcement learning or beam search tuning.

Teacher forcing trains the model's core next-token prediction capability, which is then utilized by search algorithms during inference to generate high-quality sequences.

• Training: Provides a locally accurate token distribution P(y_t | context).
• Inference: Algorithms like beam search or top-k sampling explore these distributions over multiple time steps to find globally coherent sequences.
• Key Point: The quality of the search is bounded by the accuracy of the locally trained model provided by teacher forcing.

Teacher forcing is essential for training encoder-decoder models like the original Transformer on translation tasks. During training, the decoder receives the ground truth previous word from the target language sequence (e.g., the correct French word) to predict the next word. This prevents error propagation early in training and provides a stable signal for learning complex linguistic mappings.

• Example: Training an English-to-German model. To predict the 5th German word, the decoder uses the first 4 correct German words from the training data as input, not its own potentially incorrect first 4 predictions.

In abstractive summarization, where a model generates a concise summary, teacher forcing provides the decoder with the true summary tokens during training. This is critical because the input article and output summary can have very different lengths and structures. The technique allows the model to learn the compression and rephrasing task without being derailed by its own initial poor summaries.

• Example: A model summarizing a news article. During a training step, to generate the summary token 'profits', the decoder's input is the preceding true summary tokens like 'Company', 'reported', 'rising'.

For sequence-to-sequence speech models, teacher forcing stabilizes alignment learning between acoustic frames and text or phoneme sequences.

• Automatic Speech Recognition (ASR): The decoder receives the true transcript token history to predict the next token, learning to map variable-length audio to text.
• Text-to-Speech (TTS): Systems like Tacotron 2 use teacher forcing during training, where the decoder receives the true previous mel-spectrogram frame to generate the next one, ensuring high-fidelity, stable audio output before switching to autoregressive inference.

When fine-tuning models like Codex or CodeLlama for code completion, teacher forcing is used with source code and target code pairs. The decoder receives the ground truth previous tokens of the target code snippet (e.g., a function body). This teaches precise syntax, API patterns, and logical flow by providing a perfect context, which is especially important for code where a single incorrect token can cause a cascade of syntax errors.

• Example: Training to complete a Python function. To generate return result, the model's input is the correct preceding context def calculate(...):\n result = a + b\n .

Instruction-tuned and chat models are often trained using teacher forcing on multi-turn dialogue datasets. The model learns to generate appropriate responses by being fed the true conversation history and the true previous tokens of the target response. This teaches conversational flow, coherence, and adherence to instructions without the dialogue collapsing due to early poor responses.

• Key Consideration: This can lead to exposure bias, where the model performs well in training (seeing perfect history) but may struggle at inference when it must use its own generated tokens as context, potentially causing drift or repetitive outputs.

To mitigate the exposure bias inherent in pure teacher forcing, advanced techniques blend it with autoregressive training.

• Scheduled Sampling: A training algorithm that randomly decides whether to use the model's own prediction (autoregressive) or the ground truth token (teacher forcing) as the next input. The probability of using the ground truth often decays over training, gradually transitioning the model to its inference-time conditions.
• Curriculum Learning: Training might start with 100% teacher forcing for stability, then slowly introduce the model's own predictions, making the task progressively harder and improving real-world robustness.

The foundational training process where a pre-trained model is adapted using a dataset of labeled input-output pairs. Teacher forcing is the standard algorithm used during SFT for autoregressive models, where the cross-entropy loss is calculated by comparing the model's prediction at each step against the true next token from the training sequence.

The specific loss function minimized during standard supervised fine-tuning. It measures the difference between the model's predicted probability distribution and the true distribution (a one-hot vector for the target token). Teacher forcing provides the necessary sequential context—the ground truth previous tokens—to calculate this loss accurately across the entire output sequence.

A fundamental problem that arises from the teacher forcing paradigm. During training, the model only sees sequences composed of ground truth tokens. At inference, it must generate using its own, potentially erroneous, previous predictions. This discrepancy between the training and inference data distributions can lead to cascading errors, where a single mistake compounds over the generated sequence.

A training technique designed to mitigate exposure bias. Instead of always using the ground truth previous token, it randomly decides to feed the model its own prediction from the previous step during training. This stochastic process gradually exposes the model to its own error distribution, better preparing it for the conditions of autoregressive inference.

• Algorithm: Uses a probability ϵ that decays over training, controlling how often the model's own output is used as input.

A training strategy where examples are presented in a meaningful order of increasing difficulty. This concept can be applied to teacher forcing through techniques like word dropout or graduated scheduled sampling. Initially, the model receives ample ground truth context, but the training curriculum gradually reduces this support, forcing it to learn robust generation from less reliable inputs.

The primary architectural paradigm where teacher forcing is employed. These models, like the original Transformer encoder-decoder, consist of an encoder that processes the input and a decoder that generates the output sequence token-by-token. Teacher forcing is the standard method for training the decoder, providing it with the correct prefix to predict the next token at every step during the learning phase.