Chain-of-Thought Prompting

The core mechanism relies on providing explicit demonstrations of reasoning in the prompt context.

• Format: Input-output pairs where the output includes a detailed, natural language rationale before the final answer.
• Decomposition: Exemplars teach the model to break complex problems into manageable sub-steps.
• Generalization: A small number of high-quality examples (often 4-8) can trigger step-by-step reasoning on entirely novel problems without any gradient updates.

CoT capabilities can be elicited without any examples by using a simple trigger phrase.

• Mechanism: Appending 'Let's think step by step' to a prompt activates latent reasoning pathways learned during pre-training.
• Efficacy: This zero-shot approach dramatically improves performance on arithmetic, symbolic, and commonsense reasoning benchmarks compared to standard prompting.
• Simplicity: It requires no prompt engineering overhead, making it a universal baseline for logical task execution.

CoT is uniquely effective at tasks requiring precise, multi-hop manipulation of symbols.

• Math Word Problems: The technique achieved state-of-the-art results on the GSM8K benchmark by generating intermediate equations.
• Compositional Generalization: It allows models to solve problems requiring longer reasoning chains than any single example seen during training.
• Error Analysis: The visible intermediate steps allow engineers to pinpoint whether a failure was due to a calculation error or a flawed logical plan.

A critical distinction exists between the generated text and the model's actual computation.

• Plausible vs. Faithful: A CoT rationale may be logically consistent yet not reflect the true causal factors driving the prediction.
• Post-Hoc Rationalization: In black-box models, the reasoning trace is often a generated narrative, not a direct readout of internal weights.
• Audit Utility: Despite potential unfaithfulness, explicit reasoning traces provide a surface for human auditors to detect logical inconsistencies, factual errors, or policy violations.

A complementary inference strategy that enhances CoT by sampling multiple reasoning paths.

• Mechanism: Instead of greedy decoding, the model generates several distinct chains of thought using a non-zero temperature.
• Marginalization: The final answer is selected by taking a majority vote over the conclusions of all generated paths.
• Robustness: This approach significantly reduces errors caused by a single faulty reasoning trajectory, particularly on tasks with a discrete answer space.

A decomposition strategy for problems too complex for a single CoT pass.

• Process: The main problem is first broken down into simpler sub-problems. The model solves each sub-problem sequentially, with the solutions to earlier steps fed into the context for later ones.
• Application: This enables solving compositional generalization tasks requiring longer sequences than standard CoT can handle, such as SCAN and DROP datasets.
• Scaffolding: It effectively uses the model's own output as a dynamic scaffold to manage cognitive load across extreme reasoning depths.