Multi-Step Reasoning

Chain-of-Thought (CoT) is the foundational technique for eliciting step-by-step reasoning by providing the model with examples of an explicit reasoning process before the final answer. It operates on the principle of in-context learning, where the model mimics the demonstrated reasoning structure.

• Few-Shot CoT: Provides 2-8 handcrafted examples within the prompt, each showing a full reasoning trace.
• Zero-Shot CoT: Uses a meta-instruction like 'Let's think step by step' to trigger reasoning without examples.
• Mechanism: The sequential token generation forces the model to maintain a coherent internal state across steps, reducing the likelihood of jumping to an incorrect final answer.

These techniques explicitly break a complex problem into a sequence of simpler, dependent sub-problems. The model solves each in order, using prior outputs as context for subsequent steps.

• Least-to-Most Prompting: The prompt instructs the model to first list sub-questions, then answer them sequentially. This reduces cognitive load per step.
• Plan-and-Solve: Separates the high-level planning phase (creating a solution outline) from the execution phase (solving each outlined step).
• Self-Ask: Guides the model to explicitly formulate searchable sub-questions, which can be answered via external retrieval, before synthesis.

Integrates precise, deterministic tools into the reasoning loop to overcome inherent model weaknesses in calculation, fact retrieval, or code execution.

• Program-Aided Language Models (PAL): The model generates reasoning steps as executable Python code. An external interpreter runs the code to compute the answer, ensuring mathematical precision.
• Tool-Augmented Reasoning: The model's CoT is interleaved with calls to APIs like calculators (Wolfram Alpha), databases, or code executors.
• ReAct Framework: Formally interleaves Reasoning traces (verbalized thoughts) with Actions (tool calls), creating a dynamic loop with environment feedback.

Techniques that move beyond a single, linear reasoning chain to improve robustness and correctness by exploring alternatives or verifying steps.

• Tree-of-Thoughts (ToT): The model generates multiple possible reasoning steps at each juncture, creating a search tree. Algorithms like breadth-first search explore paths, with the model scoring intermediate steps.
• Self-Consistency: Samples multiple, independent CoT paths from the model (using temperature > 0) and selects the final answer by majority voting, improving reliability.
• Chain-of-Verification (CoVe): The model generates a baseline answer, then plans and executes a series of fact-checking queries against its own response, leading to a revised, verified output.

These methods provide a structured framework or pre-generated context to guide the model's reasoning at a higher level of abstraction.

• Chain-of-Abstraction (CoA): The model first creates a high-level reasoning 'blueprint' using placeholders (e.g., [CALCULATE_PROFIT]). A subsequent step fills these placeholders with concrete facts or computations.
• Generated Knowledge Prompting: A two-stage process: 1) The model generates relevant facts about the problem domain. 2) These facts are provided as additional context in a second prompt to produce the final, informed answer.
• Instructional Scaffolding: The prompt includes meta-instructions on problem-solving strategy (e.g., 'First, identify the known variables. Second, recall the relevant formula.') without giving the solution.

Supervised fine-tuning methods that directly teach models to produce coherent, step-by-step logic, moving beyond prompting alone.

• Chain-of-Thought Fine-Tuning: The model is trained on datasets like GSM8K where each example includes a full, human-annotated reasoning chain. This internalizes the pattern of generating intermediate steps.
• Process Supervision: During training, the model receives feedback (rewards or corrections) on each individual step of its reasoning, not just the final answer. This is often implemented using a Process Reward Model (PRM).
• Reasoning Distillation: The complex CoT outputs from a large teacher model (e.g., GPT-4) are used to train a smaller, more efficient student model to replicate the final answer directly or with simplified reasoning.

Chain-of-Thought (CoT) prompting is the foundational technique for eliciting multi-step reasoning. It involves providing a language model with examples or instructions that demonstrate an explicit, step-by-step reasoning process before delivering a final answer. This technique is critical for solving complex arithmetic, commonsense, and symbolic reasoning problems that require intermediate logical deductions.

• Mechanism: By showing the model a worked example (e.g., "Step 1: Calculate X. Step 2: Compare to Y. Step 3: Conclude Z."), it learns to generate similar structured outputs.
• Impact: CoT significantly improves performance on tasks where standard prompting fails, as it reduces the cognitive load of a single-step answer.

Tree-of-Thoughts (ToT) is an advanced generalization of Chain-of-Thought that frames reasoning as a search problem over a tree structure. Instead of a single linear chain, the model explores multiple reasoning paths (branches) in parallel, evaluates intermediate steps, and uses search algorithms like breadth-first or depth-first to find an optimal solution.

• Key Difference: CoT is a greedy path; ToT is a heuristic search over a space of possible reasoning steps.
• Use Case: Ideal for problems with high branching factors, such as strategic game playing, creative writing, or complex planning, where backtracking and exploration are necessary.
• Components: Involves a thought generator, a state evaluator, and a search algorithm.

ReAct (Reasoning and Acting) is a framework that synergizes Chain-of-Thought reasoning with the ability to take actions, typically tool or API calls. The model interleaves verbalized reasoning traces ("I need to find the current weather") with executable actions (search("London weather")), enabling dynamic interaction with external environments.

• Core Loop: Thought → Action → Observation.
• Advantage: Overcomes knowledge cutoffs and computational limitations by grounding reasoning in real-time data and precise tool outputs (e.g., calculators, databases).
• Application: The foundation for many agentic workflows where an AI must use tools to gather information, perform calculations, or manipulate systems to solve a task.

Self-Consistency is a robust decoding strategy designed to improve the reliability of Chain-of-Thought outputs. Instead of generating a single reasoning chain, the model samples multiple, diverse reasoning paths for the same problem. The final answer is determined by majority voting over the conclusions of these independent chains.

• Purpose: Mitigates the greedy decoding problem, where a single plausible but incorrect reasoning path leads to a wrong answer.
• Process: 1. Sample k different CoT trajectories. 2. Extract the final answer from each. 3. Select the most frequent answer.
• Result: Dramatically increases accuracy on mathematical and logical reasoning benchmarks by aggregating over the model's inherent uncertainty.

Program-Aided Language Models (PAL) is a Chain-of-Thought technique where the model's reasoning steps are expressed as executable code (typically Python) within its response. An external interpreter then runs this generated code to compute the final answer, offloading precise computation from the language model.

• Mechanism: Prompt: "Solve: If Alice has 5 apples..." Model Output: # Step 1: Define variables\nalice_apples = 5\n# Step 2: Calculate...\nprint(alice_apples * 2)
• Advantage: Guarantees arithmetic and symbolic precision. The LM handles the algorithmic logic and problem decomposition, while the interpreter handles exact calculation, eliminating common math errors.
• Use Case: Mathematical word problems, symbolic manipulation, and data analysis tasks.

Least-to-Most Prompting is a problem decomposition technique that guides a model to solve a complex problem by first breaking it into a sequence of simpler sub-problems. The solution to each sub-problem is then used to contextualize and solve the next, more difficult step.

• Process: 1. Decomposition Prompt: "What are the sub-questions needed to answer Q?" 2. Sequential Solution Prompt: Using the answer to sub-question 1, solve sub-question 2, and so on.
• Analogy: Similar to curriculum learning for in-context reasoning.
• Benefit: Effectively manages the model's context window and cognitive load, enabling it to solve problems more complex than those seen in its few-shot examples. It's particularly powerful for compositional generalization tasks.