Multi-Step Reasoning

Integrates precise, deterministic tools into the reasoning loop to overcome inherent model weaknesses in calculation, fact retrieval, or code execution.

• Program-Aided Language Models (PAL): The model generates reasoning steps as executable Python code. An external interpreter runs the code to compute the answer, ensuring mathematical precision.
• Tool-Augmented Reasoning: The model's CoT is interleaved with calls to APIs like calculators (Wolfram Alpha), databases, or code executors.
• ReAct Framework: Formally interleaves Reasoning traces (verbalized thoughts) with Actions (tool calls), creating a dynamic loop with environment feedback.

Techniques that move beyond a single, linear reasoning chain to improve robustness and correctness by exploring alternatives or verifying steps.

• Tree-of-Thoughts (ToT): The model generates multiple possible reasoning steps at each juncture, creating a search tree. Algorithms like breadth-first search explore paths, with the model scoring intermediate steps.
• Self-Consistency: Samples multiple, independent CoT paths from the model (using temperature > 0) and selects the final answer by majority voting, improving reliability.
• Chain-of-Verification (CoVe): The model generates a baseline answer, then plans and executes a series of fact-checking queries against its own response, leading to a revised, verified output.

These methods provide a structured framework or pre-generated context to guide the model's reasoning at a higher level of abstraction.

• Chain-of-Abstraction (CoA): The model first creates a high-level reasoning 'blueprint' using placeholders (e.g., [CALCULATE_PROFIT]). A subsequent step fills these placeholders with concrete facts or computations.
• Generated Knowledge Prompting: A two-stage process: 1) The model generates relevant facts about the problem domain. 2) These facts are provided as additional context in a second prompt to produce the final, informed answer.
• Instructional Scaffolding: The prompt includes meta-instructions on problem-solving strategy (e.g., 'First, identify the known variables. Second, recall the relevant formula.') without giving the solution.

Supervised fine-tuning methods that directly teach models to produce coherent, step-by-step logic, moving beyond prompting alone.

• Chain-of-Thought Fine-Tuning: The model is trained on datasets like GSM8K where each example includes a full, human-annotated reasoning chain. This internalizes the pattern of generating intermediate steps.
• Process Supervision: During training, the model receives feedback (rewards or corrections) on each individual step of its reasoning, not just the final answer. This is often implemented using a Process Reward Model (PRM).
• Reasoning Distillation: The complex CoT outputs from a large teacher model (e.g., GPT-4) are used to train a smaller, more efficient student model to replicate the final answer directly or with simplified reasoning.

ReAct (Reasoning and Acting) is a framework that synergizes Chain-of-Thought reasoning with the ability to take actions, typically tool or API calls. The model interleaves verbalized reasoning traces ("I need to find the current weather") with executable actions (search("London weather")), enabling dynamic interaction with external environments.

• Core Loop: Thought → Action → Observation.
• Advantage: Overcomes knowledge cutoffs and computational limitations by grounding reasoning in real-time data and precise tool outputs (e.g., calculators, databases).
• Application: The foundation for many agentic workflows where an AI must use tools to gather information, perform calculations, or manipulate systems to solve a task.

Self-Consistency is a robust decoding strategy designed to improve the reliability of Chain-of-Thought outputs. Instead of generating a single reasoning chain, the model samples multiple, diverse reasoning paths for the same problem. The final answer is determined by majority voting over the conclusions of these independent chains.

• Purpose: Mitigates the greedy decoding problem, where a single plausible but incorrect reasoning path leads to a wrong answer.
• Process: 1. Sample k different CoT trajectories. 2. Extract the final answer from each. 3. Select the most frequent answer.
• Result: Dramatically increases accuracy on mathematical and logical reasoning benchmarks by aggregating over the model's inherent uncertainty.

Program-Aided Language Models (PAL) is a Chain-of-Thought technique where the model's reasoning steps are expressed as executable code (typically Python) within its response. An external interpreter then runs this generated code to compute the final answer, offloading precise computation from the language model.

• Mechanism: Prompt: "Solve: If Alice has 5 apples..." Model Output: # Step 1: Define variables\nalice_apples = 5\n# Step 2: Calculate...\nprint(alice_apples * 2)
• Advantage: Guarantees arithmetic and symbolic precision. The LM handles the algorithmic logic and problem decomposition, while the interpreter handles exact calculation, eliminating common math errors.
• Use Case: Mathematical word problems, symbolic manipulation, and data analysis tasks.

Least-to-Most Prompting is a problem decomposition technique that guides a model to solve a complex problem by first breaking it into a sequence of simpler sub-problems. The solution to each sub-problem is then used to contextualize and solve the next, more difficult step.

• Process: 1. Decomposition Prompt: "What are the sub-questions needed to answer Q?" 2. Sequential Solution Prompt: Using the answer to sub-question 1, solve sub-question 2, and so on.
• Analogy: Similar to curriculum learning for in-context reasoning.
• Benefit: Effectively manages the model's context window and cognitive load, enabling it to solve problems more complex than those seen in its few-shot examples. It's particularly powerful for compositional generalization tasks.