These are the foundational prompting paradigms for eliciting reasoning.

• Zero-Shot CoT: Appends a universal trigger phrase like 'Let's think step by step' to a prompt, instructing the model to generate reasoning without any task-specific examples.
• Few-Shot CoT: Provides the model with 2-8 exemplar problems, each demonstrating a complete reasoning chain, before presenting the target problem. This teaches the model the expected reasoning format for that specific task type.

A decoding strategy that enhances CoT's reliability by sampling multiple, diverse reasoning paths from the model for a single problem. Instead of taking the first output, it employs majority voting on the final answers from all sampled chains. This mitigates individual reasoning errors and leverages the model's latent knowledge across different reasoning trajectories, significantly improving accuracy on complex arithmetic and commonsense reasoning tasks.

A technique where the model's reasoning chain is expressed as executable code (typically Python). The model writes code that defines variables, performs calculations, and uses logic to solve the problem. This code is then executed by an external interpreter to produce the final answer.

Key Benefit: Offloads precise mathematical and symbolic computation to a deterministic runtime, eliminating the model's tendency for calculation errors while retaining its ability to set up the problem logically.

A framework that interleaves reasoning traces with actionable steps. The model generates a 'Thought' (a reasoning step about what to do next), an 'Action' (e.g., a search query, API call, or tool use), observes the 'Observation' (the tool's result), and then repeats. This creates a dynamic loop where reasoning is grounded in real-time information from external systems, enabling agents to solve problems that require up-to-date knowledge or precise tool-based computation.

A generalization of CoT that frames reasoning as a heuristic search problem over a tree of intermediate states.

• At each step, the model generates multiple potential 'thoughts' or reasoning continuations.
• A separate evaluation step (often using the same LM) scores the promise of each thought.
• A search algorithm (e.g., breadth-first, depth-first, or beam search) explores the tree to find the most promising path to a solution.

This allows for deliberate planning, backtracking from dead ends, and consideration of multiple solution strategies in parallel.

A problem decomposition technique that reduces complex questions into a sequence of simpler sub-problems. The model is first prompted to decompose the original problem. Then, it is prompted sequentially to solve each sub-problem, where the context for each step includes the solutions to previous sub-problems. This technique effectively reduces the cognitive load at each step and is particularly powerful for problems involving compositional generalization, where the model must solve novel combinations of known skills.

Tree-of-Thoughts (ToT) is a generalization of Chain-of-Thought that enables language models to explore multiple reasoning paths in parallel. Instead of a single linear chain, the model generates a branching tree of intermediate steps ("thoughts"). A search algorithm (e.g., breadth-first, depth-first) is then used to evaluate and select the most promising paths toward a solution.

• Key Mechanism: Explores a search space of reasoning steps, allowing for backtracking and consideration of alternatives.
• Use Case: Ideal for complex problems with high uncertainty, where the first reasoning path may be suboptimal, such as strategic game playing, creative writing, or complex planning.
• Contrast with CoT: While CoT produces a single, linear reasoning trace, ToT manages a heuristic search over a reasoning graph, making it more computationally intensive but potentially more robust.

ReAct (Reasoning and Acting) is a framework that synergizes Chain-of-Thought reasoning with the ability to take actions, typically tool or API calls. The model interleaves verbal reasoning traces with executable actions, creating a loop of thought, action, and observation.

• Key Mechanism: Prompts follow a pattern: Thought: [Reason about current state and next step], Action: [Tool call with parameters], Observation: [Tool result].
• Use Case: Essential for agentic systems that must interact with external environments, such as answering questions by searching a database, performing calculations, or controlling software.
• Core Benefit: Grounds the model's reasoning in real-world data and state, preventing hallucination and enabling dynamic problem-solving. It is a foundational pattern for Tool-Augmented Reasoning.

Self-Consistency is a decoding strategy used to improve the reliability of Chain-of-Thought outputs. Instead of generating one reasoning chain, the model samples multiple, diverse reasoning paths for the same problem. The final answer is determined by majority voting over the conclusions of these paths.

• Key Mechanism: Leverages the idea that correct reasoning is more likely to arrive at the same final answer via different logical routes, while incorrect reasoning leads to scattered answers.
• Use Case: Applied to complex mathematical, commonsense, or symbolic reasoning tasks where a single CoT output may be error-prone.
• Performance Impact: Often yields significant accuracy gains over standard greedy decoding (using a single chain) but requires multiple model inferences, increasing latency and cost.

Program-Aided Language Models (PAL) is a Chain-of-Thought variant where the model's intermediate reasoning steps are expressed as executable code (e.g., Python). The language model generates a code snippet that outlines the solution logic, and an external code interpreter executes it to produce the final answer.

• Key Mechanism: Offloads precise computation and algorithmic logic to a deterministic runtime environment, circumventing the language model's weaknesses in arithmetic and symbolic manipulation.
• Use Case: Highly effective for mathematical word problems, data analysis tasks, and symbolic reasoning where exact computation is required.
• Example: For a problem about calculating a total cost, the CoT might be: `# Calculate subtotal subtotal = 10 * 5.99

Apply tax

total = subtotal * 1.08 print(total)` The interpreter runs this code to get the answer.

Chain-of-Verification (CoVe) is a meta-reasoning technique designed to improve factual accuracy. The model first generates a baseline answer. It then plans a series of verification questions to fact-check its own response, executes those verifications (often via retrieval), and finally produces a revised, more accurate answer.

• Key Mechanism: Introduces an explicit self-audit loop. It separates the generation of claims from the process of verifying them.
• Use Case: Critical for applications requiring high factual precision, such as technical documentation, summarization of news, or enterprise knowledge Q&A.
• Relation to Faithfulness Metrics: CoVe is an operational method for improving faithfulness, as it forces the model to align its final answer with verifiable evidence gathered during the verification chain.

Process Supervision is a training paradigm where a model receives feedback or rewards for each individual step in a reasoning chain, rather than solely for the final output. This is often implemented using Process Reward Models (PRMs) trained to score the correctness of each intermediate step.

• Key Mechanism: Provides granular, step-by-step learning signals, encouraging the model to develop valid internal reasoning patterns. Contrasts with outcome supervision, which only rewards the final answer.
• Use Case: Training more reliable and transparent reasoning models, particularly for domains like mathematics, logic, and science, where the process is as important as the result.
• Research Impact: Shown to produce models whose reasoning is more faithful (the steps genuinely lead to the answer) and less prone to hallucination in intermediate logic.