Few-Shot Chain-of-Thought

FS-CoT relies on in-context learning, where the model infers the required reasoning pattern from a few provided examples (exemplars). Each exemplar contains:

• A query (the problem statement).
• A reasoning chain (the step-by-step logic).
• A final answer. The model is not explicitly instructed to 'think step by step'; it learns this behavior by pattern-matching the structure of the exemplars. This is distinct from Zero-Shot CoT, which uses a direct instruction like 'Let's think step by step' without examples.

The technique is designed for task generalization, not domain knowledge transfer. The exemplars teach the model a specific reasoning template (e.g., multi-step arithmetic, logical deduction, commonsense inference) applicable to novel problems of the same type. Key aspects include:

• Format Consistency: The exemplars must demonstrate a consistent reasoning format for the model to replicate.
• Complexity Matching: Exemplars should be of comparable complexity to the target task to be effective.
• The 'Few-Shot' Limit: Typically 2-8 examples are used, balancing the cost of prompt length with the clarity of the demonstrated pattern.

A hallmark of FS-CoT exemplars is the explicit decomposition of a problem into manageable sub-steps. This often involves:

• Introducing intermediate variables to hold partial results.
• Articulating implicit assumptions that bridge logical gaps.
• Performing sequential operations where the output of one step is the input to the next. For example, in a math word problem, an exemplar might first extract relevant numbers, then state the necessary formula, then perform the calculation step-by-step, and finally interpret the result. This teaches the model to avoid shortcut reasoning and build a verifiable solution path.

A primary benefit of FS-CoT is its ability to significantly reduce compositional generalization errors. Standard prompting often causes models to fail on problems that require combining known skills in novel ways. FS-CoT mitigates this by:

• Making dependencies explicit: The chain shows how sub-problems relate.
• Providing a worked blueprint: The model follows the exemplar's problem-solving strategy.
• Enabling verification: Each step can be checked for correctness, making the overall process more robust than a single, end-to-end answer generation. This is particularly powerful for tasks requiring symbolic manipulation or multi-factorial decision-making.

The performance of FS-CoT is highly sensitive to the quality and selection of exemplars. This introduces specific engineering considerations:

• Exemplar Selection: Choosing 'informative' examples that clearly illustrate the reasoning hurdle is critical. Random selection often underperforms.
• Ordering Effects: The sequence of examples can influence the model's learned pattern.
• Verbalizer Consistency: The phrasing used for reasoning (e.g., 'Therefore,', 'So,', 'Step 1:') should be consistent across exemplars.
• Answer Trigger: The transition from the reasoning chain to the final answer (e.g., 'The final answer is') must be clear. Poorly constructed exemplars can lead to the model generating reasoning but failing to output a final answer, or generating incorrect reasoning formats.

FS-CoT is a foundational method within a broader ecosystem of reasoning techniques:

• Vs. Zero-Shot CoT: FS-CoT provides concrete patterns; Zero-Shot relies on the model's internalized reasoning ability triggered by an instruction.
• Precursor to Fine-Tuning: FS-CoT demonstrations are often used to create datasets for Chain-of-Thought Fine-Tuning, which bakes the reasoning ability into the model's weights.
• Component in Larger Frameworks: FS-CoT is frequently embedded within agent frameworks like ReAct (Reasoning and Acting), where the reasoning chain interleaves with tool calls.
• Complement to Self-Consistency: FS-CoT can be combined with Self-Consistency by sampling multiple reasoning paths from the model and using majority voting on the final answers for increased reliability.

The foundational technique for eliciting explicit, step-by-step reasoning from a language model. Unlike the few-shot variant, standard CoT can be applied in a zero-shot manner by simply instructing the model to "think step by step." It works by encouraging the model to generate intermediate reasoning traces before producing a final answer, making its logic transparent. This method is particularly effective for arithmetic, commonsense, and symbolic reasoning tasks where the final answer is not immediately derivable.

A prompting technique that elicits step-by-step reasoning without any task-specific examples. It typically works by appending a simple trigger phrase like 'Let's think step by step' to the end of a query. The model then generates its own reasoning chain autonomously. This method is highly flexible and requires no example curation, but its reliability can be more variable than few-shot approaches, as the model must infer the correct reasoning format entirely from the instruction.

A decoding strategy that enhances the reliability of Chain-of-Thought outputs. Instead of generating a single reasoning path, the model samples multiple, diverse chains of thought for the same problem. The final answer is determined by majority voting across all sampled outputs. This technique mitigates the variability and potential errors in any single reasoning path, often leading to significant improvements in accuracy on complex reasoning tasks like GSM8K or MATH.

A framework that interleaves reasoning traces with actionable steps. The model generates a verbal reasoning thought (e.g., "I need to search for the current CEO of Company X") followed by an actionable step, such as a tool call to a search API. The result of that action is then observed and incorporated into the next reasoning step. This tight loop allows models to perform dynamic reasoning grounded in real-time, external information, making it essential for agentic systems that interact with the world.

A Chain-of-Thought variant where the model's intermediate reasoning steps are expressed as executable code (typically Python). The model writes a program that, when run by an external interpreter, computes the final answer. For example, for a math word problem, the model might generate Python code to parse the problem, define variables, and perform calculations. This offloads precise computation to a reliable interpreter, reducing arithmetic and logical errors inherent in pure text-based reasoning.

An advanced generalization of Chain-of-Thought that explores multiple reasoning paths in parallel. At each step, the model generates several possible subsequent thoughts, creating a tree-like structure of potential solutions. Search algorithms like breadth-first or depth-first search are then used to navigate this tree, often with a heuristic to evaluate intermediate steps. This is particularly powerful for problems with high branching factors, such as strategic game playing, creative writing, or complex planning, where a single linear path may be insufficient.