Least-to-Most Prompting

The core mechanism is the explicit decomposition of a complex query into a sequence of simpler sub-tasks. The model is first prompted to generate this step-by-step plan. This reduces cognitive load by preventing the model from attempting to solve the entire problem in one pass, which often leads to errors or hallucinations.

• Example: For the question "If a store sells apples for $2 each and oranges for $1.50 each, and Sarah buys 3 apples and 4 oranges with a $20 bill, how much change does she get?", the decomposition might be:
  1. Calculate the total cost of the apples.
  2. Calculate the total cost of the oranges.
  3. Sum the totals to find the overall cost.
  4. Subtract the overall cost from $20 to find the change.

After decomposition, the model sequentially solves each sub-problem. The solution to each step is fed forward as context for the next, creating a chain of dependent reasoning. This incremental approach ensures that foundational calculations are correct before building upon them, mirroring human problem-solving.

• Key Benefit: Errors in early steps can be contained and identified more easily than in a monolithic, opaque final answer.
• Implementation: This is often implemented via prompt chaining, where the output of one prompt (the answer to sub-problem A) is inserted into the next prompt as part of the context for solving sub-problem B.

Unlike some reasoning methods where intermediate steps are implicit, Least-to-Most Prompting mandates the explicit articulation of each intermediate result. This creates a verifiable audit trail of the model's logic, which is crucial for debugging, validation, and building trust in the system's outputs.

• Contrast with Direct Answering: A standard prompt might output "$5.00". A Least-to-Most approach outputs: Step 1: Apple cost = 3 * $2 = $6. Step 2: Orange cost = 4 * $1.50 = $6. Step 3: Total cost = $6 + $6 = $12. Step 4: Change = $20 - $12 = $8. Final Answer: $8.00

By breaking down a task, each sub-problem operates at a lower complexity ceiling than the original. This allows the model to apply its capabilities more reliably. A model may struggle with multi-operation arithmetic but excel at single multiplication or addition steps. This technique effectively reduces the reasoning depth required in a single generation.

• Applications: Highly effective for mathematical reasoning, logical puzzles, multi-hop question answering, and procedural task planning where the overall process is a composition of simpler, well-defined actions.

Least-to-Most Prompting is a specialized, structured form of Chain-of-Thought (CoT) reasoning. While standard CoT elicits a free-form reasoning trace, Least-to-Most imposes a stricter, hierarchical decomposition. It can be seen as CoT with enforced sub-goal generation. It often combines elements of Few-Shot CoT (by providing decomposition examples) and Self-Ask (by explicitly generating and answering sub-questions).

In practice, this technique is frequently implemented through programmatic prompt chaining. A controller (which could be another LLM call or a deterministic algorithm) manages the sequence:

1. Decomposition Prompt: Break down the following problem into steps: [USER_QUERY]
2. Iterative Solution Prompts: For each step i from the decomposition: Given the context from previous steps [STEP_1...STEP_i-1], solve: [STEP_i]
3. Synthesis Prompt: Based on all the steps [ALL_STEPS], provide the final answer to: [ORIGINAL_QUERY]

This architecture makes the reasoning process modular, debuggable, and suitable for integration with external tools or verification at each step.

A foundational prompting technique that instructs a language model to explicitly generate a step-by-step reasoning trace before producing its final answer. This makes the model's internal 'thought process' visible and significantly improves performance on arithmetic, commonsense, and symbolic reasoning tasks.

• Mechanism: Adds the phrase 'Let's think step by step' or provides few-shot examples with reasoning chains.
• Contrast with Least-to-Most: While CoT elicits a linear reasoning trace, Least-to-Most explicitly decomposes the problem into sub-problems that must be solved sequentially, often with the output of one step feeding into the next.

A prompting technique where the model is instructed to explicitly decompose a question into intermediate, searchable sub-questions. It is designed for integration with external tools, like a search API, to gather facts for each sub-question before synthesis.

• Core Instruction: The model learns to output follow-up questions like 'What is [sub-question]?'.
• Relation to Least-to-Most: Both are decomposition strategies. Self-Ask focuses on question generation for tool-augmented retrieval, whereas Least-to-Most is a more general framework for problem simplification that may or may not involve external tools.

A framework that interleaves language model reasoning with actions (e.g., tool/API calls). The model generates a 'Thought' to reason about the task, an 'Action' to interact with an external environment, and observes the 'Observation' result, looping until completion.

• Key Loop: Thought → Action → Observation → (Repeat).
• Synergy with Least-to-Most: ReAct provides the execution mechanism for solving the sub-problems defined by a Least-to-Most decomposition. The high-level plan from Least-to-Most can be executed via a series of ReAct cycles.

A prompting framework that models reasoning as a heuristic search over a tree structure. Each node represents a partial 'thought' or solution state. The model can explore multiple reasoning paths, backtrack, and evaluate intermediate steps.

• Core Capabilities: Enables exploration, backtracking, and look-ahead planning.
• Contrast with Least-to-Most: ToT is a breadth-first or depth-first search over a space of thoughts. Least-to-Most is a sequential, depth-first decomposition where the problem is broken down into a predetermined chain of simpler steps. ToT is more exploratory; Least-to-Most is more prescriptive.

A method where the language model generates executable code (e.g., Python) as an intermediate reasoning step. The code is then executed in a sandboxed environment to compute the answer, offloading precise calculation from the LLM.

• Use Case: Excellent for mathematical and algorithmic problems requiring precise computation.
• Relation to Least-to-Most: PoT can be the solving technique for an individual sub-problem within a Least-to-Most chain. For example, a complex physics problem might be decomposed (Least-to-Most), and a key algebraic step solved by generating and running code (PoT).

A technique that allocates a dedicated section of the context window for the model to use as a 'scratchpad' for intermediate calculations, reasoning traces, or working memory. This keeps the working process explicit and within the model's immediate context.

• Primary Benefit: Prevents the model from losing track of intermediate values in long reasoning chains.
• Utility for Least-to-Most: Provides a structured workspace for the model to record the inputs, outputs, and states of each sequential sub-problem solved during a Least-to-Most process, ensuring information persists between steps.