Least-to-Most Prompting

The core mechanism involves prompting the model to first identify and list the sequential sub-problems required to solve a complex task. This explicit decomposition phase transforms a single hard problem into a series of easier, manageable steps. The model is then queried to solve each sub-problem in isolation, starting with the simplest. The solution to the first sub-problem is injected into the prompt for the second, creating a contextual scaffold that progressively builds toward the final answer. This contrasts with standard Chain-of-Thought, which attempts to solve the entire problem in a single forward pass.

After solving a sub-problem, its answer is explicitly appended to the prompt for the subsequent sub-problem. This dynamic context window ensures the model has access to verified intermediate results, reducing the cognitive load required to hold multiple pieces of information in working memory simultaneously. This mechanism directly addresses the hallucination snowballing failure mode by grounding later reasoning steps in previously computed, factual outputs rather than relying on the model's internal recall of its own prior generation.

The strategy leverages the principle that solving easier instances of a problem teaches the model the underlying algorithmic pattern required for harder instances. By demonstrating the solution process on simplified versions, the model learns a generalizable problem-solving template. This is particularly effective for tasks like compositional generalization and symbolic manipulation, where the length of the required reasoning chain exceeds the model's typical single-pass capability. The technique was introduced by Zhou et al. (2022) to solve tasks requiring multi-hop reasoning over long contexts.

Because the reasoning process is broken into discrete, auditable steps, Least-to-Most Prompting provides natural error localization. If the final answer is incorrect, an engineer can inspect each intermediate sub-problem solution to identify the exact point of logical failure. This granularity supports process supervision and debugging workflows, allowing for targeted prompt refinement on the specific sub-step that failed rather than rewriting the entire monolithic prompt. This contrasts sharply with end-to-end reasoning methods where the source of an error is opaque.

Least-to-Most Prompting can be combined with tool-augmented reasoning by allowing the model to delegate specific sub-problems to external tools. For example, a sub-problem requiring precise arithmetic can be offloaded to a code interpreter, while a sub-problem requiring factual retrieval can trigger a search API. The results from these tools are then fed back into the sequential context for the next reasoning step. This hybrid approach grounds the reasoning chain in verifiable, external computation, significantly increasing the faithfulness of the final output.

Unlike Tree-of-Thoughts (ToT) , which explores multiple reasoning paths in parallel using a branching structure and a heuristic evaluation function, Least-to-Most Prompting follows a linear, deterministic sequence. ToT is suited for problems requiring global search and backtracking, while Least-to-Most is optimal for tasks with a clear, inherent sequential dependency structure. The linear approach is more token-efficient for decomposable tasks, avoiding the computational overhead of generating and evaluating multiple candidate paths at each step.

A broader class of techniques where a complex problem is explicitly broken into simpler sub-problems before solving. Least-to-most is a specific instantiation of this paradigm.

• Core operation: A decomposition step generates sub-questions from the original query
• Variants: Recursive decomposition, parallel decomposition, and dependency-aware decomposition
• Benefit: Reduces the cognitive load on the model for each individual reasoning step

A reasoning trace that accurately reflects the true causal process by which the model arrived at its answer, free from post-hoc rationalization. Critical for validating that least-to-most sub-problem solutions are genuine rather than confabulated.

• Challenge: Models can generate plausible-sounding but causally inaccurate justifications
• Detection: Causal intervention techniques like activation patching can test faithfulness
• Relevance: Ensures sub-problem solutions in least-to-most are causally connected to the final answer