Self-Ask

The model is prompted to break a complex question into smaller, atomic sub-questions. This is a deliberate planning step that forces the system to articulate its reasoning path before execution.

• Example: For "What is the population density of the country where the inventor of the telephone was born?", the model might generate: "1. Who invented the telephone? 2. Where was that person born? 3. What is the population of that country? 4. What is the land area of that country? 5. Calculate density: population / area."
• This decomposition creates a verifiable plan and isolates points where external knowledge (via tools) is required.

After decomposition, the model enters an execution loop, answering each sub-question sequentially. Crucially, for factual sub-questions, it is prompted to use a retrieval tool (e.g., Search(...)).

• The model's output format alternates between reasoning and tool calls: Question: [sub-question]\nFollow up: Search([search query]).
• This loop grounds reasoning in external data, preventing the model from relying solely on its parametric memory, which may be incomplete or outdated.
• The tool's response is then fed back as context for the next step.

Answers from earlier sub-questions become contextual premises for later ones. The model synthesizes information stepwise, building towards the final answer.

• Example: Using the earlier decomposition, the answer to "Alexander Graham Bell" (Q1) becomes a key term for the search query "country of birth of Alexander Graham Bell" (Q2). The answer "Scotland" then informs the searches for population and area data (Q3, Q4).
• This chaining creates a causal reasoning trace where each step's output is directly utilized, making the final answer auditable and the process less prone to hallucination.

After all sub-questions are resolved, the model is prompted to synthesize the gathered facts into a coherent, direct final answer. This step moves from the procedural trace back to a concise response.

• The prompt typically includes an instruction like: Therefore, the final answer is: ...
• This aggregation is distinct from the search loop; it requires the model to perform final computations (like the density calculation) and format the answer based on the original query.
• It ensures the output is user-friendly, not just a log of intermediate steps.

Self-Ask differs from basic Chain-of-Thought (CoT) prompting in its explicit use of external tools and structured output.

• Standard CoT: Reasoning is internal, verbalized, and relies on the model's stored knowledge. Example: "The inventor was Alexander Graham Bell, who was born in Scotland..."
• Self-Ask: Reasoning is operationalized into actionable sub-questions that often trigger tool use. Example: Follow up: Search('Alexander Graham Bell birthplace').
• This makes Self-Ask more factually grounded for knowledge-intensive tasks but adds latency due to tool calls. It is a precursor to more advanced agent frameworks like ReAct.

A Self-Ask prompt has a specific, multi-part structure that guides the model's behavior.

Typical Prompt Components:

1. Instruction: Defines the Self-Ask protocol and available tool (Search).
2. Few-Shot Examples: 1-3 demonstrations of the full loop: question → decomposition → tool use → synthesis.
3. Current Query: The new problem for the model to solve.
4. Structured Output Prefix: The prompt often starts the model's response with Question: to initiate the loop.

Key Design Choice: The few-shot examples are critical for teaching the model the strict output format required to parse tool calls and intermediate answers programmatically.

Chain-of-Thought (CoT) prompting is the foundational technique for eliciting step-by-step reasoning from a language model. It involves providing the model with examples or instructions that demonstrate an explicit reasoning process before delivering a final answer.

• Core Mechanism: The model is conditioned to generate intermediate reasoning tokens that logically connect the problem to the solution.
• Primary Benefit: Significantly improves performance on arithmetic, commonsense, and symbolic reasoning tasks by decomposing complex problems.
• Relation to Self-Ask: Self-Ask is a specialized variant of CoT where the decomposition explicitly produces searchable sub-questions for tool use.

ReAct (Reasoning and Acting) is a framework that interleaves verbalized reasoning traces with actionable steps, such as tool or API calls. It enables language models to perform dynamic reasoning while interacting with external environments.

• Key Pattern: The model output alternates between Thought, Action, and Observation steps.
• Comparison to Self-Ask: While both integrate reasoning with tools, Self-Ask focuses on a strict question-decomposition pattern. ReAct allows for more free-form reasoning and action interleaving within a single step.
• Use Case: Ideal for interactive tasks like question answering with a Wikipedia API, where the agent must decide what to search for based on evolving context.

Least-to-Most Prompting is a technique that decomposes a complex problem into a sequence of simpler sub-problems. It guides a language model to solve each sub-problem in order, using the solution of prior steps to address subsequent ones.

• Decomposition Strategy: Problems are broken down into a linear chain of increasingly difficult steps.
• Contrast with Self-Ask: Both involve decomposition. However, Least-to-Most does not necessarily frame sub-problems as explicit queries for a retrieval tool; it often relies on the model's internal knowledge to solve each step.
• Example: Solving a multi-part word problem by first extracting numerical values, then setting up equations, and finally performing calculations.

Program-Aided Language Models (PAL) is a Chain-of-Thought technique where a language model generates reasoning steps as executable code (e.g., Python) within its response. An external interpreter then runs this code to compute the final answer.

• Core Innovation: Offloads precise computation and logic to a deterministic runtime environment.
• Relation to Self-Ask: Both are tool-augmented reasoning methods. Self-Ask typically uses a retrieval tool (e.g., search), while PAL uses a code interpreter. They can be combined: a Self-Ask agent might use PAL to answer a computational sub-question.
• Benefit: Eliminates arithmetic and symbolic manipulation errors common in pure LLM reasoning.

Tree-of-Thoughts (ToT) is an extension of Chain-of-Thought reasoning where a language model explores multiple reasoning paths in parallel. It evaluates intermediate steps and uses search algorithms (e.g., breadth-first, depth-first) to find an optimal solution.

• Key Difference from Linear CoT: Instead of a single chain, the model explores a branching tree of possible reasoning steps.
• Contrast with Self-Ask: Self-Ask follows a single, deterministic decomposition path. ToT is a search-based methodology for problems where the correct solution path is not obvious and requires backtracking or evaluation of alternatives.
• Application: Complex planning, creative writing, and strategy games where multiple viable approaches exist.

Chain-of-Verification (CoVe) is a method where a language model first generates a baseline answer, then plans and executes a series of verification questions to fact-check its own response, and finally produces a revised, more accurate answer.

• Core Loop: Generate → Plan Verifications → Execute Verifications → Revise.
• Relation to Self-Ask: Both employ self-directed questioning. Self-Ask's questions are for decomposition and information gathering. CoVe's questions are for auditing and validating an existing claim. They represent different phases of a robust reasoning pipeline: gathering facts vs. verifying conclusions.
• Outcome: Reduces hallucinations and improves factual accuracy in open-domain generation.