Prompt Chaining

The fundamental pattern where a complex objective is broken into a directed acyclic graph (DAG) of simpler steps. Each node is a discrete LLM call with a specific instruction, and edges represent data flow.

• Example: A customer service bot chain: 1) Classify intent, 2) Retrieve relevant policy, 3) Draft a response, 4) Check for safety compliance.
• Key Benefit: Enables modular reasoning where each step can be independently optimized, tested, and swapped.

Prompt chains maintain and propagate a state object (often JSON) through the sequence. This state contains the cumulative outputs, intermediate decisions, and execution metadata.

• Mechanism: The output of step N is parsed and injected into the prompt template for step N+1 as a structured variable.
• Critical Engineering: Requires robust output parsing (e.g., using Pydantic) to ensure the state remains clean and machine-readable for the next step.

Chains incorporate decision nodes where an LLM or a rule-based classifier evaluates the current state to determine the next step in the execution path.

• Dynamic Workflows: Enables non-linear, adaptive processes. For example, if an analysis step finds insufficient data, the chain can branch to a retrieval step before proceeding.
• Implementation: Often uses an LLM call tasked with classification or a router prompt that selects from a set of subsequent sub-chain options.

A key characteristic for high-stakes applications is the designed ability to pause execution and solicit human validation, correction, or guidance at specific checkpoints.

• Use Case: In a content moderation chain, a step may flag a post as high-risk and route it to a human reviewer before the final response is generated.
• Architecture: Requires designing interruption points and mechanisms to resume the chain with the human-provided context.

Robust chains are engineered with explicit strategies to manage LLM failures, such as non-compliance, hallucinations, or structured output parsing errors.

• Common Tactics: Retry logic with refined instructions, fallback to a simpler model, or branching to a corrective sub-chain.
• Observability: Each step should emit detailed telemetry (latency, token usage, success/failure flags) to facilitate debugging and cost analysis.

Prompt chaining is often conflated with agents. Key differentiators:

• Predefined vs. Dynamic: Chains have a predefined, static workflow graph. Agents use dynamic planning (e.g., ReAct, Tree-of-Thoughts) to determine the next action at runtime.
• Orchestration Level: Chains are typically orchestrated by an external controller (code). Agents often have an internal loop where the LLM itself decides to 'think', 'act', or 'observe'.
• Use Case: Chains excel at deterministic, multi-step pipelines. Agents excel in open-ended problem-solving where the path is unknown.

The most fundamental pattern where a complex task is broken into a strict, linear sequence of subtasks. The output of step N becomes the primary input for step N+1.

Key Characteristics:

• Deterministic Flow: Execution follows a predefined, non-branching path.
• State Accumulation: Each step adds to or refines a growing context.
• Error Propagation: A failure in one step typically halts the entire chain.

Example: A content generation pipeline: 1. Generate Topic Outline → 2. Draft Section for Each Outline Item → 3. Compile Drafts into Full Article → 4. Generate SEO Meta Description.

A pattern that introduces decision points within a chain, where the output of one LLM call determines which subsequent prompt or sub-chain to execute.

Key Characteristics:

• Dynamic Routing: The execution path is not fixed at design time.
• Classification Step: Often uses a dedicated LLM call to classify the input or intermediate result (e.g., sentiment, intent, topic).
• Modular Sub-Chains: Each branch can be a distinct, optimized sequence.

Example: A customer service router: 1. Analyze User Query → 2. Classify Intent (Sales, Technical, Billing) → 3. Branch to specialized sub-chain for the classified intent.

A pattern for processing multiple independent data units in parallel, then synthesizing the results. The Map step applies the same prompt to many inputs concurrently. The Reduce step consolidates the parallel outputs.

Key Characteristics:

• Parallelizable First Stage: Dramatically reduces latency for batch tasks.
• Synthesis Challenge: The reduce prompt must be designed to handle and integrate multiple, potentially conflicting, responses.
• High Token Usage: Can be expensive due to many concurrent LLM calls.

Example: Summarizing a long document: 1. Split Document into Chapters (Map) → 2. Generate Summary for Each Chapter in Parallel → 3. Combine Chapter Summaries into a Cohesive Whole (Reduce).

A pattern where an initial output is iteratively refined through repeated LLM calls, often using the same or a slightly modified prompt. This is a chain that loops back on itself until a termination condition is met.

Key Characteristics:

• Iterative Improvement: Focuses on incremental enhancement (clarity, conciseness, adherence to rules).
• Termination Condition: Requires a clear stopping rule (e.g., max iterations, self-evaluation score, human-in-the-loop approval).
• Risk of Divergence: Poorly designed loops can cause the output to drift from the original goal.

Example: Code generation: 1. Generate Initial Function → 2. Analyze for Bugs & Edge Cases → 3. Refine Code Based on Analysis → Loop back to Step 2 until no more issues are found.

A hierarchical pattern where a primary router LLM call decomposes a task and dispatches specific subtasks to specialized agent sub-chains, each with its own system prompt and tools.

Key Characteristics:

• Centralized Orchestration: The router acts as a planner and coordinator.
• Agent Modularity: Each agent is a self-contained chain optimized for a specific function (e.g., web search, calculation, database query).
• Result Aggregation: The router often synthesizes the results from multiple agents.

Example: Research assistant: Router: 'Plan research on topic X' → Dispatches to: Web Search Agent, Academic Paper Agent, Data Analysis Agent → Router: 'Compile final report from all agent findings'.

A pattern that explicitly chains a generation step with a subsequent validation step. If validation fails, a correction prompt is triggered, creating a self-correcting workflow.

Key Characteristics:

• Explicit Quality Gates: Validation can check for factual accuracy, format compliance, safety, or code syntax.
• Automated Feedback Loop: The correction prompt receives the validation failure reason as context.
• Fallback Strategies: May include retry limits or escalation to a different model or human.

Example: Generating JSON from text: 1. Generate JSON from User Description → 2. Validate JSON Schema & Syntax → 3. If Invalid: Prompt to Correct JSON Based on Validation Errors.

Chain-of-thought (CoT) prompting is a technique that encourages an LLM to articulate its intermediate reasoning steps before delivering a final answer. It is a core reasoning technique often used within a single step of a prompt chain.

• Mechanism: The prompt explicitly instructs the model to "think step by step" or provides few-shot examples showing a reasoning trace.
• Impact: Dramatically improves performance on complex arithmetic, commonsense, and symbolic reasoning tasks by decomposing the problem.
• Relation to Chaining: While CoT happens within one LLM call, prompt chaining can use CoT in individual links to ensure robust intermediate reasoning.

ReAct prompting is a paradigm that synergistically combines chain-of-thought style reasoning with actionable steps (tool/API calls) within a single LLM interaction. It is a direct architectural precursor to multi-step agentic systems.

• Mechanism: The model output interleaves Thought, Action, and Observation steps, allowing it to reason about what to do and then execute it via a tool.
• Use Case: Ideal for tasks requiring dynamic information gathering, such as question answering with a search API.
• Relation to Chaining: ReAct can be seen as a tightly coupled form of chaining where reasoning and action are in one loop. Prompt chaining is a more general framework that can orchestrate multiple ReAct-style or simpler calls.

Tree-of-thoughts (ToT) prompting is a framework that generalizes chain-of-thought by enabling an LLM to explore multiple concurrent reasoning paths (a tree), using search algorithms to find optimal solutions.

• Mechanism: The LLM is prompted to generate multiple possible next steps for a problem, evaluate them, and then continue exploring the most promising branches, akin to a depth-first or breadth-first search.
• Use Case: Complex planning, strategic game play, or creative brainstorming where a single linear path is insufficient.
• Relation to Chaining: ToT represents a branching exploration within a step. Prompt chaining is typically sequential, but a sophisticated chain could implement a ToT node for complex decision points, making chaining a higher-level orchestrator.

Function calling (often called tool use) is a prompting paradigm where an LLM is instructed to respond with a structured request to execute a predefined function or API call. It is the primary mechanism for LLMs to interact with external systems.

• Mechanism: The prompt (often via a system message) defines available tools with their schemas (name, description, parameters). The LLM outputs a JSON object matching the schema to invoke the tool.
• Use Case: Getting live data (weather, stock prices), performing calculations, updating databases, or controlling external software.
• Relation to Chaining: Function calling is the atomic action unit in many prompt chains. A chain's step often involves an LLM deciding which function to call and with what parameters, based on the context from previous steps.

Self-consistency prompting is an advanced technique that improves upon a single chain-of-thought by sampling multiple, diverse reasoning paths from an LLM and selecting the most consistent final answer through a majority vote or other aggregation.

• Mechanism: Instead of taking one CoT output, the same prompt is run multiple times with a non-zero temperature to generate several reasoning paths and answers. The most frequent final answer is selected.
• Impact: Increases accuracy and robustness, as it mitigates the variability and potential errors in any single reasoning trace.
• Relation to Chaining: Self-consistency can be applied to a critical step within a prompt chain (e.g., a complex deduction). The chain would pass the aggregated, high-confidence answer to the subsequent step, increasing overall reliability.

Meta-prompting is a technique where a large language model is given a higher-order prompt that instructs it to generate, analyze, critique, or refine another prompt. It automates aspects of prompt engineering.

• Mechanism: A meta-prompt might ask an LLM to "write a prompt that accomplishes X" or "improve this prompt for clarity and effectiveness."
• Use Case: Automated prompt optimization, generating task-specific prompts from a description, or creating the individual step prompts for a planned chain.
• Relation to Chaining: Meta-prompting can be used to design the chain itself. A master process could use a meta-prompt to generate the specific instructions for each link in a complex workflow, making prompt chaining a dynamic, self-improving system.