Chain-of-Thought Synthesis Prompt with Source References

This prompt is for RAG system builders who need more than a final answer. It produces a complete reasoning chain where every intermediate inferential step is explicitly stated and anchored to a source. Use it when your application requires auditability, when answers depend on multi-hop inference across documents, or when a downstream reviewer—human or automated—needs to trace exactly how the system arrived at a conclusion. The core job-to-be-done is transforming a question and a set of retrieved documents into a conclusion whose logic is transparent, verifiable, and defensible.

This is not a prompt for simple fact lookup or single-hop extraction. It adds significant latency and token cost because the model must generate the reasoning trace before the final answer. Reserve it for high-stakes Q&A in legal, clinical, financial, or compliance settings where the reasoning path is as important as the answer itself. Ideal users are AI architects and engineers who already have a working retrieval pipeline and now need to add a layer of explainability for regulators, auditors, clinical reviewers, or internal QA teams. Required context includes the user's question, a set of retrieved passages with source identifiers, and a defined output schema that separates reasoning steps from the final synthesis.

Do not use this prompt when latency budgets are tight, when the answer is obvious from a single passage, or when the end user only needs a quick factual response. It is also a poor fit for open-ended creative tasks or scenarios where the retrieved context is noisy and the reasoning chain would amplify irrelevant details. Before adopting this prompt, confirm that your evaluation pipeline can assess reasoning validity—not just answer correctness—and that you have a plan for logging and reviewing the generated traces. If you only need a grounded answer without the intermediate steps, start with a simpler evidence-only generation prompt and add chain-of-thought only when auditability becomes a hard requirement.

This template is designed to be pasted directly into your RAG pipeline's system prompt or as a user message wrapper. It forces the model to externalize its reasoning in discrete steps, each anchored to a specific piece of retrieved evidence. The primary value is auditability: every logical jump from evidence to conclusion is traceable, making it possible to debug why an answer was generated and pinpoint exactly where a hallucination or faulty inference occurred. This is essential for regulated domains, customer-facing Q&A with high accuracy requirements, and any system where a wrong answer carries significant cost.

textCopy
You are an expert reasoning engine. Your task is to answer a user's question by synthesizing information from the provided context. You must show your work step-by-step, citing the source for every factual claim and inferential hop.

# REASONING RULES
1.  **Decompose the Question:** Break down the user's question into a sequence of simpler sub-questions that must be answered to reach a final conclusion.
2.  **Sequential Reasoning:** For each sub-question, perform a reasoning step. In each step, you must:
    *   State the sub-question you are trying to answer.
    *   Identify the relevant evidence from the provided context.
    *   Cite the source of that evidence using its `[SOURCE_ID]`.
    *   If quoting directly, wrap the text in quotation marks.
    *   State the intermediate conclusion derived from that evidence.
3.  **Handle Missing Evidence:** If a sub-question cannot be answered from the context, explicitly state that the evidence is missing and stop the chain. Do not guess.
4.  **Synthesize Final Answer:** After all sub-questions are addressed, synthesize the intermediate conclusions into a single, coherent final answer. The final answer must not contain any new information not present in the reasoning chain.

# INPUTS
**User Question:** [USER_QUESTION]

**Retrieved Context:**
[RETRIEVED_CONTEXT]

# OUTPUT FORMAT
Your response must be a JSON object with the following schema:
{
  "reasoning_chain": [
    {
      "step_number": 1,
      "sub_question": "string",
      "evidence_used": "string",
      "source_id": "string",
      "intermediate_conclusion": "string"
    }
  ],
  "final_answer": "string",
  "abstention_flag": boolean
}

To adapt this template, replace [USER_QUESTION] with the user's query and [RETRIEVED_CONTEXT] with your pre-fetched document chunks. Crucially, your retrieval pipeline must attach a unique [SOURCE_ID] to each chunk before injecting it into the prompt. The [OUTPUT_SCHEMA] is enforced here as a JSON object, which is critical for programmatic validation. Before deploying, run this prompt against a golden dataset of questions and manually inspect the reasoning_chain for hallucinated source_id values or intermediate conclusions that cannot be logically derived from the cited evidence_used. A common failure mode is the model correctly citing a source but drawing an incorrect inference from it; your eval must check for this logical leap, not just citation presence.

This prompt is designed to be the final synthesis step in a RAG pipeline, sitting between retrieval and the user-facing response. It expects a structured [CONTEXT] object containing retrieved passages with source metadata and a [QUERY] from the user. The harness must assemble this context, invoke the model, validate the reasoning trace, and decide whether to return the answer, retry, or escalate. Because the output includes explicit reasoning steps with citations, the harness has a unique opportunity to perform automated verification on each inferential hop before the answer reaches a user.

Wire the prompt as a single-turn generation call with response_format set to { "type": "json_object" } to enforce the structured output schema. The application layer must construct the [CONTEXT] array by mapping each retrieved chunk to an object with source_id, text, and optional metadata (date, author, authority level). Before calling the model, validate that the context is not empty and that each source has a unique identifier. After receiving the response, run a reasoning validity check: iterate through the reasoning_steps array and confirm that every citation references a source_id present in the original context. Flag any step where the claim cannot be verified against the cited text using a lightweight NLI model or string-match heuristic. If hallucinated citations or unsupported claims are detected, log the failure and either retry with a stricter [CONSTRAINTS] block (e.g., 'You must only cite sources that directly support your claim. If no source supports a step, mark it as UNSUPPORTED.') or escalate to a human review queue with the full trace attached.

For high-stakes deployments, implement a dual-model verification harness. After the primary model generates the reasoning trace, send the same [QUERY] and [CONTEXT] to a second, smaller model with a verification prompt: 'For each reasoning step, confirm whether the cited source actually supports the claim. Return a JSON array of pass/fail flags.' If the verifier flags any step, route the entire response for human review. Log every response—including the full reasoning trace, citations, verifier output, and final disposition—to an audit store with the prompt version, model ID, and timestamp. This traceability is essential for debugging hallucination patterns and for compliance in regulated domains. Avoid using this prompt for real-time chat without a latency budget; the chain-of-thought generation adds 2-5 seconds of model time. For latency-sensitive applications, consider caching reasoning traces for common query patterns or using a faster model for the synthesis step while reserving the full chain-of-thought for audit sampling.