Structured Output Answer Prompt with JSON Citation Schema

This prompt is for product teams building RAG-powered features where the generated answer must be consumed by a downstream UI component, API, or database—not just displayed as a free-text block. The primary job-to-be-done is producing a machine-readable JSON payload that contains the answer text, a structured array of citations with source metadata, a confidence score, and an explicit abstention flag. The ideal user is an AI engineer or backend developer integrating a model into a customer-facing Q&A system, a support copilot, or a document intelligence product where every claim must be auditable and every response must fit a strict contract. You need this prompt when your application logic depends on parsing citation offsets, rendering source links, or routing low-confidence answers for human review. You do not need this prompt if you are prototyping a simple chat interface, building an internal Slack bot where free-text answers are acceptable, or working in a domain where citation precision is not a product requirement.

Before using this prompt, you must have a retrieval pipeline that supplies a set of candidate passages with stable identifiers (such as doc_id, chunk_index, or a URL). The prompt assumes that the retrieved context is already deduplicated and relevance-ranked; it does not perform its own evidence selection. You must also define the output JSON schema in your application layer so you can validate the model's response before it reaches any user. The prompt includes a [CONSTRAINTS] placeholder where you can inject domain-specific rules—for example, maximum quote length, required citation fields, or abstention triggers for regulated industries. If your use case involves legal, clinical, or financial advice, you must add a human review step after generation and before delivery. This prompt is not a substitute for a safety policy or a compliance review process.

Do not use this prompt when the question is purely conversational, when the user expects a creative or opinion-based response, or when the retrieved context is likely to be empty or irrelevant. In those cases, an abstention or clarification prompt is more appropriate. This prompt is also not designed for multi-turn conversational RAG where answer consistency across turns matters; for that, pair it with a conversation history integration prompt. Finally, if your latency budget is extremely tight and you cannot afford a validation pass on the JSON output, consider a simpler inline citation format that does not require structured parsing. The next section provides the copy-ready prompt template you can adapt to your schema and retrieval pipeline.

This prompt template is designed for product teams that need RAG answers to fit into UI components or downstream API contracts. It produces a predictable JSON payload containing the answer text, an array of citations with source metadata, a confidence score, and an explicit abstention flag. The template uses square-bracket placeholders that you replace with your application's specific inputs, constraints, and output schema requirements before sending the request to the model. Copy the template below and adapt the placeholders to match your retrieval pipeline and output contract.

textCopy
You are an answer generation system that produces only valid JSON output. Your task is to answer a user question using only the provided context passages. You must not fabricate, infer, or use outside knowledge. Every factual claim in your answer must be directly supported by at least one passage.

## INPUT
Question: [QUESTION]
Context Passages:
[CONTEXT_PASSAGES]

## CONSTRAINTS
- Answer only from the provided context. If the context is insufficient, set abstention to true and explain what is missing.
- For each factual claim in the answer, include at least one citation to a specific passage.
- Direct quotes from passages must be wrapped in quotation marks and must not exceed [MAX_QUOTE_LENGTH] words.
- If passages contradict each other, surface the conflict rather than choosing sides.
- Confidence must reflect evidence quality: high (multiple corroborating passages), medium (single passage or partial support), low (weak or ambiguous support).
- Output only the JSON object. No markdown fences, no preamble, no commentary.

## OUTPUT SCHEMA
{
  "answer": "string (the synthesized answer, empty if abstention is true)",
  "abstention": "boolean (true if context is insufficient to answer)",
  "abstention_reason": "string or null (explanation of what is missing if abstention is true)",
  "confidence": "high | medium | low",
  "citations": [
    {
      "passage_id": "string (identifier from the provided context)",
      "quote": "string (the relevant excerpt, truncated if needed)",
      "relevance": "string (brief explanation of how this passage supports the answer)"
    }
  ],
  "conflicts": [
    {
      "description": "string (description of the contradiction)",
      "passage_ids": ["string", "string"]
    }
  ]
}

## EXAMPLES
[FEW_SHOT_EXAMPLES]

## RISK LEVEL
[RISK_LEVEL]

To adapt this template, start by replacing [QUESTION] with the user's query and [CONTEXT_PASSAGES] with your retrieved chunks, each labeled with a unique passage_id. Set [MAX_QUOTE_LENGTH] to a value appropriate for your UI—typically 30 to 50 words prevents verbatim copy-paste while preserving evidence fidelity. The [FEW_SHOT_EXAMPLES] placeholder should contain 2-3 example input-output pairs demonstrating correct abstention, conflict handling, and citation formatting for your domain. Set [RISK_LEVEL] to 'high' if wrong answers carry regulatory or safety consequences; this signals the model to prefer abstention over uncertain answers. For production use, always validate the output against the schema before surfacing it to users, and log any parse failures for prompt refinement.

After copying the template, wire it into your application with a JSON schema validator that checks required fields, enum values for confidence, and citation array completeness. If the model returns malformed JSON, implement a retry with the error message appended to the prompt. For high-risk domains, route abstention=true responses to a human review queue. Do not remove the abstention mechanism to force answers—this increases hallucination risk. Test the prompt with edge cases: empty context, single-passage context, contradictory passages, and questions that cannot be answered from the provided evidence. Measure both abstention accuracy and hallucination rate before shipping.

The structured output prompt is not a standalone artifact—it is a component inside a larger application harness. The harness is responsible for assembling the prompt, parsing the response, validating the JSON schema, and deciding what happens next. For this prompt, the harness must enforce the output contract: a JSON object containing answer_text, citations (array of objects with source_id, quote, and relevance_score), confidence_score (0.0–1.0), and abstention (boolean with abstention_reason when true). Any response that fails schema validation should not reach the user without repair or escalation.

Wire the prompt into your application by first constructing the model request with the prompt template, the retrieved context chunks, and the user's question. Use a model that supports structured output modes natively (such as GPT-4o with response_format set to json_schema or Claude with tool-use parsing) to reduce malformed JSON rates. After receiving the response, run a validation pass: check that citations is a non-empty array when abstention is false, verify that each citation's source_id maps to a provided context chunk, confirm confidence_score is a float between 0 and 1, and ensure that abstention_reason is populated when abstention is true. If validation fails, log the failure with the raw response and the validation error, then retry with the same prompt plus an explicit error message appended to the system instruction describing what was wrong. Limit retries to two attempts before escalating to a human review queue or returning a safe fallback response.

For high-stakes deployments, add an eval harness that runs alongside the validation layer. Sample a percentage of production responses and run a separate LLM judge prompt that checks claim-by-claim grounding against the provided context. Track metrics like schema compliance rate, hallucination flag rate, abstention appropriateness, and retry frequency. When the prompt template changes, run regression tests against a golden dataset of question-context-answer triples with known expected outputs. Avoid deploying this prompt without a monitoring dashboard that surfaces validation failures, retry spikes, and confidence score distributions—these are your leading indicators that the prompt, the retrieval pipeline, or the model behavior has drifted.