Answer Support Level Classification Prompt Template

Use this prompt when you have a generated answer and the retrieved context that was used to produce it, and you need a structured, claim-level audit of how well the evidence supports each statement. The ideal user is an AI engineer or QA pipeline builder who already has a working RAG system and needs automated grounding verification before answers reach users or downstream systems. This prompt is not for initial answer generation, retrieval quality assessment, or deciding whether to answer at all—it assumes an answer already exists and focuses exclusively on mapping claims back to source evidence.

The prompt works best in pipelines where factual accuracy is non-negotiable: clinical decision support, legal research, compliance documentation, financial analysis, and technical support where wrong answers carry real cost. It requires the retrieved context to be passed alongside the answer, and it expects the context to be the same evidence the model used during generation. If your pipeline modifies or summarizes context before generation, pass the original retrieved chunks, not the compressed version. The classification tiers—fully supported, partially supported, inferred, and unsupported—give downstream systems enough granularity to decide whether to publish, flag for review, or block the answer entirely.

Do not use this prompt when you need a single overall confidence score rather than per-claim grounding. For that, use the Confidence Threshold Evaluation or Answer Certainty Calibration prompts. Do not use it when the primary goal is to decide whether to answer at all—that's the Answer Abstention Decision or Pre-Answer Sufficiency Check prompts. Avoid this prompt in low-latency user-facing chat where per-claim classification adds unacceptable delay; reserve it for asynchronous verification, batch evaluation, or guardrail pipelines that run after generation but before delivery. Always pair this prompt with human review for high-risk domains, and run periodic spot checks comparing its classifications against manual annotation to catch drift in grounding judgments.

This template is the core of the Answer Support Level Classification workflow. It takes a generated answer and the retrieved context that was used to produce it, then decomposes the answer into individual claims and classifies each one. The output is a structured audit that downstream systems can use for verification, logging, or triggering human review. The template uses square-bracket placeholders for all variable inputs so you can drop it directly into your prompt assembly pipeline.

textCopy
You are an evidence auditor. Your job is to compare a generated answer against the provided retrieved context and classify every factual claim in the answer.

## INPUT

**Generated Answer:**
[ANSWER]

**Retrieved Context (numbered passages):**
[CONTEXT]

## INSTRUCTIONS

1. Decompose the generated answer into discrete factual claims. A claim is any statement that asserts something true about the world, including entities, events, dates, quantities, relationships, or properties.
2. For each claim, search the retrieved context for evidence that supports or contradicts it.
3. Classify each claim into exactly one of these categories:
   - **fully_supported**: The context contains explicit, unambiguous evidence for the claim. Quote the supporting passage.
   - **partially_supported**: The context supports some but not all aspects of the claim, or supports it with qualifications. Explain what is and isn't covered.
   - **inferred**: The context does not directly state the claim, but a reasonable reader would infer it from the available evidence. Explain the inference chain.
   - **unsupported**: No evidence in the context supports the claim, or the context contradicts it. If contradicted, mark as 'contradicted' and quote the conflicting passage.
4. If the answer contains no factual claims (e.g., it is purely procedural, a refusal, or a clarification question), return an empty claims array.

## CONSTRAINTS

- Do not classify claims based on your own knowledge. Only use the provided retrieved context.
- If a claim is partially supported, you must specify which part is unsupported.
- If multiple context passages are relevant, cite all of them.
- If the context is empty or irrelevant, classify all claims as 'unsupported'.
- Do not modify or rephrase the claims. Quote them verbatim from the answer.

## OUTPUT SCHEMA

Return a JSON object with this exact structure:

{
  "answer_id": "string or null",
  "claims": [
    {
      "claim_text": "verbatim claim from answer",
      "classification": "fully_supported | partially_supported | inferred | unsupported",
      "supporting_context_ids": ["passage-1", "passage-3"],
      "supporting_quotes": ["relevant quote from context"],
      "explanation": "why this classification was assigned",
      "contradicted": false,
      "contradicting_quote": null
    }
  ],
  "summary": {
    "total_claims": 0,
    "fully_supported": 0,
    "partially_supported": 0,
    "inferred": 0,
    "unsupported": 0,
    "contradicted": 0
  }
}

## EXAMPLES

**Example 1: Fully Supported Claim**

Context: "Acme Corp was founded in 2012 by Jane Smith. The company is headquartered in Austin, Texas."
Answer: "Acme Corp was founded in 2012."

Output claim:
{
  "claim_text": "Acme Corp was founded in 2012.",
  "classification": "fully_supported",
  "supporting_context_ids": ["passage-1"],
  "supporting_quotes": ["Acme Corp was founded in 2012 by Jane Smith."],
  "explanation": "The context explicitly states the founding year.",
  "contradicted": false,
  "contradicting_quote": null
}

**Example 2: Inferred Claim**

Context: "The product launch event is scheduled for March 15. All senior executives will attend."
Answer: "The CEO will be at the product launch."

Output claim:
{
  "claim_text": "The CEO will be at the product launch.",
  "classification": "inferred",
  "supporting_context_ids": ["passage-1"],
  "supporting_quotes": ["All senior executives will attend."],
  "explanation": "A CEO is typically a senior executive, so their attendance can be inferred, but the context does not name the CEO explicitly.",
  "contradicted": false,
  "contradicting_quote": null
}

**Example 3: Unsupported / Contradicted Claim**

Context: "The software license costs $500 per user per year."
Answer: "The software license is free for academic use."

Output claim:
{
  "claim_text": "The software license is free for academic use.",
  "classification": "unsupported",
  "supporting_context_ids": [],
  "supporting_quotes": [],
  "explanation": "The context states a price of $500 per user per year with no mention of academic pricing or free tiers.",
  "contradicted": true,
  "contradicting_quote": "The software license costs $500 per user per year."
}

To adapt this template for your pipeline, replace [ANSWER] with the generated answer text and [CONTEXT] with your retrieved passages, each prefixed with an ID like passage-1. If your RAG system already includes source metadata, add a [SOURCE_METADATA] placeholder and adjust the output schema to include source titles or URLs. For high-stakes domains such as healthcare or legal, add a [RISK_LEVEL] placeholder that gates whether inferred claims should be treated as unsupported for safety. Always validate the output JSON against the schema before passing it downstream—malformed JSON or missing claim_text fields are common failure modes when the model attempts to summarize rather than quote verbatim.

This prompt is designed to sit after answer generation and before the answer reaches a user or downstream system. It acts as a claim-level grounding audit, classifying each atomic claim in a generated answer as fully_supported, partially_supported, inferred, or unsupported against the retrieved context. The harness must first extract discrete claims from the answer (either via a separate extraction step or by instructing this prompt to perform extraction and classification in one pass), then feed each claim alongside the full retrieved context and the original user question. The output is a structured JSON array of claim objects, each with a claim_text, support_level, evidence_citation, and rationale.

Validation and retries: Parse the JSON output and validate that every claim in the answer is accounted for and that support_level is one of the four allowed enum values. If the output is malformed, retry once with a stricter schema instruction appended to the prompt. If unsupported claims exceed a configured threshold (start with 30% of total claims), route the answer to a human review queue rather than publishing it. Log every classification result with the answer ID, claim index, support level, and rationale for offline eval analysis. For high-stakes domains (healthcare, legal, finance), require human sign-off on any answer containing unsupported or inferred claims before it reaches the user.

Model choice and latency: This classification task benefits from models with strong instruction-following and structured output capabilities. GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro are good starting points. Expect 1-3 seconds of latency per answer depending on claim count. If latency is critical, batch multiple answers in a single request or use a smaller model (e.g., Claude 3 Haiku) with stricter output validation. Tool integration: Wire the classification result into your observability stack—tag traces with support_level distributions and set alerts when the unsupported ratio spikes above baseline. This turns the prompt from a one-off check into a continuous quality signal.

What to avoid: Do not use this prompt as a substitute for retrieval quality improvements. If unsupported rates are consistently high, fix retrieval before adding more classification gates. Do not treat inferred as equivalent to unsupported—inference from context is often legitimate, but it requires separate tracking. Finally, never expose raw classification output to end users; the audit is for internal quality control and routing decisions, not a user-facing feature.