Pre-Release Answer Fact-Check Gate Prompt

This prompt is the final automated quality barrier before a generated answer is delivered to a user. It is designed for release managers and QA engineers who need a single, composite checkpoint that runs multiple hallucination and integrity checks in one pass. The prompt inspects a candidate answer against its source evidence and produces a structured go/no-go recommendation. Use it when a wrong answer carries high risk—such as in customer-facing support, clinical documentation, or legal research—and you need an auditable, automated decision before human review or final release. This is not a prompt for generating answers; it is a prompt for verifying them.

The ideal user is an AI pipeline operator who already has a generated answer and its source context. The prompt requires three inputs: the user's original question, the candidate answer, and the retrieved evidence used to generate that answer. It then executes a battery of checks, including unsupported claim detection, contradiction identification, and citation validity verification. The output is a structured JSON report with a pass or fail recommendation and a detailed failure log. This allows you to programmatically block, flag, or route answers based on a consistent, machine-readable signal.

Do not use this prompt as a substitute for retrieval quality or answer generation logic. It assumes the evidence provided is the complete and correct set of sources; it will not detect retrieval failures where the right document was never fetched. It is also not a replacement for human review in regulated domains—treat the pass recommendation as a filter that reduces review burden, not as a final sign-off. For high-stakes workflows, always route pass outputs to a human review queue and escalate fail outputs for immediate remediation. The prompt works best as the last step in a RAG pipeline, positioned after answer generation but before the response is surfaced to the user.

This prompt template acts as a final quality gate before an AI-generated answer reaches a user. It is designed to be used as a post-generation, pre-release audit step in a RAG pipeline. The prompt instructs the model to act as an auditor, systematically comparing a candidate answer against provided source evidence. It enforces three critical checks: identifying claims the evidence does not support, flagging direct contradictions, and validating that any inline citations genuinely back the statements they are attached to. The output is a structured JSON report that your application can parse to automatically block, flag, or release the answer.

textCopy
You are a final-answer quality gate auditor. Your job is to inspect a candidate answer against provided source evidence and determine whether it is safe to release to a user. You must run three checks: (1) unsupported claims, (2) contradictions with source evidence, and (3) citation validity if citations are present. You must then produce a go/no-go recommendation with a structured failure report.

---

## USER QUESTION
[USER_QUESTION]

## CANDIDATE ANSWER
[CANDIDATE_ANSWER]

## SOURCE EVIDENCE
[SOURCE_EVIDENCE]

---

## AUDIT INSTRUCTIONS

### Check 1: Unsupported Claims
Extract every factual claim from the Candidate Answer. For each claim, determine if it is directly supported by the Source Evidence. A claim is supported only if the evidence explicitly states it or it is a trivial logical deduction from the evidence. Mark each claim as SUPPORTED or UNSUPPORTED. For unsupported claims, quote the exact claim text and explain why the evidence does not support it.

### Check 2: Contradictions
Compare each factual claim in the Candidate Answer against the Source Evidence. Identify any claim that directly contradicts what the evidence states. A contradiction occurs when the answer asserts something that the evidence explicitly denies or states differently. For each contradiction, quote the contradicting claim and the conflicting source passage.

### Check 3: Citation Validity
If the Candidate Answer contains inline citations, verify each one. For each citation, check whether the cited source passage genuinely supports the specific claim it is attached to. Flag any citation that is fabricated, misattributed, or attached to a claim the source does not make.

### Final Recommendation
Based on the three checks above, produce a single recommendation:
- **GO**: No unsupported claims, no contradictions, and all citations are valid. The answer is safe to release.
- **NO_GO**: One or more critical failures detected. The answer must be blocked or sent for human review.
- **CONDITIONAL_GO**: Minor issues detected that do not affect factual accuracy. The answer may be released with caveats logged.

---

## OUTPUT FORMAT
Return a valid JSON object with this exact structure:
{
  "recommendation": "GO | NO_GO | CONDITIONAL_GO",
  "summary": "Brief explanation of the recommendation",
  "unsupported_claims": [
    {
      "claim_text": "Exact claim from the answer",
      "status": "UNSUPPORTED",
      "reason": "Why the evidence does not support this claim"
    }
  ],
  "contradictions": [
    {
      "claim_text": "Exact contradicting claim from the answer",
      "source_passage": "Conflicting passage from the evidence",
      "explanation": "How the claim contradicts the source"
    }
  ],
  "citation_issues": [
    {
      "citation": "The citation as it appears in the answer",
      "attached_claim": "The claim the citation is attached to",
      "issue": "Description of the validity problem"
    }
  ]
}

If no issues are found for a check, return an empty array for that field.

To adapt this template, replace the three bracketed placeholders with your runtime data: [USER_QUESTION] should contain the original user query, [CANDIDATE_ANSWER] should contain the full generated answer you intend to audit, and [SOURCE_EVIDENCE] should contain the concatenated retrieved passages that were used to generate that answer. For best results, ensure the source evidence includes document identifiers or chunk IDs that can be referenced in citations. If your answer generation step does not produce inline citations, the citation validity check will simply return an empty array, and the gate will still function for unsupported claim and contradiction detection.

When integrating this into a production harness, treat the recommendation field as the primary signal for your release logic. A NO_GO should block the answer and either trigger a retry with different evidence, fall back to a safe canned response, or escalate to a human review queue. A CONDITIONAL_GO should log the issues and potentially surface a warning to the end-user. Always validate that the model's output conforms to the specified JSON schema before acting on the recommendation; a malformed response should default to NO_GO to avoid releasing unaudited content. For high-stakes domains like healthcare or finance, route all CONDITIONAL_GO results to human review rather than releasing them automatically.

This prompt is designed to be the last automated check before a generated answer reaches a user. It should be called after the primary answer generation step and any post-generation repair or citation formatting steps. The gate receives the final candidate answer and the complete set of source evidence used to generate it. Its output is a structured JSON object containing a recommendation (GO, CONDITIONAL_GO, or NO_GO) and a detailed failure_report. The prompt is deterministic given its inputs, meaning that for the same answer and evidence, it will produce the same recommendation. Do not retry this prompt on a NO_GO without first changing the answer or the provided evidence; doing so will only waste inference resources and delay the inevitable human review.

To wire this into an application, call the model with response_format set to json_object or use a structured output mode that enforces the expected schema. On receiving the response, validate the JSON structure before acting on the recommendation. If the recommendation is NO_GO, route the entire answer payload—including the generated text, source evidence, and the full audit report—to a human review queue. Do not show the answer to the user. If the recommendation is CONDITIONAL_GO, log the specific issues found in the failure report and consider surfacing a confidence note or caveat to the end user alongside the answer. For high-throughput systems, implement a circuit breaker: if more than a configurable percentage of answers (e.g., 5%) receive a NO_GO within a rolling time window, pause automated releases and alert the operations team. This protects users from systemic failures in retrieval or generation.

Every audit result must be logged with the answer ID, the recommendation, and the counts of each failure type (unsupported claims, contradictions, invalid citations) for observability and trend analysis. Use these logs to build dashboards that track hallucination rates over time, by source, or by query type. When a NO_GO occurs, the human reviewer should have access to the full audit report to quickly identify which claims failed and why. After human remediation, the corrected answer can be re-submitted to the gate for a final check before release. Avoid the temptation to lower the gate's strictness to reduce NO_GO rates; instead, fix the upstream generation or retrieval issues that are causing the failures.