Pass/Fail Groundedness Gate Prompt

This prompt is designed for automated pipelines that require a clear, binary decision on whether an AI-generated output is sufficiently grounded in provided source material. It is not a nuanced quality scorer or a detailed hallucination audit; it is a minimum-bar safety check that returns a structured pass/fail verdict. The primary job-to-be-done is to act as a release gate: before an output reaches a user, a downstream system, or a public surface, this prompt determines if the output meets a predefined factual grounding threshold. Ideal users are MLOps engineers, platform teams, and technical decision makers embedding AI into production systems where ungrounded outputs pose a reputational, legal, or operational risk.

Use this prompt in CI/CD release gates for prompt or model updates, content moderation queues where AI-generated text must be verified against a knowledge base, or any workflow where an accept/reject decision must be logged and acted upon automatically. The prompt requires two inputs: the generated text to evaluate and the source material it should be grounded in. It returns a structured JSON verdict with a pass boolean, a list of failure_reasons when applicable, and an array of evidence_gaps that identify specific unsupported claims. This structure makes it suitable for logging, alerting, and automated blocking. The prompt is designed to be strict by default—when evidence is missing or ambiguous, it fails closed rather than risking a false pass.

Do not use this prompt when you need a calibrated groundedness score on a continuous scale, a detailed hallucination severity classification, or a sentence-level grounding annotation. For those use cases, refer to the Factual Consistency Check Against Source Prompt Template or the Hallucination Severity Classification Prompt in this pillar. This prompt is also not suitable for evaluating outputs where source material is intentionally incomplete or where creative interpretation is expected. In high-risk domains such as healthcare, legal, or finance, always route failures to a human review queue and never rely solely on this gate for final approval. The prompt is a filter, not a substitute for domain expert verification.

This prompt template implements a strict pass/fail groundedness gate for CI/CD pipelines and release automation. It forces a binary decision on whether a generated output is sufficiently supported by the provided evidence, and it requires explicit failure reasons and evidence gaps when the output fails. The template is designed to be dropped directly into a system prompt or user message, with square-bracket placeholders replaced by your application logic before each evaluation run.

textCopy
You are a strict factual grounding auditor. Your only job is to decide whether a generated output is sufficiently grounded in the provided source material. You must return a binary pass/fail decision with explicit reasoning.

## INPUT

**Generated Output to Evaluate:**
[OUTPUT]

**Source Material (ground truth):**
[SOURCE_CONTEXT]

## CONSTRAINTS

[CONSTRAINTS]

## EVALUATION INSTRUCTIONS

1. Extract every factual claim from the Generated Output.
2. For each claim, determine whether it is:
   - **Grounded**: directly supported by the Source Material.
   - **Unsupported**: not found in the Source Material.
   - **Contradicted**: the Source Material says the opposite.
3. Apply the pass/fail threshold defined in CONSTRAINTS.
4. If the output fails, list the specific claims that caused the failure and explain what evidence is missing.

## OUTPUT SCHEMA

Return ONLY valid JSON matching this schema:

{
  "pass": boolean,
  "score": number,
  "failure_reasons": [
    {
      "claim": "string",
      "status": "unsupported | contradicted",
      "explanation": "string",
      "missing_evidence": "string"
    }
  ],
  "grounded_claims_count": number,
  "unsupported_claims_count": number,
  "contradicted_claims_count": number,
  "summary": "string"
}

## RULES

- If no Source Material is provided, automatically fail with the reason "No source material provided for grounding verification."
- If the Generated Output is empty, automatically fail.
- Do not evaluate style, tone, or helpfulness. Only evaluate factual grounding.
- When a claim is partially supported, classify it as unsupported and explain what portion is missing.
- Be strict. When in doubt, fail and explain why.

To adapt this template, replace [OUTPUT] with the AI-generated text under evaluation and [SOURCE_CONTEXT] with the retrieved documents, reference text, or ground-truth passages. The [CONSTRAINTS] placeholder should contain your specific pass/fail threshold rules, such as "Fail if any claim is unsupported or contradicted" for a zero-tolerance gate, or "Fail if more than 5% of claims are unsupported" for a lenient threshold. After pasting, validate that the output schema matches your downstream logging and alerting systems. For high-stakes domains like healthcare or finance, always route failures to a human review queue before blocking a release.

The Pass/Fail Groundedness Gate Prompt is designed to be a deterministic, binary decision point in an automated pipeline, not a human-facing advisory tool. Its primary integration surface is a CI/CD runner, a pre-release validation script, or a content moderation queue where an AI-generated output must be blocked from publication if it fails a minimum factual grounding bar. The harness must treat the model's ACCEPT or REJECT decision as the single source of truth for the gate, log the structured failure reasons and evidence gaps for downstream audit, and halt the pipeline on a REJECT verdict. This prompt is not a scoring rubric; it is a release gate, and the implementation must reflect that by making the pass/fail outcome the primary control flow signal.

To wire this into an application, wrap the prompt call in a function that accepts the generated text and the source evidence as inputs, injects them into the [GENERATED_OUTPUT] and [SOURCE_EVIDENCE] placeholders, and parses the model's response into a strict schema: { "decision": "ACCEPT" | "REJECT", "failure_reasons": [string], "evidence_gaps": [string] }. Use a model with strong instruction-following and JSON mode enabled (e.g., gpt-4o, claude-3-5-sonnet) to minimize format drift. Implement a retry loop with a maximum of 2 attempts if the response fails schema validation or JSON parsing. After a REJECT, write the full payload—including the input text, source evidence, model response, and timestamp—to an immutable audit log before halting the pipeline. Never silently swallow a REJECT; the gate exists to stop bad outputs from shipping, and the audit trail is the only way to diagnose why.

Before deploying this gate to production, build a small golden dataset of 20–30 known-correct and known-hallucinated output-evidence pairs with expected ACCEPT/REJECT labels. Run the gate against this dataset in CI and assert 100% recall on known hallucinations (no ACCEPT on a hallucinated output) and at least 90% precision on grounded outputs (no false REJECT that blocks good content). If the gate is too aggressive, adjust the [GROUNDEDNESS_THRESHOLD] placeholder or add clarifying examples to the prompt's few-shot demonstrations. For high-stakes domains like healthcare or legal content, insert a human review step after a REJECT rather than auto-discarding the output; the gate should flag, not delete, until a human confirms the failure. Finally, monitor the gate's decision rate in production—if the rejection rate spikes after a model or prompt update, the gate itself may need recalibration, not the downstream content.