Hallucination Rate Gate Prompt

This prompt is a precision instrument for quality engineers and platform teams who need to gate a prompt's release on its factual accuracy. Its core job is to take a model's output and a set of verified ground-truth references, then produce a structured, machine-readable hallucination report. The ideal user is someone integrating this check into a CI/CD pipeline as a blocking gate, preventing a prompt from being promoted from staging to production if it fabricates unsupported claims. This is essential for workflows where hallucination carries direct business risk, such as customer-facing RAG applications, regulated document summarization, or any system where the model must not invent facts.

The prompt is designed with a specific assumption: you already have a candidate output and a curated set of reference material. It is not a general-purpose quality scorer or a chatbot evaluator. Do not use it to assess creativity, tone, or helpfulness. Do not use it when you lack a ground-truth source to verify against, as the prompt's entire logic depends on aligning claims to evidence. It is also not a substitute for a full RAG evaluation; it focuses narrowly on the factual accuracy of a single generated text against a known set of documents, not on the retrieval step itself.

Before using this prompt, ensure your workflow can provide the required inputs: a clear [MODEL_OUTPUT] and a [REFERENCE_MATERIAL] that contains the complete set of admissible facts. The prompt works best when the reference material is a closed set of documents, such as a specific knowledge base article or a set of legal clauses. If the model is allowed to use general world knowledge, you must explicitly define the boundary between acceptable general knowledge and forbidden fabrication within the [CONSTRAINTS] block. The next section provides the exact prompt template you will need to wire this gate into your release process.

This prompt template is the core instruction set for the Hallucination Rate Gate. It takes an AI-generated [OUTPUT] and a set of [GROUND_TRUTH_REFERENCES], extracts every factual claim, aligns each claim to the evidence, and classifies the result. The output is a machine-readable hallucination report that your CI/CD harness can consume to enforce a maximum fabrication rate before promotion. Use this template as the evaluation step in your release gate pipeline, not as a one-off manual check.

textCopy
You are a factual accuracy auditor. Your task is to evaluate an AI-generated output against provided ground-truth references and produce a structured hallucination report.

## INPUT

### AI-Generated Output to Audit
[OUTPUT]

### Ground-Truth References
[GROUND_TRUTH_REFERENCES]

### Constraints
- Maximum acceptable fabrication rate: [MAX_FABRICATION_RATE]
- Severity classification rules: [SEVERITY_RULES]
- Output schema: [OUTPUT_SCHEMA]

## INSTRUCTIONS

1. **Claim Extraction**: Extract every discrete factual claim from the AI-generated output. A claim is any statement that can be verified as true or false against the references. Include the exact text span and its position in the output.

2. **Evidence Alignment**: For each claim, search the ground-truth references for supporting or contradicting evidence. Record the exact reference text and location that supports or contradicts the claim.

3. **Verification Status**: Classify each claim as one of:
   - `VERIFIED`: Claim is directly supported by reference evidence.
   - `CONTRADICTED`: Claim is directly contradicted by reference evidence.
   - `UNVERIFIABLE`: Claim cannot be confirmed or denied using available references.
   - `FABRICATED`: Claim appears to be invented with no basis in references and no qualifying language.

4. **Severity Classification**: For each FABRICATED or CONTRADICTED claim, assign a severity level using [SEVERITY_RULES].

5. **Fabrication Rate Calculation**: Calculate the fabrication rate as: (number of FABRICATED claims) / (total number of claims).

6. **Gate Decision**: Compare the fabrication rate to [MAX_FABRICATION_RATE]. If the rate exceeds the threshold, mark the gate as FAILED. Otherwise, mark as PASSED.

## OUTPUT

Produce a JSON object conforming exactly to [OUTPUT_SCHEMA]. Do not include commentary outside the JSON structure.

Adapt this template by replacing each square-bracket placeholder with concrete values from your evaluation harness. [OUTPUT] receives the AI-generated text under test. [GROUND_TRUTH_REFERENCES] should contain the authoritative source material, formatted as structured text with location markers. [MAX_FABRICATION_RATE] is your release gate threshold, typically a decimal between 0.0 and 1.0. [SEVERITY_RULES] defines how you classify the impact of each fabrication, such as 'CRITICAL: safety or financial impact, HIGH: materially misleading, MEDIUM: minor factual error, LOW: stylistic or trivial.' [OUTPUT_SCHEMA] should be a JSON Schema object describing the exact structure your harness expects. Wire this prompt into an evaluation runner that executes it against every candidate output before promotion. If your domain is regulated or high-risk, add a human review step for any output where the fabrication rate is non-zero, even if it passes the numeric gate.

The Hallucination Rate Gate Prompt is not a standalone chat interaction; it is a programmatic evaluation step designed to run inside a testing harness. The prompt expects a structured input containing the model's generated output and the corresponding ground-truth reference. The harness is responsible for preparing this input, invoking the LLM, parsing the structured hallucination report, and enforcing the defined rate thresholds. This section covers the integration points, validation logic, and failure handling required to make this gate reliable in an automated release pipeline.

To implement the gate, first build a claim extraction and alignment pre-processor. Before calling the gate prompt, use a separate extraction prompt or a deterministic parser to split the generated output into discrete, verifiable claims. Pair each claim with the relevant evidence span from the ground-truth reference. The gate prompt's [CLAIMS_WITH_EVIDENCE] placeholder should receive a JSON array of objects, each with a claim_id, claim_text, and evidence_text field. The harness must then parse the LLM's JSON response, which should conform to a strict schema: a list of verdicts with claim_id, verdict (one of SUPPORTED, CONTRADICTED, UNVERIFIABLE, FABRICATED), and reasoning. Validate this output immediately. If the JSON is malformed, retry up to two times with a repair prompt that includes the raw output and the expected schema. If parsing still fails, fail the gate with a HARNESS_ERROR status.

The core enforcement logic lives in the threshold evaluator. After successful parsing, calculate the hallucination rate as (FABRICATED claims / total claims) * 100. Compare this rate against the [MAX_HALLUCINATION_RATE] threshold provided in the prompt's constraints. Additionally, check for CONTRADICTED claims, which may trigger a separate, lower threshold. Log every evaluation run with a unique eval_id, the prompt version, the model version, the input hash, the full verdict array, and the calculated rates. This log becomes your audit trail for release decisions. For high-risk domains, route any gate failure or UNVERIFIABLE claim cluster to a human review queue before blocking the release. Never auto-promote a prompt when the hallucination rate exceeds the threshold; the gate's output should be a boolean passed field and a structured report that downstream deployment scripts can consume.

When integrating into a CI/CD pipeline, treat this gate as a blocking quality check in the staging environment. Run it against a curated golden dataset of at least 50 input-output pairs with verified ground-truth references. Cache the LLM responses to avoid cost fluctuations during repeated runs. If the gate fails, the pipeline must halt promotion and surface the hallucination report to the release manager. Avoid the temptation to lower the threshold to let a failing prompt through; instead, use the per-claim verdicts to debug whether the failure is due to retrieval gaps, ambiguous grounding, or model behavior drift. The next step after a gate failure should be a targeted prompt repair or evidence improvement cycle, not a threshold adjustment.