Domain-Specific Hallucination Detection Prompt for Legal Outputs

This prompt is a specialized guardrail for legal-tech product teams who need to validate AI-generated outputs before they reach an attorney or client. It detects fabricated case citations, misstated statutes, incorrect party names, and unsupported legal conclusions in contract analysis, case summarization, or regulatory responses. Use this prompt when the cost of a hallucinated legal detail is high and a human-in-the-loop verification step is required. It is designed for high-recall detection, meaning it will flag more potential issues to ensure nothing slips through.

Deploy this prompt as a mandatory validation step in any pipeline where a language model generates a legal document, summary, or response based on provided source material. The prompt expects two structured inputs: a set of verified legal source passages and the AI-generated output to audit. It is not a general-purpose fact-checker; it requires the source material to be pre-validated and formatted. The prompt works best when the generated output contains specific, verifiable claims—citations, statutory references, party names, or legal conclusions—that can be directly matched against the source evidence. Do not use this prompt to evaluate creative legal arguments, assess the quality of legal advice, or replace a qualified attorney's review.

Before integrating this prompt, ensure your pipeline can supply clean, chunked source documents with stable identifiers. The prompt's effectiveness depends entirely on the quality and completeness of the source context provided. If your retrieval system cannot reliably surface the relevant statutes, case law, or contract clauses, this guardrail will generate false positives that erode user trust. Start by running this prompt against a golden dataset of known hallucinations and valid outputs to calibrate your flagging threshold. Pair it with a human review queue for all flagged outputs, and log every detection event to build a feedback loop for improving both your generation and retrieval systems over time.

This prompt template is designed to be pasted directly into your model's system message. It instructs the model to act as a legal hallucination auditor, comparing a generated legal output against provided source documents. The template uses square-bracket placeholders for all variable inputs—replace each with your actual data before sending the request. The prompt is structured for high-recall detection, meaning it prioritizes flagging potential issues over avoiding false positives, with the understanding that a human reviewer will verify all flags.

textCopy
You are a legal hallucination auditor. Your task is to compare a [LEGAL_OUTPUT] against provided [SOURCE_DOCUMENTS] and identify any statements that are not supported by the evidence.

## INPUT
[LEGAL_OUTPUT]

## SOURCE DOCUMENTS
[SOURCE_DOCUMENTS]

## DETECTION RULES
1. Flag any case citation where the case name, year, court, or holding cannot be verified in the source documents.
2. Flag any statutory reference where the statute number, section, or quoted language does not appear in the sources.
3. Flag any party name, date, dollar amount, or procedural fact that is not present in the sources.
4. Flag any legal conclusion stated as fact when the sources only provide factual background without reaching that conclusion.
5. Flag any quotation marks containing language not found verbatim in the sources.
6. Do NOT flag paraphrased statements that accurately reflect source content.
7. Do NOT flag reasonable inferences clearly supported by the sources.

## CITATION FORMAT VALIDATION
For each citation in the output, verify:
- Case citations follow [CITATION_STYLE] format (e.g., Bluebook, ALWD).
- Statutory citations include correct title, section, and jurisdiction.
- Pinpoint citations reference actual page or paragraph numbers from the source.

## OUTPUT FORMAT
Return a JSON object with this exact schema:
{
  "hallucination_flags": [
    {
      "flagged_text": "exact text from output",
      "flag_type": "fabricated_citation | misstated_statute | incorrect_party | unsupported_conclusion | fabricated_detail | citation_format_error",
      "severity": "critical | high | medium | low",
      "explanation": "why this is flagged",
      "source_evidence": "relevant excerpt from sources or null if none exists",
      "suggested_correction": "correction if known, or null"
    }
  ],
  "overall_assessment": {
    "total_flags": number,
    "critical_flags": number,
    "requires_human_review": boolean,
    "summary": "brief assessment of output reliability"
  }
}

## CONSTRAINTS
- Only flag statements that lack source support. Do not flag stylistic choices or formatting.
- If a citation appears correct but the source does not include enough detail to verify, flag it as "unverifiable" with severity "medium."
- For [RISK_LEVEL] = "high", apply stricter scrutiny to legal conclusions and procedural statements.
- If no hallucinations are detected, return an empty flags array with requires_human_review set to false.

To adapt this template, replace the bracketed placeholders with your actual data. [LEGAL_OUTPUT] should contain the full text of the AI-generated legal analysis, contract summary, or case brief. [SOURCE_DOCUMENTS] should include all evidence passages, case texts, or statutory excerpts that were provided to the generation model. [CITATION_STYLE] should specify your required citation format (e.g., "Bluebook 21st edition" or "ALWD Guide 7th edition"). [RISK_LEVEL] accepts "high," "medium," or "low" to adjust detection sensitivity. For production use, always route flagged outputs to a human reviewer before they reach end users—this prompt is a detection tool, not a replacement for legal judgment.

This prompt is designed as a pre-release guardrail, not a post-hoc audit tool. It should sit between the primary legal reasoning model's output and the end user, evaluating every response before it is shown. The harness must treat the detection prompt as a high-recall safety net: it is better to flag a grounded statement for human review than to let a fabricated case citation reach a lawyer. Wire this prompt into an asynchronous validation pipeline where the primary model's output and the source context are passed together. The detection prompt returns a structured JSON report; your application must parse this report and enforce a routing decision—pass, block, or escalate—based on the flagged_statements array and the overall_risk_score.

Validation and Routing Logic: Parse the model's JSON output with a strict schema validator. If the output fails to parse, retry once with a repair prompt that includes the raw output and the expected schema. Once parsed, check overall_risk_score. If the score is below your low-risk threshold (e.g., < 0.3) and flagged_statements is empty, route the original output to the user. If the score exceeds a high-risk threshold (e.g., > 0.7) or any flagged statement has severity: 'critical', block the output entirely and log the incident. For scores in the middle band, or when flagged_statements contains only severity: 'minor' items, queue the output for human review with the detection report attached. Citation format validation requires an additional deterministic check: extract all citation strings from the original output using a regex pattern for legal citations (e.g., \d+\s+[A-Za-z.]+\s+\d+ for case reporters, or \d+\s+U\.S\.C\.\s+§\s+\d+ for statutes) and verify each against a known citation database or court API before the LLM detection step. This hybrid approach—deterministic citation lookup plus LLM semantic grounding check—catches fabricated citations that look real but don't exist.

Model Choice and Latency Budget: Use a fast, instruction-following model for this detection task (e.g., GPT-4o, Claude 3.5 Sonnet, or a fine-tuned open-weight model). The detection prompt is a secondary call that adds latency; budget 2–5 seconds for this step. If your primary legal output generation already takes 10+ seconds, this overhead is acceptable. For streaming applications, buffer the full output before running detection—do not attempt to run this prompt on partial streams. Logging and Observability: Log every detection result with the original output, source context, detection report, and routing decision. Include a unique trace_id that links the primary generation and the detection step. This audit trail is essential for legal-tech products where every decision must be explainable. Human Review Integration: When escalation is triggered, present the reviewer with a side-by-side view: the original output on the left, the detection report on the right, with flagged statements highlighted inline. The reviewer should be able to approve, edit, or reject each flagged statement. Track reviewer decisions to measure the detection prompt's precision and recall over time, and use this data to tune your risk thresholds.

What to avoid: Do not use this prompt as the sole defense in high-stakes legal workflows. Always combine it with deterministic citation validation, source document retrieval quality checks, and human review for critical outputs. Do not run this prompt without a schema validator—the model may return malformed JSON under load. Do not skip logging; without traceability, you cannot improve the detection pipeline or defend its decisions in an audit. Start with a high-recall threshold (flag more, block less) and tighten precision as you gather reviewer feedback on false positives.