Audit-Ready Confidence Explanation Generation Prompt Template

This prompt is for compliance teams, verification system architects, and AI pipeline operators who need to produce explainable confidence scores that can survive an audit. The job-to-be-done is not just assigning a confidence level to a claim, but generating a structured, traceable explanation that links that confidence to specific evidence items, source assessments, and the scoring logic applied. The ideal user is someone integrating a verification step into a product where downstream decisions—publishing, escalating, rejecting, or acting on a claim—require a documented rationale that a human reviewer or auditor can inspect later.

Use this prompt when your verification pipeline must produce outputs that answer why a confidence score was assigned, not just what the score is. This is essential in regulated domains, internal governance workflows, and any system where confidence scores influence high-stakes decisions. The prompt requires several inputs to function correctly: a specific claim text, a set of evidence items with source metadata, the scoring rubric or criteria used, and any relevant domain-specific evidence standards. Do not use this prompt for real-time, low-latency applications where a simple numeric score suffices, or when the evidence set is empty—this prompt is designed to explain a decision, not to make one in the absence of data. It is also inappropriate for binary true/false classification tasks where explanation overhead adds no value.

Before wiring this prompt into production, ensure you have a defined confidence scoring method that the explanation can reference. The prompt works best when it can cite concrete factors such as source authority tiers, corroboration counts, recency windows, and contradiction flags. If your scoring logic is a black box or purely model-generated without explicit criteria, the resulting explanations will be unfalsifiable and fail audit review. Plan to pair this prompt with a validation step that checks whether the explanation actually references the evidence items provided and whether the cited scoring logic matches your documented rubric. For high-risk domains, always route outputs through a human review queue before they become part of a permanent audit trail.

The output of this prompt should be treated as a draft audit artifact, not a final compliance document. It provides the structured reasoning, but your application layer should attach metadata such as the prompt version, model version, timestamp, and a unique trace ID before storage. This ensures that if a confidence decision is later challenged, you can replay the exact inputs and verify the explanation's fidelity. Avoid using this prompt for claims where the evidence is highly ambiguous or where multiple contradictory sources carry equal authority—those cases require explicit contradiction resolution prompts before explanation generation, or the resulting explanation will paper over unresolved conflicts with vague language.

This prompt template is the core instruction set for generating an audit-ready confidence explanation. It forces the model to decompose its reasoning, linking the final confidence score directly to the evidence used, the authority of each source, and any contradictions found. The output is a structured object designed for logging, compliance review, and debugging, not just a human-readable paragraph. Use this template as the foundation for your explanation generation step, adapting the placeholders to your specific verification pipeline's inputs and output schema.

markdownCopy
You are an expert verification auditor. Your task is to produce a structured, traceable explanation for a confidence score assigned to a specific claim. Every statement in your explanation must be grounded in the provided [EVIDENCE_SET] and [SOURCE_ASSESSMENTS]. Do not introduce external knowledge.

**Inputs:**
- **Claim:** [CLAIM_TEXT]
- **Confidence Score:** [CONFIDENCE_SCORE] (on a scale of [SCORE_RANGE], e.g., 0.0 to 1.0)
- **Evidence Set:** [EVIDENCE_SET] (A list of evidence items, each with an ID, source, and text snippet)
- **Source Assessments:** [SOURCE_ASSESSMENTS] (A mapping of source IDs to authority scores, recency, and domain relevance)
- **Scoring Logic Summary:** [SCORING_LOGIC_SUMMARY] (A brief description of how the confidence score was calculated, e.g., 'weighted average of corroboration and authority, penalized by contradiction severity')

**Output Requirements:**
Generate a JSON object conforming to the following [OUTPUT_SCHEMA].

**Constraints:**
- [CONSTRAINTS]
  1. Every evidence reference must use the exact Evidence ID from [EVIDENCE_SET].
  2. The `explanation_summary` must be a single, concise paragraph suitable for a non-technical auditor.
  3. The `evidence_chain` must explicitly state whether each piece of evidence supports, contradicts, or is neutral to the claim.
  4. If the confidence score is below [LOW_CONFIDENCE_THRESHOLD], the `recommended_action` field must be set to 'HUMAN_REVIEW'.
  5. Do not hallucinate any evidence or source details not present in the input.

**Output Schema:**
{
  "claim": "string",
  "confidence_score": "number",
  "explanation_summary": "string",
  "evidence_chain": [
    {
      "evidence_id": "string",
      "relevance": "SUPPORTS | CONTRADICTS | NEUTRAL",
      "rationale": "string (how this evidence item influenced the score)"
    }
  ],
  "source_authority_impact": [
    {
      "source_id": "string",
      "authority_score": "number",
      "impact_on_confidence": "INCREASED | DECREASED | NO_IMPACT",
      "rationale": "string"
    }
  ],
  "contradiction_analysis": {
    "detected": "boolean",
    "details": "string (explain the contradiction and its effect on the score, or state 'None detected.')",
    "severity": "HIGH | MEDIUM | LOW | NONE"
  },
  "audit_trail": {
    "scoring_logic_applied": "string (from [SCORING_LOGIC_SUMMARY])",
    "missing_evidence_notes": "string (flag any expected evidence types that were absent)"
  },
  "recommended_action": "AUTO_APPROVE | HUMAN_REVIEW | REJECT"
}

To adapt this template, replace the square-bracket placeholders with your system's actual data structures. The [EVIDENCE_SET] and [SOURCE_ASSESSMENTS] are the most critical inputs; they must be pre-processed and formatted correctly before being injected into the prompt. The [OUTPUT_SCHEMA] can be simplified or extended, but the core elements—evidence chain, source impact, and contradiction analysis—are essential for auditability. For high-stakes domains like finance or healthcare, the [CONSTRAINTS] should be treated as a minimum set; add domain-specific rules, such as requiring specific regulatory citations or flagging evidence older than a certain date. Always test the prompt's adherence to the schema with a validation step in your application code.

The Audit-Ready Confidence Explanation prompt is not a standalone artifact—it is a component inside a verification pipeline that must produce traceable, defensible outputs. The implementation harness wraps the prompt call in validation checks, retry logic, structured logging, and a human-review escape hatch. Because the output is intended for compliance and audit contexts, the harness must guarantee that every explanation is structurally complete, internally consistent, and linked to specific evidence items before it reaches a downstream system or reviewer.

Wire the prompt into your application with a pre-call evidence assembly step that collects the claim, its confidence score, the evidence items used, source metadata, and any contradiction flags into a structured input object. After the model responds, run a post-call structural validator that checks for required fields: claim_id, confidence_score, explanation_summary, evidence_items (each with source_id, relevance, and impact_on_confidence), scoring_factors, and uncertainty_notes. If the validator fails, retry once with the validation errors appended to the prompt as [CONSTRAINTS]. If the retry also fails, route the claim to a human-review queue with the raw model output and validation failure details attached. Log every attempt—input, output, validator result, and retry decision—with a unique explanation_run_id for audit trail reconstruction.

For high-stakes domains, add a consistency check after structural validation: confirm that the explanation's stated confidence score matches the input score within a tolerance of ±0.05, that every cited evidence item exists in the input evidence set, and that the explanation's reasoning direction (supporting vs. contradicting) aligns with the evidence polarity. If the consistency check fails, do not retry automatically—escalate directly to human review with a consistency_failure flag and the specific mismatch described. Model choice matters here: use a model with strong instruction-following and structured output reliability (GPT-4o, Claude 3.5 Sonnet, or equivalent). Avoid smaller or older models that are prone to hallucinating evidence references or fabricating source metadata. This prompt does not require tool use or retrieval at runtime—all evidence must be assembled before the call—but it pairs naturally with upstream retrieval and scoring pipelines that feed it verified inputs.