Evidence Transparency Report Generation Prompt Template

This prompt is designed for governance teams, AI auditors, and compliance engineers who need to produce a standardized, structured transparency report for a query-answer pair. The report documents source selection criteria, ranking methodology, conflict resolution steps, and grounding verification results. Use this prompt when an AI system's output must be accompanied by an auditable evidence trail for internal review, regulatory compliance, or customer trust requirements. This prompt assumes that the query, final answer, and a set of retrieved evidence passages are already available. It does not perform retrieval or answer generation; it documents the evidential basis of an existing answer.

The ideal user is someone who already has a complete query-answer-evidence tuple and needs to generate a machine-readable, schema-compliant report that can be stored, reviewed, or submitted as part of an audit package. This prompt is not a replacement for a retrieval pipeline, a hallucination detector, or an answer generator. It is a documentation tool that sits downstream of those components. If you need to verify whether an answer is actually supported by evidence, use the Answer Grounding Verification Prompt Template or Hallucination Flagging with Source Evidence Prompt Template instead. If you need to rank evidence before generating an answer, use the Evidence Ranking and Prioritization playbook.

Do not use this prompt when the underlying evidence is unavailable, when the answer was generated without retrieval, or when the system cannot provide the raw passages that informed the response. The report's value depends entirely on the completeness and honesty of the inputs. If the evidence set is incomplete or the answer was generated from model parametric knowledge, the transparency report will be misleading. For high-risk domains such as healthcare, legal, or financial compliance, always pair this prompt with a human review step and ensure that the evidence passages are logged immutably before report generation begins.

The prompt template below generates a standardized evidence transparency report for a given query-answer pair. It requires the original query, the generated answer, the full set of retrieved evidence passages, and the ranking methodology used. Replace each square-bracket placeholder with real data before sending this prompt to the model. The template enforces a strict output schema so that downstream systems can parse and store the report reliably.

textCopy
You are an evidence transparency auditor. Your task is to produce a structured transparency report for a query-answer pair, documenting how evidence was selected, ranked, and used to ground the answer.

## INPUTS

**Query:** [QUERY]

**Generated Answer:** [ANSWER]

**Retrieved Evidence Passages:**
[EVIDENCE_PASSAGES]

**Ranking Methodology:** [RANKING_METHODOLOGY]

**Source Metadata (optional):** [SOURCE_METADATA]

## OUTPUT SCHEMA

Return a JSON object with the following structure:

```json
{
  "report_id": "string",
  "query": "string",
  "answer": "string",
  "evidence_summary": {
    "total_passages_retrieved": "integer",
    "total_passages_selected": "integer",
    "selection_criteria": ["string"]
  },
  "ranking_methodology": {
    "description": "string",
    "criteria": [
      {
        "name": "string",
        "weight": "number",
        "description": "string"
      }
    ]
  },
  "selected_evidence": [
    {
      "passage_id": "string",
      "passage_text": "string",
      "rank": "integer",
      "relevance_score": "number",
      "selection_rationale": "string",
      "source_authority": "string",
      "recency": "string"
    }
  ],
  "conflict_analysis": {
    "conflicts_detected": "boolean",
    "conflicts": [
      {
        "passage_ids": ["string"],
        "conflict_description": "string",
        "resolution": "string"
      }
    ]
  },
  "grounding_verification": {
    "claims_in_answer": [
      {
        "claim": "string",
        "supporting_passage_ids": ["string"],
        "support_level": "fully_supported | partially_supported | unsupported",
        "explanation": "string"
      }
    ],
    "overall_grounding_score": "number",
    "unsupported_claims_count": "integer"
  },
  "sufficiency_assessment": {
    "evidence_sufficient": "boolean",
    "information_gaps": ["string"],
    "recommendation": "answer | refuse | request_more_context"
  },
  "confidence_statement": "string"
}

CONSTRAINTS

• Every claim in the answer must be mapped to at least one evidence passage in the grounding verification section.
• If a claim has no supporting passage, mark it as "unsupported" and include it in the unsupported_claims_count.
• For conflicts, describe the nature of the disagreement and how it was resolved. Do not fabricate resolution if none exists.
• The confidence statement must reflect evidence quality, source agreement, and information completeness. Do not express confidence higher than the evidence supports.
• If evidence is insufficient to answer the query, set evidence_sufficient to false and recommend "refuse" or "request_more_context."
• All scores must be on a 0.0 to 1.0 scale.
• Do not invent evidence passages, source metadata, or ranking criteria not present in the inputs.

RISK LEVEL: [RISK_LEVEL]

If RISK_LEVEL is "high," add a human_review_required flag to the output and include a reviewer_notes field explaining what a human reviewer should verify before the report is finalized.

To adapt this template, replace [QUERY] and [ANSWER] with the original user question and the model's generated response. Populate [EVIDENCE_PASSAGES] with the full set of retrieved documents, including passage IDs and text. Provide [RANKING_METHODOLOGY] as a description of how passages were ordered, including any scoring functions or criteria. If source metadata such as publication dates or author credentials is available, include it in [SOURCE_METADATA]. Set [RISK_LEVEL] to "high" for regulated domains, safety-critical applications, or audit-facing reports; otherwise use "standard." After generating the report, validate the JSON against the schema and run eval checks for orphan claims, hallucinated citations, and overconfident sufficiency assessments before storing or presenting the report.

This prompt is designed to be called after a query-answer pair has been generated and the full retrieval context is available. The implementation harness should treat the transparency report as a post-generation audit artifact, not a real-time user-facing response. Wire it into an asynchronous pipeline that triggers after the primary RAG response is logged, ensuring the report is generated for every high-risk or regulated interaction. The prompt expects structured inputs: the original query, the final answer, the ranked evidence list with metadata, and any conflict or sufficiency flags already computed by upstream pipeline stages.

Validation and Schema Enforcement: The prompt template includes an [OUTPUT_SCHEMA] placeholder that should be populated with a strict JSON Schema definition. After the model returns a response, run a server-side validator (e.g., jsonschema in Python, zod in TypeScript) against the output. If validation fails, implement a single retry loop that feeds the validation error back into the prompt's [CONSTRAINTS] field, instructing the model to repair the specific structural issue. Do not retry more than twice; after two failures, log the raw output and flag the report for human review. For high-stakes compliance use cases, never silently accept a malformed report.

Logging, Review, and Model Choice: Every generated transparency report should be logged immutably alongside the original query, answer, evidence set, and model version. Use a model with strong instruction-following and structured output capabilities (e.g., GPT-4o, Claude 3.5 Sonnet) for this task, as weaker models often drop required fields or hallucinate source metadata. For regulated industries, route all reports through a human review queue before they become part of an official audit trail. The review interface should display the report side-by-side with the source evidence, allowing a reviewer to accept, reject, or annotate each section. Finally, avoid calling this prompt in the hot path of a user-facing chat; its latency and token cost are justified only for audit, compliance, and governance workflows where traceability is non-negotiable.