Source Provenance Tracking Prompt for Audit Trails

Use this prompt when you need to generate an answer that is not just factually grounded, but fully auditable. The primary job is to produce a response where every single claim is explicitly linked to its origin: a specific document ID, a retrieval timestamp, and a chunk reference. This is essential for compliance workflows in finance, healthcare, and legal domains where downstream reviewers or auditors must verify the exact lineage of information without redoing the entire research process. The ideal user is an AI engineer or compliance architect integrating a Retrieval-Augmented Generation (RAG) system into a regulated product where evidence traceability is a hard requirement, not a nice-to-have.

The prompt requires a specific set of structured inputs to function correctly. You must supply a [USER_QUERY], a [RETRIEVED_CONTEXT] array where each object contains doc_id, chunk_id, content, and retrieval_timestamp, and an [AUDIT_REQUIREMENTS] block specifying the expected provenance granularity. The prompt is designed to fail safely: if the model cannot trace a claim back to a source, it must explicitly flag the gap in a missing_provenance alert rather than silently generating an unmoored statement. Do not use this prompt for low-latency chat applications where the overhead of structured provenance generation would degrade user experience, or in scenarios where the retrieval source lacks stable identifiers.

Before deploying this prompt, you must implement a validation harness that checks the output schema, not just the text. The expected output is a JSON object containing an answer string, a provenance_chain array, and a provenance_completeness score. A post-processing script should verify that every sentence in the answer marked as a claim has a corresponding entry in the provenance_chain. If the provenance_completeness score falls below your defined threshold, the system should either trigger a retry with stricter instructions or escalate the request for human review. Avoid using this prompt with models that have weak instruction-following for complex JSON schemas, as partial or malformed provenance objects will break the audit trail.

This template is designed for compliance workflows where every claim must be traceable to its origin. It instructs the model to produce an answer where each factual statement is accompanied by a full provenance record, including the source document identifier, the retrieval timestamp, and the specific chunk or section reference. Use this template when downstream auditors, regulators, or internal review processes require a complete, machine-readable lineage from output back to source data. Do not use this template for low-latency chat applications where provenance overhead would degrade user experience.

textCopy
You are an audit-grade assistant. Your task is to answer the user's question using ONLY the provided [CONTEXT] passages. You must produce a response that includes a complete provenance chain for every factual claim.

## INPUT
User Question: [USER_QUESTION]

## CONTEXT
[CONTEXT]

## OUTPUT_SCHEMA
Your response must be a valid JSON object with the following structure:
{
  "answer_id": "[GENERATE_A_UNIQUE_ID]",
  "answer_summary": "A concise summary of the answer.",
  "claims": [
    {
      "claim_id": "claim-1",
      "claim_text": "The exact factual statement made.",
      "provenance": [
        {
          "source_document_id": "[DOC_ID]",
          "source_chunk_id": "[CHUNK_ID]",
          "retrieval_timestamp": "[TIMESTAMP]",
          "source_text_excerpt": "The exact text from the source supporting this claim."
        }
      ],
      "confidence": "HIGH | MEDIUM | LOW",
      "confidence_rationale": "Brief explanation for the confidence level."
    }
  ],
  "missing_provenance_alerts": [
    {
      "alert_type": "MISSING_SOURCE | STALE_TIMESTAMP | CONFLICTING_EVIDENCE",
      "detail": "Description of the provenance gap.",
      "affected_claim_ids": ["claim-1"]
    }
  ],
  "audit_trail_completeness": "COMPLETE | PARTIAL | INCOMPLETE"
}

## CONSTRAINTS
1. Every claim in the 'claims' array MUST have at least one entry in its 'provenance' array.
2. If a claim cannot be fully supported by the provided [CONTEXT], set its confidence to LOW and add a MISSING_SOURCE alert.
3. If a source's retrieval timestamp is older than [STALENESS_THRESHOLD_HOURS] hours, add a STALE_TIMESTAMP alert.
4. If two sources provide conflicting information for the same claim, add a CONFLICTING_EVIDENCE alert and set the claim's confidence to LOW.
5. Do not invent any source_document_id, source_chunk_id, or retrieval_timestamp. Use only the metadata provided in the [CONTEXT] passages.
6. If the [CONTEXT] is empty or contains no relevant information, return an empty 'claims' array, set 'audit_trail_completeness' to INCOMPLETE, and add a single MISSING_SOURCE alert explaining the gap.

To adapt this template, replace the [USER_QUESTION] and [CONTEXT] placeholders with your application's runtime values. The [CONTEXT] placeholder should be populated with pre-formatted text blocks, each prepended with its metadata (document ID, chunk ID, timestamp) in a consistent, parseable format. The [STALENESS_THRESHOLD_HOURS] placeholder should be set to your organization's data freshness policy. Before deploying, validate that your retrieval pipeline consistently provides the required metadata fields; a missing retrieval_timestamp in the source data will cause the model to either hallucinate a value or trigger a false MISSING_SOURCE alert. Always pair this prompt with a post-generation validation step that parses the JSON output and programmatically verifies that every source_document_id exists in your system of record.

The Source Provenance Tracking Prompt is not a standalone chat interaction—it is a component inside a broader compliance pipeline. The typical harness wraps the prompt in a retrieval step, a generation step, a structured validation step, and a persistence step that writes the audit record. Before calling the model, assemble the [INPUT] (the user question or analysis request) and the [CONTEXT] (retrieved documents with their IDs, chunk references, and retrieval timestamps). The [OUTPUT_SCHEMA] should be a strict JSON schema that includes a claims array where each claim object has claim_text, source_document_id, source_chunk_id, retrieval_timestamp, and confidence fields. The [CONSTRAINTS] block should enforce that every claim must have a non-null source reference and that claims without provenance must be listed in a separate unattributed_claims array with an explicit provenance_gap reason.

After the model returns, run a provenance completeness validator before the output enters any audit log. This validator checks that every claim in the claims array has a populated source_document_id and source_chunk_id, that the referenced document IDs actually exist in the retrieval set you provided, and that the retrieval_timestamp values are within an acceptable freshness window. If the validator finds unattributed claims in the main claims array (not the unattributed_claims array), or if source references point to documents not in the input context, flag the output for repair. Implement a retry loop with a maximum of two additional attempts: on retry, append the validation errors to the prompt as [PREVIOUS_ERRORS] and instruct the model to fix the provenance gaps or move unsupported claims to the unattributed_claims array. If the third attempt still fails validation, escalate to a human review queue rather than silently accepting a broken audit trail. Log every attempt, the validation results, and the final disposition (accepted, repaired, or escalated) with timestamps and operator IDs.

For high-risk compliance workflows, add a human approval gate even when validation passes. Route outputs with low confidence scores, multiple unattributed claims, or source conflicts to a review interface that displays the original question, the retrieved documents, and the model's provenance chain side by side. The reviewer can approve, reject, or amend the audit record before it is persisted. When choosing a model, prefer variants with strong instruction-following for structured output (GPT-4o, Claude 3.5 Sonnet, or equivalent). Avoid models known to drop fields or invent source references under pressure. Wire the entire harness into your observability stack: trace every prompt assembly, model response, validation result, retry, and human action so that the audit trail itself is auditable. The prompt is the instruction layer—the harness is what makes it trustworthy in production.