Source Attribution Audit Prompt for Generated Text

This prompt is for compliance officers, governance teams, and AI safety engineers who need to verify that generated text properly attributes every factual claim to a source. The job-to-be-done is a systematic audit: given a piece of generated text and the source material it claims to be based on, produce a structured report that identifies missing citations, over-citation where a source does not support the claim, and attribution errors where the wrong source is cited. The ideal user is someone responsible for sign-off before content is published, submitted to a regulator, or used in a high-stakes decision. They need more than a spot check—they need a repeatable, evidence-backed audit trail.

Use this prompt when the cost of an unattributed or misattributed claim is high: regulatory filings, medical summaries, legal briefs, financial reports, or customer-facing answers where trust is the product. The prompt requires two concrete inputs: the [GENERATED_TEXT] to audit and the [SOURCE_MATERIAL] that should contain supporting evidence. It works best when source material is chunked or paragraph-numbered so the model can cite specific locations. Do not use this prompt for real-time chat moderation or for evaluating creative writing where factual grounding is not expected. It is also not a replacement for a full human audit in life-safety or legally binding contexts—always pair it with human review for high-risk workflows.

Before running this prompt, ensure you have a clear definition of what counts as a 'claim' in your domain. The prompt includes a [CLAIM_DEFINITION] placeholder so you can scope it to factual assertions, numerical statements, or interpretative claims depending on your risk tolerance. After the audit, feed the structured output into your observability stack and track metrics like 'claims per paragraph without a citation' or 'over-citation rate' to detect drift in your generation pipeline. If the audit report flags systemic failures, escalate to a human reviewer and consider tightening your RAG retrieval or citation insertion logic before regenerating.

This template is designed to be dropped into an AI application harness that audits generated text for source attribution quality. It instructs the model to act as a compliance auditor, cross-referencing claims in the generated text against a provided set of source passages. The prompt uses square-bracket placeholders for all dynamic inputs, making it easy to parameterize in code. Before using this prompt in production, ensure you have a reliable method for extracting the structured output and validating it against a defined schema.

textCopy
You are a source attribution auditor for an AI system. Your task is to audit the provided [GENERATED_TEXT] against the [SOURCE_PASSAGES] to detect missing citations, over-citation, and attribution errors.

[GENERATED_TEXT]
[SOURCE_PASSAGES]

[OUTPUT_SCHEMA]

[CONSTRAINTS]

[EXAMPLES]

[RISK_LEVEL]

To adapt this template, replace each placeholder with concrete content. [GENERATED_TEXT] should contain the AI-generated text to be audited. [SOURCE_PASSAGES] should be a list of source objects, each with a unique ID and text content. [OUTPUT_SCHEMA] must define the exact JSON structure for the audit report, including fields for claim extraction, source mapping, and error flags. [CONSTRAINTS] should specify rules like 'Flag any claim not directly supported by a source passage' or 'Ignore stylistic attributions.' [EXAMPLES] should provide a few-shot demonstration of a correct audit report. [RISK_LEVEL] should be set to 'HIGH' for regulated domains, which will trigger stricter internal checks and a mandatory human review recommendation in the output. After generating the audit report, always validate it against the [OUTPUT_SCHEMA] in your application code before presenting it to a user or logging it for compliance.

The Source Attribution Audit Prompt is not a one-shot generation call; it is a verification step that should be integrated into a post-generation pipeline. After your primary system produces a cited answer, route the output through this audit prompt before it reaches the user or a downstream system of record. The harness must supply the generated text and the full set of retrieved source passages as [INPUT] and [EVIDENCE] respectively. Without the original evidence set, the audit cannot detect fabricated citations or unsupported claims, which is its primary value.

Implement the harness as an asynchronous validation worker. On receiving a generation result, call the audit prompt with a structured output schema that includes audit_verdict (pass/fail), violations (an array of objects with claim, cited_source, actual_support, and severity), and remediation_instructions. Parse the response and route based on the verdict: if pass, release the answer. If fail, check your retry budget. A common pattern is to feed the remediation_instructions back into a regeneration prompt that has access to the same evidence set, then re-audit the corrected output. Cap this loop at 2-3 retries to prevent infinite correction cycles. Log every audit result, including the full violation array, for observability and compliance traceability.

For high-stakes domains such as legal, clinical, or financial compliance, insert a human review gate after the first audit failure. Do not auto-regenerate and release without human approval when severity is critical or when violations involve fabricated sources. The audit prompt's output should be stored alongside the final answer as an audit artifact. Use a model with strong instruction-following and structured output support (such as gpt-4o or claude-3.5-sonnet) and set temperature=0 to maximize consistency. Avoid using this prompt on streaming outputs; it requires the complete generated text to perform a thorough claim-by-claim audit.