Extraction Audit Log Prompt with Confidence Trail

This prompt is for compliance engineers, data platform architects, and audit teams who need to prove exactly where every extracted field came from. The job-to-be-done is not just extraction—it's producing a defensible, record-level audit trail that maps each output field to its source span, a confidence score, a null reason code if absent, and an extraction timestamp. Use this when the downstream consumer is an auditor, a regulator, or an internal governance process that will reject any extraction lacking traceable provenance.

The ideal user has already defined a target schema and has source documents ready. Required context includes the raw document text, the extraction schema with field definitions, and a null reason code taxonomy. This prompt is overkill for low-stakes extraction where a missing field is acceptable without explanation. Do not use it for real-time chat applications, high-throughput streaming pipelines where per-record audit overhead is prohibitive, or when the source text is already structured and machine-readable—a simple parser will be faster, cheaper, and more reliable.

Before deploying, confirm that your pipeline can store and index the audit log payload. The prompt produces verbose output: every field gets a source span, confidence score, and null reason. If your system cannot retain that evidence, the prompt's value collapses. Also recognize that this prompt trades latency and token cost for traceability. If you need fast, best-effort extraction without audit requirements, start with a simpler field extraction prompt and add audit logging only when governance demands it.

The prompt below is designed to be dropped into an extraction pipeline where every field must be traceable back to its origin in the source text. It enforces a strict output schema that includes per-field confidence scores, source spans, null reason codes, and extraction timestamps. Use this template when downstream consumers—compliance reviewers, audit systems, or data quality dashboards—need to understand not just what was extracted, but how certain the extraction was and where the evidence lives.

textCopy
You are an extraction auditor. Your job is to read the provided source text and extract the fields defined in [OUTPUT_SCHEMA]. For every field, you must produce an audit record that includes the extracted value, the exact source span that supports it, a confidence score between 0.0 and 1.0, a null reason code if the value is absent, and an extraction timestamp.

## INPUT
Source text:
[INPUT]

## OUTPUT SCHEMA
Return a single JSON object conforming to this structure:
{
  "record_id": "[RECORD_ID]",
  "extraction_timestamp": "ISO-8601 timestamp of extraction",
  "fields": [
    {
      "field_name": "name from schema",
      "extracted_value": "the value or null",
      "source_span": "exact substring from source text that supports this value, or null",
      "confidence": 0.0-1.0,
      "null_reason": "one of: MISSING_IN_SOURCE | REDACTED | NOT_APPLICABLE | EMPTY_VALUE | AMBIGUOUS | EXTRACTED",
      "justification": "brief explanation of the confidence score and null reason"
    }
  ],
  "completeness_check": {
    "total_fields": "number of fields in schema",
    "fields_extracted": "number of fields with non-null values",
    "fields_null": "number of null fields",
    "average_confidence": "mean confidence across all fields",
    "low_confidence_fields": ["list of field names with confidence below [CONFIDENCE_THRESHOLD]"]
  }
}

## CONSTRAINTS
[CONSTRAINTS]

## NULL REASON CODES
- MISSING_IN_SOURCE: The field is expected but no evidence exists in the source text.
- REDACTED: Evidence appears to have been intentionally removed or obscured.
- NOT_APPLICABLE: The field does not apply to this record type.
- EMPTY_VALUE: The source contains the field but the value is explicitly empty.
- AMBIGUOUS: Multiple possible values exist and none can be selected with confidence above [CONFIDENCE_THRESHOLD].
- EXTRACTED: A value was successfully extracted.

## CONFIDENCE SCORING RULES
- 0.9-1.0: Value is explicitly stated in source with no ambiguity.
- 0.7-0.89: Value is clearly implied but not verbatim.
- 0.5-0.69: Value requires inference from multiple spans or indirect evidence.
- 0.3-0.49: Value is a best guess with significant uncertainty.
- 0.0-0.29: Value is speculative or unsupported.

## EXAMPLES
[EXAMPLES]

## RISK LEVEL
[RISK_LEVEL]

Return ONLY the JSON object. Do not include markdown fences, commentary, or additional text.

To adapt this template, replace each square-bracket placeholder with your specific configuration. [OUTPUT_SCHEMA] should define the exact fields you need extracted—include field names, types, and whether each field is required or optional. [CONSTRAINTS] is where you add domain-specific rules, such as date format requirements, enum value restrictions, or normalization rules. [CONFIDENCE_THRESHOLD] sets the bar for flagging fields in the low_confidence_fields array; start with 0.7 and tune based on your tolerance for false positives versus missed extractions. [EXAMPLES] should include at least two few-shot demonstrations showing both high-confidence and low-confidence extractions with correct null reason codes. [RISK_LEVEL] controls the tone of the instructions—set to HIGH for regulated domains where the prompt should refuse to guess, or STANDARD for internal analytics where some inference is acceptable. Before deploying, validate that the output JSON matches your schema exactly and run at least 20 test cases covering missing fields, ambiguous spans, redacted content, and edge cases where the source text is empty or malformed.

This prompt is designed to be called as part of a record-level extraction pipeline, not as a one-off chat interaction. The application layer is responsible for providing the source document, the extraction schema, and the audit log configuration. The model's output is a structured audit log that must be validated before it enters any downstream system of record. Treat this prompt as a stateless function: every call should be self-contained with its own [INPUT_DOCUMENT], [EXTRACTION_SCHEMA], and [AUDIT_CONFIG]. Do not rely on conversation history or prior turns for extraction fidelity.

Wiring pattern: Wrap the prompt in an API call with a structured output mode (JSON mode or function calling) that enforces the audit log schema. On response, run a completeness validator that checks: (1) every field in the extraction schema has a corresponding entry in the audit log, (2) every entry has a non-null source_span, confidence_score, and extraction_timestamp, (3) null_reason is present when the extracted value is null, and (4) confidence scores fall within [0.0, 1.0]. If validation fails, retry once with the validation errors appended to the [CONSTRAINTS] block. If the retry also fails, route the record to a human review queue with the partial output and validation failure details attached. Log every extraction attempt—including the raw prompt, model response, validation result, and retry count—to an audit event store for downstream traceability.

Model choice and tool use: This prompt works best with models that support strict JSON mode and have strong instruction-following for structured extraction tasks. If using a model without native JSON mode, add a post-processing step that parses and repairs the output against the expected schema. Do not use this prompt with models that have high hallucination rates on span-level citation tasks without adding a source grounding check: after extraction, verify that each source_span actually appears in the input document. For high-compliance use cases, consider pairing this prompt with a retrieval step that pre-segments the document into citable chunks before extraction begins. Avoid tool-calling patterns here—the prompt should produce the audit log directly rather than delegating to external tools, as tool-call latency and non-determinism complicate the audit trail.

What to avoid: Do not skip validation and trust the model's output directly. Do not use this prompt for real-time, user-facing extraction where latency must be under 500ms—the audit log generation adds token overhead. Do not run this prompt on documents larger than the model's context window without a chunking strategy that preserves cross-field consistency. If you must chunk, assemble the final audit log in the application layer by merging per-chunk logs and resolving span offsets. Finally, never expose raw confidence scores to end users without calibration context—low confidence does not mean 'wrong,' and high confidence does not guarantee correctness. Route uncertain records to review, and track confidence calibration over time using the eval harness described in the Testing and Evaluation section.