Verification Workflow Audit Log Prompt

This prompt is for MLOps engineers and platform developers who need a machine-readable, structured audit log of a fact-checking or claims verification pipeline run. It takes a sequence of pipeline step descriptions, timestamps, model identifiers, and intermediate outputs, and produces a single, comprehensive JSON audit trail. Use this when you must prove exactly what happened during an automated verification run for compliance, debugging, or governance reviews. This prompt assumes you have already captured step-level metadata and now need to assemble it into a coherent, traceable log. It is not a replacement for a runtime logging framework; it is the final assembly step that turns raw execution traces into an auditor-ready artifact.

The ideal user is an engineer who already has structured execution traces—such as claim extraction outputs, evidence retrieval results, matching decisions, and confidence scores—and needs to compose them into a single, ordered, and timestamped record. The prompt requires you to provide a complete sequence of pipeline steps, each with a step identifier, a description of what the step did, a start and end timestamp, the model or tool version used, the input and output payloads, and any errors or warnings encountered. Without this granular step data, the prompt cannot produce a useful audit log. You should also supply an output schema that defines the exact JSON structure you need, including fields for traceability metadata like run IDs, pipeline versions, and reviewer identities.

Do not use this prompt as a substitute for real-time logging or observability tooling. It is designed for post-hoc assembly of already-captured traces, not for live monitoring or alerting. If your pipeline steps are not already instrumented to capture structured metadata, invest in that instrumentation first. Also, avoid this prompt when you need a human-readable narrative summary for non-technical stakeholders; use the Audit Trail Narrative Generation Prompt instead. For high-risk verification domains such as healthcare claims or regulatory compliance filings, always include a human review step after log assembly to validate completeness and correctness before the audit log is submitted to external parties. The assembled log should be treated as a derived artifact that must be cross-checked against raw execution traces.

This prompt template is designed to be injected into your verification pipeline's logging step. It takes a raw execution trace—including timestamps, model versions, intermediate outputs, and tool calls—and transforms it into a structured, queryable audit log. The output is a JSON object that captures every pipeline step, its inputs, outputs, and the evidence chain, making the entire verification process traceable for compliance and debugging.

textCopy
SYSTEM: You are an audit log generator for an automated fact-checking pipeline. Your task is to convert a raw execution trace into a structured, immutable-style audit log. The log must capture every pipeline step, its inputs, outputs, and the evidence chain. Adhere strictly to the output schema. If any required field is missing from the trace, mark it as 'MISSING' and flag the log entry as incomplete. Do not invent data.

USER:
Generate a structured verification workflow audit log from the following execution trace.

[EXECUTION_TRACE]

OUTPUT_SCHEMA:
{
  "pipeline_run_id": "string",
  "generated_at": "ISO 8601 timestamp",
  "steps": [
    {
      "step_id": "string",
      "step_type": "claim_extraction | evidence_retrieval | evidence_matching | confidence_scoring | human_review_routing | other",
      "started_at": "ISO 8601 timestamp",
      "completed_at": "ISO 8601 timestamp",
      "model_version": "string",
      "input_summary": "string (truncated to 500 chars)",
      "output_summary": "string (truncated to 500 chars)",
      "full_input_hash": "string (SHA-256)",
      "full_output_hash": "string (SHA-256)",
      "status": "success | failure | warning",
      "warnings": ["string"],
      "evidence_references": ["string (source IDs or URIs)"]
    }
  ],
  "completeness_checks": {
    "total_steps_in_trace": "integer",
    "total_steps_logged": "integer",
    "missing_required_fields": ["string (field paths)"],
    "overall_status": "complete | incomplete"
  }
}

CONSTRAINTS:
- Map every event in [EXECUTION_TRACE] to a step in the output array.
- Generate SHA-256 hashes for all full_input and full_output fields. If the full text is unavailable, use the hash of the provided summary and add a warning.
- If a step's status is 'failure', include the error message in the warnings array.
- The 'completeness_checks' section must accurately compare the number of steps in the trace to the number logged.
- This log is for audit purposes; prioritize accuracy and traceability over fluency.

To adapt this template, replace the [EXECUTION_TRACE] placeholder with the raw, structured data from your pipeline runner. This trace should be a JSON or formatted log containing the sequence of events, timestamps, and model identifiers. The prompt enforces a strict output schema, making it suitable for direct ingestion into a logging database or an audit dashboard. For high-risk domains, always pair this prompt with a post-generation validation step that checks the completeness_checks.overall_status field and flags any incomplete logs for human review before they are committed to an official audit trail.

The audit log prompt is not a standalone artifact; it is a pipeline termination node that must receive structured execution context from upstream verification steps. Wire this prompt as the final stage in your verification DAG, after claim extraction, evidence retrieval, matching, and scoring have completed. The prompt expects a complete execution trace object containing step identifiers, timestamps, model versions, input/output snapshots, and verdict metadata. Do not call this prompt with raw, unstructured logs—pre-process pipeline events into the [PIPELINE_EXECUTION_TRACE] schema before invocation. The output is a structured audit log record that serves as the immutable evidence artifact for compliance review, debugging, and pipeline observability.

Integration pattern: Implement a post-execution hook in your pipeline orchestrator (e.g., Prefect, Airflow, Temporal) that collects all step outputs, model metadata, and timing information into the trace object. Pass this object to the prompt via your model gateway. Validation layer: After the model returns the audit log JSON, run a schema validator that checks for required fields (log_id, pipeline_run_id, steps[], traceability_hash, completeness_checks). If validation fails, retry once with the validation errors appended to [CONSTRAINTS]. If the retry also fails, log the failure to your observability platform and flag the pipeline run for human review. Model choice: Use a model with strong JSON-following behavior (GPT-4o, Claude 3.5 Sonnet, or equivalent) and set response_format to json_schema where the provider supports it. For high-throughput batch pipelines, consider a smaller fine-tuned model if you have sufficient training examples of correct audit log outputs.

Traceability and immutability: After successful validation, write the audit log record to an append-only store (e.g., DynamoDB with stream logging, an S3 bucket with object lock, or a compliance ledger). Generate a content hash of the log record and include it in the traceability_hash field before storage. Observability hooks: Emit metrics on audit log generation success rate, validation failure rate, and latency. Log every generation attempt—including failed validations—to your tracing system (e.g., LangSmith, Arize, Datadog LLM Observability) with the pipeline run ID as the trace correlation key. Human review trigger: If the completeness_checks array in the output contains any FAIL entries, or if the verification_gaps field is non-empty, route the entire pipeline run to a human review queue with the audit log attached. Do not silently accept incomplete audit trails—compliance value depends on completeness. What to avoid: Do not use this prompt for real-time, latency-sensitive verification flows; the full trace assembly and audit log generation add 2-5 seconds of overhead. Do not skip the schema validation step—model-generated JSON can drift under load or with prompt changes. Do not treat the audit log as the source of truth for pipeline state; it is a derived record that must match the actual execution, not replace it.