Policy Violation Trace Reconstruction Prompt

This prompt is for incident responders and AI safety engineers who need to reconstruct a single policy violation event from fragmented production trace data. The job-to-be-done is forensic: you have a user report, an automated alert, or a compliance flag indicating that a model generated or allowed content that violates your safety policy, and you must produce a clear, chronological narrative of what happened. The ideal user has access to raw trace data—user messages, model outputs, tool calls, retrieved context, and safety filter signals—but that data is spread across log entries, timestamps, and system events that are hard to read as a coherent story. This prompt turns scattered evidence into a structured incident timeline.

Use this prompt when you have a known violation event and need to understand the exact turn where the violation occurred, the content that triggered it, and any preceding context that contributed to the failure. It is designed for single-incident deep-dive analysis, not batch processing or aggregate trend reporting. The prompt expects you to provide the raw trace segments, the applicable policy definition, and any known metadata such as the model version and prompt version. It works best when the trace includes the full conversation history, system prompt, tool outputs, and safety filter activations—partial traces will produce lower-confidence reconstructions with explicit gaps flagged in the output.

Do not use this prompt for real-time blocking decisions, automated enforcement, or batch classification of violation severity. It is a diagnostic tool for human investigators, not a replacement for your safety filter or policy engine. The output is a narrative for human review and incident documentation; it should never directly trigger account actions, content removal, or user notifications without human sign-off. If you need to categorize refusal reasons at scale, use the Refusal Event Categorization Prompt instead. If you need to compare two similar requests to understand a policy boundary decision, use the Policy Boundary Decision Audit Prompt. This prompt is specifically for reconstructing a single violation event when the cause is not yet understood and you need a full chronological account before determining next steps.

This template is designed for incident responders who need to reconstruct a chronological narrative of a policy violation from raw, often fragmented, trace data. The prompt instructs the model to act as a forensic analyst, identifying the exact turn where the violation occurred, the content that triggered it, and the preceding context that contributed to the failure. The output is a structured report suitable for post-incident review and audit trails. Adapt the placeholder values to match your specific trace schema, policy definitions, and required output format.

textCopy
You are a forensic AI incident analyst. Your task is to reconstruct a policy violation event from a fragmented production trace. You will be given raw trace data as [TRACE_DATA], a set of active policy definitions as [POLICY_DEFINITIONS], and the specific violation type to investigate as [VIOLATION_TYPE].

Your goal is to produce a chronological narrative of the user-model interaction, highlighting the exact turn where the violation occurred, the content that triggered it, and any preceding context that contributed to the failure.

Follow these steps:
1. Parse the [TRACE_DATA] into a linear sequence of turns, including user messages, model responses, tool calls, and system events.
2. Identify the specific turn where the [VIOLATION_TYPE] occurred. Cite the exact message or event.
3. Analyze the preceding [CONTEXT_WINDOW_SIZE] turns to identify any context, such as user priming, prior model outputs, or retrieved documents, that may have contributed to the violation.
4. Cross-reference the violating content against the provided [POLICY_DEFINITIONS] to confirm the specific clause that was breached.
5. Construct a final report in the [OUTPUT_SCHEMA] format.

[TRACE_DATA]:
{trace_data}

[POLICY_DEFINITIONS]:
{policy_definitions}

[VIOLATION_TYPE]: "{violation_type}"

[CONTEXT_WINDOW_SIZE]: {context_window_size}

[OUTPUT_SCHEMA]:
{
  "incident_id": "string, a unique ID for this report",
  "violation_type": "string, the type of policy violated",
  "violation_turn_index": "integer, the zero-based index of the turn where the violation occurred",
  "violating_content": "string, the exact content that triggered the violation",
  "breached_policy_clause": "string, the specific clause from [POLICY_DEFINITIONS] that was breached",
  "contributing_context": [
    {
      "turn_index": "integer",
      "role": "string, user, model, or tool",
      "content_summary": "string, a brief summary of the contributing content",
      "contribution_explanation": "string, how this context contributed to the violation"
    }
  ],
  "chronological_narrative": "string, a markdown-formatted narrative of the full interaction leading up to and including the violation",
  "severity_assessment": "string, one of: LOW, MEDIUM, HIGH, CRITICAL",
  "recommended_actions": ["string, a list of immediate remediation steps"]
}

[CONSTRAINTS]:
- If the [TRACE_DATA] is empty or unparseable, output a JSON object with a single key "error" and a description.
- If no violation of type [VIOLATION_TYPE] is found, set "violation_turn_index" to -1 and explain in the narrative.
- Do not hallucinate any content not present in the [TRACE_DATA].
- Base all policy analysis strictly on the provided [POLICY_DEFINITIONS].

To adapt this template, replace the curly-brace variables like {trace_data} with your application's actual values at runtime. The square-bracket placeholders like [TRACE_DATA] are part of the static system prompt and should be kept as-is to guide the model's reasoning. For high-severity incidents, always route the generated report for human review before taking any automated remediation action. Validate the output JSON against the [OUTPUT_SCHEMA] in your application code and implement a retry with a stricter prompt if the model fails to produce valid JSON or sets violation_turn_index to -1 when you have external confirmation that a violation occurred.

This prompt is designed to be integrated into an incident response pipeline, not used as a one-off chat interaction. The primary integration point is a backend service that receives a trigger—such as a PagerDuty alert, a user report, or a scheduled audit job—and programmatically assembles the prompt with the required trace data. The service must first query your observability platform (e.g., LangSmith, Arize, Datadog LLM, or a custom trace store) to retrieve the full trace for the session in question, including all user messages, assistant responses, tool calls, retrieved documents, and safety filter metadata. This raw trace becomes the [TRACE_DATA] input. The [POLICY_DOCUMENT] should be fetched from your policy-as-code repository or a versioned configuration store, ensuring the exact policy version active at the time of the incident is used for the reconstruction.

The implementation harness must enforce strict validation on both the input and output. Before calling the LLM, validate that [TRACE_DATA] is a well-formed JSON array of trace events with required fields (timestamp, role, content, type). If the trace is incomplete or malformed, abort and return a TRACE_DATA_INVALID error to the incident responder. After receiving the model's output, parse the JSON response and validate it against the expected [OUTPUT_SCHEMA]. A critical check is that the violation_turn_index is a valid integer within the bounds of the trace array. If validation fails, implement a retry loop with a maximum of 2 attempts, feeding the validation error back into the prompt as additional context. If retries are exhausted, log the failure, flag the incident for manual review, and surface the raw trace and partial output to the responder. All prompt executions must be logged immutably, capturing the full prompt, the model response, the model version, and the validation result for audit trail purposes.

Model choice and tool use are important considerations. Use a model with a large context window (e.g., 200k tokens) and strong instruction-following capabilities, such as Claude 3.5 Sonnet or GPT-4o, as production traces can be verbose. Set temperature=0 to maximize determinism in the reconstruction. Do not provide the model with any tools or function-calling capabilities during this analysis; the task is purely analytical and self-contained. For high-severity incidents, route the validated output to a human-in-the-loop review queue before the reconstruction is accepted as the official incident record. The review UI should display the reconstructed narrative side-by-side with the raw trace, highlighting the violation turn and the evidence cited by the model. Only after human sign-off should the reconstruction be attached to the incident ticket and used for stakeholder communication.