Human-in-the-Loop Escalation Audit Prompt

This prompt is built for reliability engineers, AI ops SREs, and governance teams who need to audit why an AI system escalated a task to a human. It is not a real-time escalation decision prompt—it is a post-hoc diagnostic tool. Use it when you have a conversation trace that ended in a handoff and you need to answer three questions: which instruction rule triggered the escalation, whether the handoff summary preserved enough context for a human to act, and whether the escalation path matched the configured policy. If you are designing the escalation logic itself, start with the Human Approval and Escalation Prompts pillar instead.

The ideal input is a complete conversation trace that includes system instructions, user messages, tool outputs, and the final escalation event. The prompt expects you to supply the escalation policy rules that were supposed to govern the handoff, so it can compare actual behavior against expected behavior. Do not use this prompt for real-time decision-making, for auditing non-escalation failures, or for sessions where the escalation was triggered by an external timeout or infrastructure error rather than a model decision. Those scenarios require different diagnostic prompts from the Prompt Debugging and Failure Analysis or Operational Incident and Runbook pillars.

This is a high-stakes audit workflow. The output should be reviewed by a human before it becomes part of any compliance record. The prompt includes explicit uncertainty markers and requires evidence citations for every claim it makes about which rule fired. If the trace is incomplete or the escalation policy is ambiguous, the prompt is designed to flag those gaps rather than fabricate a clean narrative. Wire this into your post-incident review process, not your real-time escalation pipeline.

This template is designed to be dropped into an audit or review pipeline after a human-in-the-loop escalation has occurred. It instructs the model to act as an impartial reliability engineer, reconstructing the escalation event from a provided trace. The prompt forces the model to identify the specific instruction rule that triggered the handoff, evaluate whether the summary passed to the human preserved all necessary decision context, and verify that the escalation path matched the configured policy. Use this template when you need a consistent, structured post-mortem on why a task left the automated workflow.

textCopy
You are an impartial escalation auditor. Your task is to analyze a conversation trace and tool-call log to produce a structured audit report.

## INPUT
Conversation Trace:
[CONVERSATION_TRACE]

Configured Escalation Policy:
[ESCALATION_POLICY]

## AUDIT INSTRUCTIONS
1.  **Trigger Identification:** Identify the exact turn where the escalation was initiated. Quote the specific rule from [ESCALATION_POLICY] that was triggered.
2.  **Context Preservation:** Evaluate the handoff summary sent to the human reviewer. List any critical pieces of context from [CONVERSATION_TRACE] that were present before the escalation but are missing from the summary.
3.  **Path Validation:** Compare the actual escalation path (e.g., which queue, team, or individual was notified) against the path mandated by the triggered rule in [ESCALATION_POLICY]. Note any deviations.
4.  **Instruction Layer Attribution:** For the decision to escalate, identify which instruction layer (system, developer, user, tool) provided the governing rule.

## OUTPUT_SCHEMA
{
  "audit_report": {
    "escalation_turn_index": <integer>,
    "triggering_rule": "<exact text from policy>",
    "instruction_layer": "<system|developer|user|tool>",
    "context_preservation": {
      "passed": <boolean>,
      "missing_context_items": ["<string>"]
    },
    "path_validation": {
      "matched_policy": <boolean>,
      "expected_path": "<string>",
      "actual_path": "<string>"
    },
    "overall_compliance": "<compliant|non_compliant|inconclusive>"
  }
}

## CONSTRAINTS
- If the triggering rule cannot be found in [ESCALATION_POLICY], set `overall_compliance` to "inconclusive" and explain why in a new `notes` field.
- Do not invent missing context. Only list items explicitly present in [CONVERSATION_TRACE] but absent from the handoff summary.

To adapt this template, replace the placeholders with data from your production systems. [CONVERSATION_TRACE] should be a serialized log of user messages, model responses, and tool calls. [ESCALATION_POLICY] should contain the exact text of your configured escalation rules, such as 'Escalate to on-call if sentiment is negative and issue type is billing.' For high-risk domains like healthcare or finance, always route the final audit_report to a human for approval before it is logged as an official record. You can extend the OUTPUT_SCHEMA to include a severity assessment or a recommended_remediation field if the audit feeds directly into a policy improvement loop.

The Human-in-the-Loop Escalation Audit Prompt is not a real-time decision prompt—it is a post-hoc diagnostic tool designed to run after an escalation event has occurred. Wire it into your incident review pipeline, not your live escalation path. When an AI system escalates a task to a human operator, capture the full conversation trace, the active instruction version, the escalation trigger metadata, and the handoff summary as a structured log event. Queue that event for audit processing within your observability platform or a dedicated review database. The audit prompt should run asynchronously, triggered by a webhook from your escalation logger or on a scheduled batch cadence for compliance sampling.

Before calling the model, assemble the [CONVERSATION_TRACE], [ACTIVE_INSTRUCTIONS], [ESCALATION_TRIGGER], and [HANDOFF_SUMMARY] into a single structured input. Validate that each field is present and non-empty—missing traces or instruction versions should halt the audit and flag the record for manual reconstruction. Use a deterministic pre-check to confirm that the escalation trigger field matches a known rule ID in your instruction policy registry. If the trigger cannot be mapped to a declared rule, the audit should immediately classify the escalation as 'unattributed' and escalate to a human auditor without consuming model tokens on a broken input. For model choice, prefer a model with strong instruction-following and long-context handling, such as Claude 3.5 Sonnet or GPT-4o, since the prompt requires reasoning over multi-turn traces and layered instruction hierarchies.

The output must conform to a strict JSON schema. Implement a post-generation validation layer that checks for required fields: escalation_rule_id, rule_triggered_correctly, context_preserved_in_handoff, escalation_path_matches_policy, and audit_findings. If the model output fails schema validation, retry once with the validation errors appended to the prompt as [PREVIOUS_ERRORS]. If the retry also fails, log the failure, attach the raw output, and route the record to a human review queue. For high-risk domains such as healthcare or finance, every audit output should require human sign-off before the finding is committed to the compliance record. Store the final audit result alongside the original escalation event, the instruction version hash, and the auditor's approval timestamp for downstream governance reporting.

Avoid wiring this prompt into synchronous escalation flows. The audit is a diagnostic, not a gate—it should never block a human operator from receiving an escalated task. Running the audit in the critical path introduces latency and creates a false sense of safety, since the model auditing the escalation is itself fallible. Instead, treat the audit as a monitoring signal: aggregate findings over time to detect systemic escalation misconfiguration, instruction drift, or handoff quality degradation. Feed audit pass/fail rates into your operational dashboards and set alerts for sudden regressions in rule_triggered_correctly or context_preserved_in_handoff rates. The goal is to make escalation quality observable, not to add another brittle AI checkpoint before a human can act.