Human-in-the-Loop Handoff Prompt with Confidence Metadata

This prompt is designed for a single, high-stakes job: packaging an AI-driven support conversation into a structured handoff package for a human agent. The ideal user is a support automation engineer or product developer building a system where an AI assistant handles initial customer interactions but must gracefully transfer control when it reaches the limits of its capability or authority. The required context is a complete or partial conversation transcript, the AI's internal decision log, and any unresolved customer issues. This prompt is not a general summarization tool; it is a critical infrastructure component for a human-in-the-loop (HITL) workflow.

You should use this prompt when the cost of a bad handoff is high—for example, when a human agent receiving a confusing, incomplete, or misleading summary would lead to customer churn, a compliance violation, or a safety incident. It is appropriate for support automation, technical troubleshooting, and high-touch account management. Do not use this prompt for low-stakes informational queries where a simple 'I don't know' response is sufficient, or for purely autonomous workflows where no human handoff is planned. The prompt assumes you have already made the decision to escalate; its job is to execute that handoff with maximum clarity and completeness, not to decide if escalation is necessary.

Before implementing this prompt, ensure your system can capture the required inputs: the full conversation context, a list of decision points the AI encountered, and the confidence scores associated with each. The output is a machine-readable package, not just a human-readable summary, so your application must be able to parse the structured JSON and display it appropriately in the human agent's interface. The next step is to integrate this prompt into your escalation pipeline and rigorously test it with the provided evaluation criteria to ensure handoffs are consistently complete and actionable, preventing the common failure mode of a human agent having to re-ask the customer for information already provided to the AI.

This prompt template is designed to be the final step in an AI-driven support or operational workflow. It instructs the model to stop autonomous action, synthesize the conversation state, and produce a single, structured payload that a human agent can consume immediately. The template uses square-bracket placeholders for all dynamic inputs, making it straightforward to integrate into an application harness that populates these fields before inference. The core goal is to eliminate the human agent's need to re-read the full transcript by packaging a summary, unresolved items, and explicit confidence scores for every decision point the AI encountered.

textCopy
SYSTEM:
You are a handoff specialist for an AI support system. Your only job is to compile a complete, structured handoff record when a conversation is transferred to a human agent. You do not continue the conversation. You do not apologize on behalf of the AI. You package context for a human.

USER:
Generate a handoff package using the following inputs.

CONVERSATION_TRANSCRIPT:
[CONVERSATION_TRANSCRIPT]

UNRESOLVED_ITEMS:
[UNRESOLVED_ITEMS]

AI_DECISION_LOG:
[AI_DECISION_LOG]

HANDOFF_REASON:
[HANDOFF_REASON]

OUTPUT_SCHEMA:
{
  "handoff_summary": "A 2-3 sentence executive summary of the customer's issue and what has happened so far.",
  "customer_sentiment": "Brief assessment of the customer's emotional state (e.g., frustrated, neutral, confused).",
  "unresolved_items": [
    {
      "item": "Description of the unresolved issue.",
      "ai_confidence": 0.0-1.0,
      "confidence_rationale": "Why the AI's confidence is at this level (e.g., missing information, conflicting policy, out-of-scope)."
    }
  ],
  "decisions_made": [
    {
      "decision": "Description of an action taken or conclusion reached by the AI.",
      "confidence": 0.0-1.0,
      "evidence_used": "Summary of the data or policy that informed this decision."
    }
  ],
  "handoff_instructions": "Specific, actionable questions the human agent needs to resolve. Phrase these as direct tasks.",
  "required_actions": ["List of concrete next steps for the human agent."],
  "escalation_path": "If this issue cannot be resolved by the current agent, specify the team or process to escalate to (e.g., 'Billing Tier 2', 'Engineering Escalation')."
}

CONSTRAINTS:
- Do not fabricate any information not present in the input fields.
- If the AI_DECISION_LOG is empty, the 'decisions_made' array must be empty.
- All confidence scores must be between 0.0 and 1.0.
- The 'handoff_instructions' must be a single, plain-text paragraph addressed directly to the human agent.

To adapt this template, replace the placeholders with data from your application's state. [CONVERSATION_TRANSCRIPT] should be the full, formatted chat history. [UNRESOLVED_ITEMS] is a pre-compiled list of topics the AI explicitly flagged as incomplete, which your application logic should generate before calling this prompt. [AI_DECISION_LOG] is a structured record of every tool call, classification, or routing decision the AI made, including its internal confidence scores. If your model does not natively provide logprobs, you must generate these scores using a separate confidence extraction prompt earlier in the pipeline. The [HANDOFF_REASON] is a critical field that sets the context for the human agent (e.g., 'low confidence in billing calculation', 'user requested manager'). Before deploying, validate that the generated JSON strictly conforms to the OUTPUT_SCHEMA. Any deviation, such as a missing field or a confidence score outside the 0-1 range, should trigger a retry or a fallback to a simpler, more robust template. For high-stakes domains like healthcare or finance, always append a final instruction to the human agent to independently verify all AI-generated summaries and confidence claims before taking action.

This prompt is designed to be the final step in an AI support pipeline before a conversation is transferred to a human agent. It should be invoked only when the primary support agent has exhausted its ability to resolve the issue, either because of low confidence, a policy boundary, or an explicit user request. The harness must supply the full conversation transcript, the agent's internal decision log, and any tool-call results as the [CONVERSATION_CONTEXT] and [DECISION_LOG] inputs. Do not call this prompt for every turn; it is a handoff trigger, not a continuous summarizer.

In the application layer, wrap the prompt call in a validation function that checks the output against the expected [OUTPUT_SCHEMA] before the handoff package is sent to the agent queue. The schema should require a handoff_summary, an array of unresolved_items with per-item confidence_scores, and explicit handoff_instructions. If the model returns malformed JSON or omits a required field, implement a single retry with a repair prompt that includes the raw output and the validation error. If the retry also fails, log the failure and fall back to a minimal handoff containing the raw transcript and a flag indicating an automated handoff failure. For high-risk domains like healthcare or finance, route any handoff package with an average confidence score below a configurable threshold (e.g., 0.7) to a priority review queue and require a human supervisor to acknowledge the handoff before the receiving agent sees it.

Model choice matters here. Use a model with strong instruction-following and structured output capabilities, such as GPT-4o or Claude 3.5 Sonnet, because the task requires synthesizing a long context into a precise, schema-adherent JSON object. Avoid smaller or older models that may drop fields or hallucinate resolution steps. Log the full prompt, the raw output, the validation result, and the final handoff payload for every invocation. This audit trail is essential for debugging handoff quality and for governance reviews. Next, pair this handoff prompt with an eval suite that measures context completeness, handoff clarity, and whether the human agent can resume the conversation without re-asking questions the AI already covered.