Human-in-the-Loop Gate Prompt for High-Risk Tools

This prompt is for product security teams and platform engineers who need a hard gate before executing any tool tagged as high-risk. Unlike confidence-based abstention prompts that let the model decide when it is uncertain, this prompt enforces a policy: if the tool is on the high-risk registry, the model must never execute it directly. Instead, it produces a structured approval request that a human operator must sign off on. Use this when the cost of a wrong tool call is irreversible, such as deleting production data, initiating financial transfers, sending bulk customer communications, or modifying access controls. This prompt belongs in the pre-execution layer of your tool-calling architecture, after tool selection and argument construction but before the tool call is dispatched.

The gate prompt is not a substitute for proper authorization logic in your application layer. It is a safety net that catches high-risk tool calls before they reach execution, regardless of the model's confidence. You should deploy this prompt when your tool registry includes operations that are irreversible, financially significant, or compliance-relevant. For example, a delete_customer_account tool should always trigger this gate, even if the model is 99% confident. The prompt's job is to ignore confidence and enforce policy: if the tool is on the list, stop and request approval. Do not use this prompt for low-stakes operations where the friction of human approval would degrade user experience without proportional safety benefit—a confidence-based abstention prompt is more appropriate there.

Before implementing this prompt, you must have a high-risk tool registry—a simple list or database table of tool names that require human approval. Without this registry, the prompt has no policy to enforce. You also need an approval workflow endpoint that can receive the structured approval request, queue it for a human operator, and return a decision. The prompt template is designed to output a machine-readable JSON payload that your approval system can ingest directly. Do not use this prompt if you lack the operational capacity to review and approve requests within your latency budget; a gate that blocks execution indefinitely is worse than a well-calibrated confidence threshold.

This prompt template acts as a pre-execution policy gate for any tool tagged as high-risk. It forces the model to produce a structured approval request instead of a tool call, ensuring that irreversible or sensitive operations—such as sending a payment, deleting a production database, or publishing a regulatory filing—always require human review. The template is designed to be placed in your system instructions or as a pre-tool-call policy prompt that runs before the model is allowed to emit a function call. It does not rely on the model's own confidence estimation; it is a hard rule enforced by instruction hierarchy.

textCopy
You are an AI assistant operating in a regulated production environment. You have access to tools, some of which are classified as HIGH_RISK. HIGH_RISK tools are defined in [HIGH_RISK_TOOL_LIST] and include any operation that is irreversible, financially significant, or compliance-relevant.

Before you call any HIGH_RISK tool, you MUST follow this procedure:

1. STOP. Do not emit a function call for the HIGH_RISK tool.
2. Construct a human approval request using the following structure:

APPROVAL_REQUEST:
{
  "tool_name": "[TOOL_NAME]",
  "tool_arguments": [TOOL_ARGUMENTS_AS_JSON],
  "user_request_summary": "[ONE_SENTENCE_SUMMARY_OF_USER_INTENT]",
  "risk_classification": "[RISK_LEVEL_FROM_POLICY]",
  "impact_description": "[CONCRETE_DESCRIPTION_OF_WHAT_WILL_HAPPEN]",
  "context_sources": [LIST_OF_GROUNDING_SOURCES_USED]
}

3. Present this APPROVAL_REQUEST to the user and WAIT for explicit confirmation.
4. Only if the user responds with an approved confirmation message matching [APPROVAL_CONFIRMATION_PATTERN] may you proceed to call the tool.
5. If the user denies, asks for clarification, or does not respond with the exact confirmation pattern, DO NOT call the tool. Instead, respond with: "This operation requires explicit approval. The tool was not called."

For tools NOT in [HIGH_RISK_TOOL_LIST], follow the standard tool-calling policy defined in [STANDARD_TOOL_POLICY].

If you are uncertain whether a tool is HIGH_RISK, treat it as HIGH_RISK and follow this procedure.

To adapt this template, replace the square-bracket placeholders with your specific configuration. [HIGH_RISK_TOOL_LIST] should contain the exact tool names or a reference to a dynamic registry your application injects at runtime. [APPROVAL_CONFIRMATION_PATTERN] should be a string or regex that your application can validate—for example, a signed token, a specific phrase, or a structured confirmation payload. [STANDARD_TOOL_POLICY] should reference your existing tool-use instructions for non-gated operations. The template intentionally separates the gate logic from the rest of your system prompt so that the approval rule cannot be overridden by user messages or conversational context. In production, pair this prompt with a post-generation validator that blocks any HIGH_RISK tool call emitted without a preceding APPROVAL_REQUEST in the conversation history. Log every approval request and outcome for auditability.

This prompt is not a standalone decision engine—it is a gate component that must be integrated into a broader execution harness. The prompt's output is a structured JSON decision (approval_required, risk_category, reasoning, escalation_payload) that your application layer must parse, validate, and act on. The harness is responsible for enforcing the gate: if approval_required is true, the application must block tool execution and route the escalation payload to a human review queue. If approval_required is false, the application may proceed, but only after confirming that the model's risk classification matches your organization's policy definitions for low-risk operations. Never trust the model's output alone to authorize execution—the harness is the final enforcer.

Validation and Schema Enforcement. Parse the model's JSON output and validate it against a strict schema before acting on it. Required fields include approval_required (boolean), risk_category (enum: low, medium, high, critical), reasoning (string), and escalation_payload (object containing user_request, selected_tool, tool_arguments, confidence_score, and ambiguity_summary). If the output fails schema validation, retry the prompt once with the validation error injected into the [CONSTRAINTS] or as a follow-up message. If the retry also fails, default to the safest behavior: block execution and escalate to a human with the raw model output attached. Log every validation failure for prompt improvement cycles.

Human Review Queue Integration. When approval_required is true, the harness must construct a review task from the escalation_payload. This task should include the original user request, the tool the model wanted to call, the exact arguments it would have passed, the model's confidence score, and a summary of what made the decision ambiguous or high-risk. Route this task to a review queue keyed by risk_category and selected_tool. The reviewer's decision—approve, reject, or modify arguments—must be logged alongside the model's original gate output to create an audit trail. After human approval, the harness executes the tool with the approved (or modified) arguments and records the reviewer's identity and timestamp.

Retries, Timeouts, and Degradation. The gate prompt itself should have a timeout (e.g., 5 seconds) and a retry budget (e.g., 2 attempts). If the model call fails entirely, the harness must not silently proceed. Default to escalation: create a review task with the user request and a note that the gate model was unavailable. This prevents a model outage from becoming a safety bypass. For latency-sensitive applications, consider running the gate prompt in parallel with a lower-cost classifier that can pre-filter obviously safe requests, but always let the gate prompt have veto power over the classifier's decision.

Observability and Audit Logging. Log every gate decision—approval, denial, schema failure, model timeout, and human reviewer action—as a structured event. Each event should include the prompt version, model identifier, input context hash, gate decision, and the final execution outcome (tool executed, blocked, or escalated). These logs feed into eval pipelines that measure abstention rate, false-positive escalations (unnecessary human reviews), and false-negative approvals (high-risk tools executed without review). Use these metrics to tune the [RISK_LEVEL] and [CONSTRAINTS] in the prompt template, and to adjust the risk category thresholds in your harness. Never tune the gate to reduce review volume at the expense of missing high-risk executions—false negatives in this workflow are irreversible.