Confidence Threshold Handoff Prompt for Safety Classifiers

This prompt is designed for ML engineers and safety platform builders who need to operationalize confidence-based handoff decisions in production safety classifiers. When a safety classifier returns a prediction with confidence below a calibrated threshold, the system should not guess. Instead, it should route the decision to a human review queue with structured context, preserving auditability and preventing both unsafe passes and unnecessary false-positive escalations. Use this prompt when you have a classifier output with a confidence score and need a structured, policy-aware handoff decision that a downstream orchestration layer can act on.

Do not use this prompt as a standalone safety classifier. It assumes an upstream classifier has already produced a risk category and confidence score. This prompt is the decision layer between classification and action. It should be deployed behind a validation harness that checks for missing confidence scores, malformed classifier outputs, and threshold configuration drift. In high-stakes domains such as child safety, self-harm detection, or regulated professional advice, always require human review for any output this prompt flags as uncertain, and log the full decision payload for audit.

Before implementing this prompt, calibrate your confidence thresholds against a labeled evaluation set that includes both clear violations and boundary cases. Measure false-positive escalation rates (benign content sent to review) and false-negative pass rates (harmful content that slipped through). If your classifier confidence scores are not well-calibrated, this prompt will amplify those errors. Start with a conservative threshold that errs toward escalation, then tighten based on review queue capacity and observed accuracy. Never deploy this prompt without a feedback loop from human reviewers back to threshold configuration.

This prompt template is the core handoff logic you embed after your safety classifier returns a prediction. It takes the classifier's raw output—the predicted label and its confidence score—and decides whether to auto-apply the verdict or escalate to a human review queue. The template is designed to be stateless and idempotent: given the same inputs, it produces the same routing decision. Wire this directly into your safety orchestration layer, immediately after classification and before any response generation or tool execution. The primary goal is to prevent both false positives (blocking benign requests) and false negatives (allowing harmful requests through) by deferring to human judgment when the model is uncertain.

textCopy
SYSTEM:
You are a safety decision router. Your only job is to decide whether a safety classification should be applied automatically or escalated to human review. Do not generate user-facing responses. Do not explain the policy. Output only a structured JSON decision.

INPUT:
- Classifier Prediction: [PREDICTED_LABEL]
- Confidence Score: [CONFIDENCE_SCORE] (float between 0.0 and 1.0)
- User Request: [USER_REQUEST]
- Session Risk Score: [SESSION_RISK_SCORE] (optional, float between 0.0 and 1.0)

CONFIGURABLE THRESHOLDS:
- Auto-Apply Threshold: [AUTO_APPLY_THRESHOLD] (default: 0.95)
- Escalation Threshold: [ESCALATION_THRESHOLD] (default: 0.80)
- Below Escalation Threshold: auto-apply the SAFE label

DECISION RULES:
1. If [CONFIDENCE_SCORE] >= [AUTO_APPLY_THRESHOLD]: auto-apply [PREDICTED_LABEL]
2. If [CONFIDENCE_SCORE] < [ESCALATION_THRESHOLD]: auto-apply "SAFE"
3. If [ESCALATION_THRESHOLD] <= [CONFIDENCE_SCORE] < [AUTO_APPLY_THRESHOLD]: escalate to human review
4. Override: If [SESSION_RISK_SCORE] > 0.85, escalate regardless of confidence

OUTPUT_SCHEMA:
{
  "decision": "auto_apply" | "escalate" | "auto_apply_safe",
  "applied_label": string | null,
  "reasoning": "Brief explanation of which rule was triggered.",
  "escalation_context": {
    "predicted_label": string,
    "confidence": float,
    "session_risk": float | null,
    "trigger_rule": string
  } | null
}

CONSTRAINTS:
- Do not modify the decision rules.
- Do not consider the content of [USER_REQUEST] beyond what is reflected in the classifier's output.
- If [PREDICTED_LABEL] is not one of the expected policy labels, escalate.

To adapt this template, start by calibrating the two thresholds against your production data. The AUTO_APPLY_THRESHOLD should be set at the precision level where you are comfortable automating enforcement without human review—typically 0.95 or higher for high-stakes domains. The ESCALATION_THRESHOLD defines the lower bound of the uncertainty band; requests scoring below this are treated as safe to avoid flooding your review queue with low-confidence false positives. The session risk override is critical for multi-turn adversarial scenarios: a single borderline request in a high-risk session should trigger escalation even if the turn-level confidence is moderate. Before deploying, run this prompt against a labeled golden dataset and measure the precision-recall trade-off at each threshold pair. Log every escalation decision with the full escalation_context payload so your operations team can audit the router's behavior and feed corrections back into classifier retraining.

The Confidence Threshold Handoff Prompt is not a standalone decision-maker—it is a gating component inside a broader safety orchestration pipeline. In production, this prompt sits between your upstream safety classifier and your downstream routing system. The classifier emits a risk score and a confidence value; this prompt receives both, compares the confidence against your configured thresholds, and returns a structured routing decision. The implementation harness must enforce that the prompt never makes a final decision on high-severity classifications without human review, regardless of confidence. Wire this prompt as a synchronous step: the classifier output flows in, the handoff decision flows out, and the orchestrator acts on the decision immediately—no queuing, no batching, no caching that could delay a high-risk escalation.

Build the harness around three configurable thresholds stored outside the prompt: HIGH_CONFIDENCE_THRESHOLD (e.g., 0.95), LOW_CONFIDENCE_THRESHOLD (e.g., 0.70), and CRITICAL_RISK_OVERRIDE (a boolean that forces human review for specific risk categories regardless of confidence). The prompt receives these as [CONFIDENCE_THRESHOLDS] in the template. On the output side, validate the JSON response against a strict schema before acting on it: the decision field must be one of auto_allow, auto_refuse, or escalate_to_human; the confidence field must be a float between 0 and 1; the rationale must be a non-empty string. If validation fails, retry once with the same input and a stronger constraint instruction. If the retry also fails, default to escalate_to_human and log the validation failure as a high-severity incident. Never auto-allow on a parse failure.

Logging is non-negotiable in this harness. Every invocation must produce an audit record containing: the classifier's raw output, the prompt's routing decision, the thresholds used, the model and version, a trace ID linking to the user session, and a timestamp. Store these records in an append-only log that your trust-and-safety team can query. For escalate_to_human decisions, the harness must also package a review payload: the original user input, the classifier scores, the prompt's rationale, and any relevant session context. Route this payload to your review queue with a priority derived from the risk category. Avoid wiring this prompt directly to user-facing refusal messages—the harness should use the decision to select a pre-approved response template, not to generate free-text refusals that could drift under adversarial pressure. Start with a high LOW_CONFIDENCE_THRESHOLD during initial deployment, monitor the escalation rate and false-positive ratio for two weeks, then tune thresholds based on reviewer feedback and incident data.