Abuse Escalation and Human Review Routing Prompt

This prompt is for trust and safety operations engineers who need a deterministic, auditable decision layer between abuse detection classifiers and the human review queue. It consumes a flagged input, its detection metadata, user history, and regulatory context, then produces a routing decision with an SLA tier and rationale. Use it when your detection pipeline generates flags that cannot all go to human review and you need a consistent, policy-aware triage step that reduces reviewer fatigue, controls cost, and meets regulatory escalation requirements.

The prompt assumes an upstream classifier has already flagged the content and provided structured metadata including the violation type, confidence score, and the raw content. It does not perform the primary abuse detection itself. The ideal user is an engineer integrating this prompt into a production moderation pipeline, not a manual reviewer. The prompt requires several inputs to function correctly: the flagged content, classifier metadata, user history summary, applicable regulatory context, and a defined SLA framework. Without these, the routing decision will be unreliable. The output is a structured routing decision—escalate for immediate human review, queue for batch investigation, or resolve with automated action—paired with a rationale and SLA tier.

Do not use this prompt as the primary abuse detector. It is a routing layer only. Do not use it to perform the actual content review; it only decides who should review it and when. Avoid deploying this prompt without a human-in-the-loop validation step for high-severity decisions, especially in regulated domains where incorrect routing can have legal consequences. Before production use, test the prompt against a golden dataset of known routing decisions to calibrate its alignment with your organization's risk tolerance and operational capacity.

This prompt is the decision engine for your human review queue. It takes a pre-flagged input—already classified as abusive, spam, or policy-violating by an upstream detector—and determines the appropriate routing action. The model must weigh the severity of the violation, the confidence of the upstream classifier, the user's history, and any applicable regulatory requirements to produce a structured routing decision. Do not use this prompt as the initial abuse detector; it assumes a violation has already been identified and requires a disposition.

textCopy
You are a trust and safety escalation router. Your task is to review a flagged user input and determine the correct routing action. You must consider the severity of the violation, the confidence of the detection system, the user's history, and any regulatory or legal obligations.

## INPUT
[FLAGGED_CONTENT]

## CONTEXT
- Detection Source: [DETECTION_SOURCE]
- Detection Confidence: [CONFIDENCE_SCORE]
- Violation Category: [VIOLATION_CATEGORY]
- User History Summary: [USER_HISTORY]
- Jurisdiction: [JURISDICTION]
- Regulatory Framework: [REGULATORY_FRAMEWORK]

## ROUTING OPTIONS
1. **IMMEDIATE_HUMAN_REVIEW**: High-severity, high-confidence violations, or any content with mandatory reporting requirements. Must be reviewed within [SLA_TIER_1] minutes.
2. **QUEUED_INVESTIGATION**: Medium-severity or ambiguous cases that require human judgment but are not time-critical. Review within [SLA_TIER_2] hours.
3. **AUTOMATED_ACTION**: Low-severity, high-confidence violations where automated enforcement (e.g., content removal, temporary mute) is appropriate. Log for audit.
4. **DISMISS**: Probable false positive based on context, user history, or low confidence. Log the dismissal reason.

## OUTPUT_SCHEMA
Return a single JSON object with the following fields:
{
  "routing_decision": "IMMEDIATE_HUMAN_REVIEW" | "QUEUED_INVESTIGATION" | "AUTOMATED_ACTION" | "DISMISS",
  "confidence": 0.0-1.0,
  "rationale": "Concise explanation referencing severity, confidence, user history, and regulatory factors.",
  "sla_tier": "[SLA_TIER_1]" | "[SLA_TIER_2]" | null,
  "recommended_action": "Specific action for the reviewer or automated system to take.",
  "evidence_summary": "Key evidence from the flagged content and context that supports the decision.",
  "regulatory_flag": true | false,
  "regulatory_notes": "If regulatory_flag is true, cite the specific obligation."
}

## CONSTRAINTS
- If [REGULATORY_FRAMEWORK] mandates reporting, you MUST route to IMMEDIATE_HUMAN_REVIEW regardless of confidence.
- If [CONFIDENCE_SCORE] is below 0.7, do not route to AUTOMATED_ACTION.
- If [USER_HISTORY] indicates a first-time offense and the violation is low-severity, prefer DISMISS or QUEUED_INVESTIGATION.
- Never output PII from [USER_HISTORY] in your rationale or evidence_summary. Use anonymized references.
- If the content involves child safety, self-harm, or terrorism, you MUST route to IMMEDIATE_HUMAN_REVIEW and set regulatory_flag to true.

To adapt this template, replace the square-bracket placeholders with values from your upstream detection pipeline and user database. The [FLAGGED_CONTENT] should be the raw text that triggered the detector. [DETECTION_SOURCE] identifies which classifier or rule engine produced the flag (e.g., spam_classifier_v2, hate_speech_model). [CONFIDENCE_SCORE] must be a float between 0.0 and 1.0. [VIOLATION_CATEGORY] should map to your internal policy taxonomy. [USER_HISTORY] is a brief summary of prior violations, account age, and trust status—never include raw PII. [JURISDICTION] and [REGULATORY_FRAMEWORK] are critical for mandatory reporting logic; common values include GDPR, DSA, NCMEC, or None. The [SLA_TIER_1] and [SLA_TIER_2] placeholders should be replaced with your organization's specific time commitments in minutes and hours.

Before deploying this prompt, validate its behavior against a golden dataset of 50-100 cases covering each routing outcome. Pay special attention to the regulatory override logic: test cases where a low-confidence, low-severity violation must still escalate due to a mandatory reporting framework. The most common production failure mode is over-escalation of borderline content, which floods your human review queue. Calibrate the confidence threshold (0.7 in the template) against your operational capacity. After each run, log the full routing decision alongside the eventual human disposition to measure alignment and drift over time.

This prompt is designed to be the final decision node in an abuse detection pipeline, not a standalone classifier. It should receive pre-enriched input that already includes a severity score from a detection model, a user history summary from an internal service, and the original flagged content. The prompt's job is to synthesize these signals into a single, auditable routing decision with an SLA tier. Do not call this prompt on every user input; invoke it only after an upstream detector has flagged content with a confidence above a configurable threshold, such as 0.7, to control cost and latency.

The implementation harness must enforce a strict contract. Before calling the LLM, validate that the [SEVERITY_SCORE] is a float between 0.0 and 1.0, the [USER_HISTORY] is a structured JSON object with fields for account_age_days, prior_violations, and current_strikes, and the [FLAGGED_CONTENT] is a non-empty string. After receiving the model's response, parse the JSON output and validate it against a schema that requires routing_decision (an enum of immediate_review, automated_action, queued_investigation), sla_tier (an enum of P1, P2, P3), and a non-empty rationale string. If parsing or schema validation fails, implement a retry loop with a maximum of two attempts, using a repair prompt that includes the raw output and the validation error. If retries are exhausted, default to immediate_review with an sla_tier of P1 as a safe fallback to ensure no high-risk content is dropped.

For high-stakes decisions, this harness must produce an immutable audit trail. Log the full input payload, the raw LLM response, the parsed decision, and any validation errors to a structured logging system before the routing action is executed. If the routing_decision is immediate_review, the system should programmatically create a ticket in your review queue, attach the audit log, and trigger a notification to the on-call trust and safety specialist. For queued_investigation, enqueue the case with the specified SLA timer. For automated_action, the harness should only proceed if the sla_tier is P3 and the action is a reversible one, like flagging an account for rate limiting; any destructive automated action must still require a human approval step, regardless of the prompt's output. This ensures the prompt advises on routing but does not become the sole authority for irreversible enforcement.