Self-Harm and Suicide Risk Content Triage Prompt

This prompt is designed for a single, critical job: classifying user-generated text for self-harm and suicide risk into one of four severity tiers—imminent risk, active ideation, passive mention, or no risk. The primary user is a trust and safety engineer integrating this classification into an automated content triage pipeline. The required context is the raw, unmodified user text, and the output is a structured routing decision, not a therapeutic message. This is the deterministic gate that sits before any generative AI response, human review queue, or crisis resource dispatch. It is not a chatbot, a counselor, or a content moderator making final decisions; it is a classification engine that produces an auditable, repeatable signal for your system to act on.

You should use this prompt when you need a consistent, high-recall safety net that can operate at scale without generating text for the user. It is ideal for social media platforms, gaming chat systems, educational tools, and any user-facing product where a user's text could indicate a crisis. The prompt is built to be wired into a larger application harness where its output triggers specific, pre-defined workflows: locking an account and surfacing crisis resources for 'imminent risk', or queuing an item for prioritized human review for 'active ideation'. Do not use this prompt if your goal is to generate a supportive reply to the user, to make a clinical diagnosis, or to handle low-stakes sentiment analysis. Its purpose is narrow, high-risk classification and routing.

Before you implement this, you must define the downstream actions for each severity tier. The prompt's value is only realized when its structured output is consumed by application logic. A classification of 'imminent risk' without an immediate, automated escalation to a crisis response team or a hard-coded resource intervention is a failure of the system, not the prompt. Your next step is to review the 'Implementation Harness' section to understand how to validate the model's output, log decisions for auditability, and build the human review queues that this prompt is designed to feed. Avoid the temptation to let a generative model respond freely after this classification; the entire point is to route the user away from an unqualified AI and toward appropriate resources.

This prompt is designed to be the single point of classification in a crisis triage pipeline. It forces the model to act as a deterministic classifier, not a conversational agent. The output is a strict JSON schema containing a severity level, a confidence score, and a routing decision. The prompt explicitly forbids therapeutic language, crisis resources, or any conversational text in the output to prevent the model from hallucinating a response to a person in crisis, which would be unsafe and untested.

textCopy
Classify the following user content for self-harm and suicide risk. Return only the structured JSON output. Do not generate a response to the user. Do not include therapeutic language, crisis resources, or conversational text in your output. Your only job is classification and routing.

[INPUT]

Classify the input into one of the following severity levels:
- "imminent_risk": The user has a specific plan, means, and timeframe for suicide or life-threatening self-harm.
- "active_ideation": The user expresses a desire to die or to harm themselves but lacks a specific plan or immediate intent.
- "passive_ideation": The user expresses hopelessness, a wish to not wake up, or vague thoughts of self-harm without active intent.
- "no_risk": The content does not contain any indicators of self-harm or suicide risk.

Return a JSON object with the following schema:
{
  "severity": "<imminent_risk|active_ideation|passive_ideation|no_risk>",
  "confidence": <float between 0.0 and 1.0>,
  "routing": "<escalate_to_crisis|human_review_queue|standard_processing>",
  "rationale": "<A single sentence explaining the key indicators that led to this classification.>"
}

Routing rules:
- If severity is "imminent_risk", routing must be "escalate_to_crisis".
- If severity is "active_ideation" or "passive_ideation", routing must be "human_review_queue".
- If severity is "no_risk", routing must be "standard_processing".

To adapt this template, replace the [INPUT] placeholder with the user-generated text from your application. The severity levels and routing rules are tightly coupled; changing the routing logic requires updating both the enum values and the explicit routing rules section to prevent misclassification. The rationale field is critical for auditability and for the human reviewer who will act on the human_review_queue routing. It must be a single sentence to prevent the model from generating a verbose analysis that could be misinterpreted as a direct response.

Before deploying, test this prompt against a golden dataset of edge cases: ambiguous statements like 'I'm so tired of this,' sarcasm, song lyrics discussing suicide, and help-seeking behavior. A common failure mode is the model classifying a user asking for crisis resources as active_ideation instead of no_risk. Another is a low confidence score on imminent_risk due to the model's safety training causing it to avoid the highest severity label. If your evaluation shows this, you may need to add few-shot examples to the prompt or implement a secondary validator that overrides low-confidence imminent_risk classifications based on keyword presence.

This prompt is a high-stakes classification component, not a standalone chatbot. It must be deployed inside a deterministic application harness that enforces safety constraints regardless of model behavior. The harness is responsible for input sanitization, model invocation, output validation, routing enforcement, and audit logging. Never expose this prompt directly to end users or allow the model's raw text output to reach a user without passing through the harness's validation and routing layer. The model's role is strictly to produce a structured severity classification and routing recommendation; the application owns all safety decisions, resource provision, and escalation actions.

Input Pre-processing: Before the prompt receives [USER_CONTENT], strip any prior system messages, metadata, or conversational context that could confuse the classifier. The input should be the raw user-generated content under review. If the content arrives from a multi-turn conversation, extract only the specific message or segment requiring triage. Model Selection: Use a model with strong instruction-following and low refusal rates on safety-classification tasks. Overly cautious models may refuse to classify disturbing content, which is a dangerous failure mode in this context. Test candidate models against a golden dataset of known self-harm content across severity levels to verify they produce the required structured output rather than refusing. Invocation Wrapper: Call the model with temperature=0 to maximize output determinism. Set a strict max_tokens limit appropriate for the expected JSON output size (typically 500-800 tokens). Implement a timeout and automatic retry with exponential backoff (max 2 retries) for network or transient API failures. Do not retry on content policy refusals; log those as a separate failure mode for model evaluation.

Output Validation: The harness must parse the model's response as JSON and validate it against the expected schema before any routing decision occurs. Check that severity_level is one of the defined enum values (none, low, moderate, high, imminent). Verify that requires_immediate_escalation is a boolean and is true only when severity_level is imminent or high. Reject any output where these invariants are violated and log the raw response for debugging. If validation fails, retry once with a simplified prompt that emphasizes the required output format. If the retry also fails, escalate to a human moderator queue with the original content and both raw model responses attached. Routing Logic: The application, not the model, executes the routing decision. Map severity_level to concrete actions: imminent triggers an immediate alert to crisis response personnel with the content and user context; high queues for priority human review within a defined SLA (e.g., 15 minutes); moderate queues for standard review; low and none allow automated processing to continue. The model's recommended_action field is advisory only; the harness enforces the actual routing based on severity_level and organizational policy.

Logging and Audit Trail: Log every classification event with a unique event ID, timestamp, model version, prompt version, raw input (redact PII if present in user content), validated output, routing decision, and the identity of any human reviewer involved. This audit trail is essential for post-incident review, false-alarm rate monitoring, and regulatory compliance. False-Alarm Monitoring: Implement a dashboard that tracks the rate of imminent and high classifications over time, broken down by reviewer disposition (confirmed, downgraded, false alarm). A sudden spike in false alarms may indicate prompt drift, model behavior change, or a new content pattern the prompt handles poorly. Set thresholds that trigger automatic review of the classification pipeline. Human Review Integration: When content is routed for human review, present the reviewer with the original content, the model's classification and confidence, and a simple interface to confirm or override the severity level. Log the reviewer's decision and use it to build an evaluation dataset for ongoing prompt and model quality measurement. Never allow the model's output to be the final word on a high-severity classification without human confirmation in the loop.