Multi-Class Safety Risk Scoring Prompt Template

This prompt is for trust-and-safety engineers and ML platform builders who need a single model call to assess user input against multiple harm categories simultaneously. Instead of running separate classifiers for violence, hate speech, self-harm, sexual content, and other policy-defined categories, this template produces per-category risk scores and an aggregated severity level in one structured response. Use it when your safety architecture requires fine-grained risk decomposition for downstream routing, audit logging, or threshold-based gating. This prompt belongs in the pre-response safety screening layer, before the model generates any user-facing output.

Do not use this prompt as a refusal prompt, a safe-alternative generator, or a conversational response. It is a scoring and classification tool, not a user-facing interaction. Pair it with a Risk Threshold Gating Decision Prompt to convert these scores into allow/block/review actions. The prompt requires a defined list of harm categories with clear policy descriptions, a structured output schema for per-category scores, and a severity aggregation rule. Without these inputs, the model will invent its own categories or produce inconsistent scoring scales. You must also define what constitutes a high-risk score in your operational context before deploying this prompt into a production gating system.

Before implementing, confirm that your safety architecture actually needs per-category decomposition. If you only need a binary safe/unsafe decision, a simpler classification prompt will be faster and cheaper. If you need to map violations to specific policy clauses with citations, use a Policy Violation Probability Estimation Prompt instead. If your primary concern is detecting jailbreak attempts rather than content policy violations, use a Jailbreak Attempt Confidence Scoring Prompt. After implementing this scoring prompt, the next step is to calibrate the scores against human reviewer judgments and configure the downstream routing logic that acts on these scores.

This prompt template is designed for trust-and-safety engineers who need fine-grained risk assessment across multiple harm categories simultaneously. It produces per-category risk scores, an aggregated severity level, and an evidence-based justification. The template uses square-bracket placeholders that you replace with your production values before sending to the model. Each placeholder maps to a specific input source in your safety pipeline: the user request, your organization's safety policy definitions, the required output schema, and any additional constraints or examples that calibrate the model's behavior.

textCopy
SYSTEM: You are a safety risk classifier. Your task is to evaluate the user request against each defined harm category and produce a calibrated risk score. You must output valid JSON matching the exact schema provided. Do not refuse to classify. Do not include commentary outside the JSON.

INPUT:
[USER_REQUEST]

HARM CATEGORIES:
[HARM_CATEGORY_DEFINITIONS]

OUTPUT SCHEMA:
{
  "request_id": "string",
  "overall_severity": "none|low|medium|high|critical",
  "category_scores": [
    {
      "category": "string (must match a defined category name)",
      "score": "float between 0.0 and 1.0",
      "confidence": "float between 0.0 and 1.0",
      "rationale": "string (cite specific evidence from the request)",
      "policy_citation": "string (cite the relevant policy clause)"
    }
  ],
  "aggregation_method": "max|weighted_average|threshold_count",
  "requires_escalation": "boolean",
  "escalation_reason": "string or null"
}

CONSTRAINTS:
[CONSTRAINTS]

EXAMPLES:
[FEW_SHOT_EXAMPLES]

INSTRUCTIONS:
1. Score every harm category listed in HARM CATEGORIES, even if the score is 0.0.
2. Base scores on explicit evidence in the request, not speculation.
3. Set confidence lower when the request is ambiguous or contains mixed signals.
4. Set requires_escalation to true when overall_severity is high or critical, or when any category confidence is below [CONFIDENCE_THRESHOLD].
5. Use the aggregation_method specified in CONSTRAINTS.
6. Output only the JSON object. No markdown, no explanation.

To adapt this template for production, replace each placeholder with your operational values. [USER_REQUEST] should contain the raw user input, sanitized for PII if necessary. [HARM_CATEGORY_DEFINITIONS] should be a structured list of your organization's safety categories with clear boundaries and examples—ambiguous category definitions are the primary cause of inconsistent scoring. [CONSTRAINTS] should specify your aggregation method, confidence threshold for escalation, and any category-specific scoring rules. [FEW_SHOT_EXAMPLES] should include at least three examples covering clear violations, edge cases, and benign requests, each with the expected JSON output. Test the template against a golden dataset before deployment, and validate that the model reliably produces parseable JSON matching the schema. For high-stakes production use, add a post-processing validation layer that checks field types, enum values, and score ranges before the output reaches downstream routing logic.

This prompt is designed to sit inside a safety scoring service that receives user inputs or model outputs and returns structured risk assessments before content is delivered or actions are taken. The prompt expects a [USER_REQUEST] or [MODEL_RESPONSE] as input, a [HARM_CATEGORY_LIST] defining the classes to score, and a [RISK_THRESHOLD_CONFIG] that maps score ranges to actions (allow, warn, redirect, review, block). The output is a JSON object containing per-category scores, an aggregated severity level, and a confidence estimate. Wire this prompt as a synchronous call in your request path, but ensure it has a strict timeout and a fallback policy—if the safety scorer is unavailable, your system must decide whether to block open, block closed, or queue for async review based on your risk posture.

Validation and retry logic is critical because malformed outputs from this prompt can cause downstream routing failures. After receiving the model response, validate that: (1) every category in [HARM_CATEGORY_LIST] has a corresponding score entry, (2) all scores are numeric and within the expected range (e.g., 0.0–1.0), (3) the aggregated severity level matches one of the allowed enum values, and (4) the confidence field is present and numeric. If validation fails, retry once with the same prompt plus the validation error message appended as a [REPAIR_CONTEXT] block. If the second attempt also fails, log the failure, flag the request for human review, and apply the most conservative routing decision (typically block or review) based on the [FAILURE_FALLBACK_POLICY].

Logging and audit trail generation should capture the full prompt input, the raw model output, the validated scores, the routing decision, and the threshold configuration version. This data is essential for threshold sensitivity analysis, calibration checks, and compliance reporting. Store logs in a structured format that supports querying by harm category, score range, and decision outcome. For high-severity categories (e.g., CSAM, self-harm, violence), consider emitting a separate alert event that triggers immediate review regardless of the automated routing decision. Do not log the raw user input if it contains PII or sensitive content unless your logging pipeline is approved for that data classification.

Model choice and latency directly impact where this prompt fits in your architecture. For real-time chat or agent workflows, use a fast model (e.g., a fine-tuned classifier or a small instruction-tuned model) and keep the harm category list concise to minimize token usage and response time. For async or batch review pipelines, you can use a more capable model with a longer category list and request detailed reasoning. If you are running multiple models with different safety characteristics, use the Cross-Model Risk Threshold Alignment Prompt sibling playbook to normalize scores before they enter this scoring pipeline. Always measure tail latency (p95, p99) and set timeouts accordingly—a safety scorer that times out silently is a safety gap.

Threshold gating and A/B testing should be implemented in the application layer, not in the prompt. After this prompt returns validated scores, pass them to a separate gating function that compares each category score against the active [RISK_THRESHOLD_CONFIG] and produces a routing decision. This separation lets you deploy the scoring prompt independently from threshold changes, run A/B tests comparing different threshold configurations, and roll back thresholds without touching the prompt. Use the Risk Threshold Gating Decision Prompt sibling playbook for the gating step. Monitor refusal rates, false-positive reports, and human review queue depth after any threshold change, and set alerts for sudden shifts in score distributions using the Safety Score Distribution Monitoring Prompt.

What to avoid: Do not use this prompt as the sole safety mechanism in high-stakes domains without human review for borderline cases. Scores near threshold boundaries (e.g., 0.45–0.55 on a 0.5 block threshold) should route to review, not auto-resolve. Do not assume the model's confidence score is calibrated without running the Safety Model Confidence Evaluation Rubric Prompt against a held-out test set. Do not deploy this prompt without a regression test suite that includes known violations, edge cases, and benign inputs—use the Safety Policy Regression Confidence Comparison Prompt to detect score drift across prompt or model changes. Finally, do not log raw user inputs in plaintext if they may contain regulated or sensitive content; redact or hash before storage unless your logging infrastructure is explicitly approved for that data class.