Hate Speech and Harassment Classification Prompt Template

This prompt is designed for trust-and-safety engineers and platform operators who need to classify user-generated text before it triggers a model response or is published. It produces a structured, multi-label classification across hate speech, harassment, and discrimination categories, each with a severity score. Use this when you need upstream detection that is more granular than a binary safe/unsafe flag and when downstream actions depend on the specific harm category and intensity. This prompt belongs in a pre-processing safety layer, not as a conversational response. It assumes the input has already passed basic profanity or spam filters and needs deeper semantic analysis.

Deploy this prompt when your platform requires audit-ready classification decisions that map to specific policy categories. The structured output enables configurable gating—you can block high-severity hate speech while flagging low-severity harassment for human review. Wire this into an API pre-processing step where the classification result determines routing: block, quarantine, escalate, or allow. The prompt is designed for single-turn classification, not multi-turn conversation analysis. For session-level risk evaluation, pair this with a multi-turn probing detection prompt that tracks cumulative risk across turns. Do not use this prompt as a standalone policy enforcement mechanism without human review for high-severity or ambiguous cases. The severity scoring provides a signal, not a final verdict.

Avoid using this prompt for real-time chat filtering where latency must stay under 100ms—the structured output and multi-category analysis add processing overhead. Do not use it to classify content in languages or dialects not represented in your evaluation dataset without additional testing for performance gaps. The prompt is not designed to detect novel, zero-day attack patterns; it classifies known harm categories. For adversarial input detection, pair this with a prompt injection or jailbreak classification prompt. Before deploying, run the prompt against a golden dataset that includes reclaimed language, in-group terminology, and dialect variations to measure false-positive rates. If your false-positive rate on benign minority-group speech exceeds your operational threshold, add few-shot examples or fine-tune rather than lowering the severity threshold.

This prompt template is designed to be pasted directly into your system prompt or safety classifier harness. It instructs the model to act as a content safety classifier, producing structured, multi-label outputs for hate speech, harassment, and discrimination. The template uses square-bracket placeholders for your specific policy definitions, output schema, and risk thresholds, making it adaptable to different platform policies without rewriting the core classification logic.

codeCopy
SYSTEM: You are a content safety classifier. Your task is to analyze the user's input and classify it according to the provided policy for hate speech, harassment, and discrimination.

[POLICY_DEFINITIONS]

Analyze the following [INPUT] against the policy definitions above.

Classify the input into one or more of the following categories only if the criteria are met. If no criteria are met, the classification should be 'none'.
- hate_speech: Content that attacks or dehumanizes people based on protected characteristics.
- harassment: Content intended to intimidate, bully, or silence a specific individual or group.
- discrimination: Content that promotes unequal treatment or exclusion of a protected group.
- none: The input does not violate any of the above policies.

For each positive classification, provide a severity score from 1 to 5, where 1 is a microaggression or veiled reference and 5 is a direct, violent attack or call to action.

[OUTPUT_SCHEMA]

[CONSTRAINTS]
- Do not classify reclaimed language used by members of the targeted group in a positive or neutral context as hate speech.
- Consider the full context of the statement; do not flag single words in isolation.
- If the input is a quote, a news report, or an educational discussion, flag it only if the user's framing endorses the harmful content.
- If you are uncertain between two severity levels, choose the lower one and set the 'low_confidence' flag to true.

[EXAMPLES]

Return your analysis in the specified JSON format only. Do not include any other text.

To adapt this template, replace the placeholders with your specific content. [POLICY_DEFINITIONS] should contain your organization's exact policy language, including definitions of protected characteristics and prohibited conduct. [OUTPUT_SCHEMA] must be a strict JSON schema defining the structure of the classifier's response, including fields for categories, severity, confidence, and evidence. [EXAMPLES] should be populated with 5-10 few-shot examples that demonstrate edge cases like reclaimed language, educational context, and ambiguous intent. After adapting, run this prompt against a golden dataset of labeled examples to calibrate severity thresholds and measure inter-annotator agreement before production deployment.

This prompt is designed as the first stage in a content safety pipeline, not a standalone classifier. In production, you should wrap it in an application harness that validates the output schema, enforces confidence thresholds, and routes low-confidence or high-severity results to human review. The prompt returns a structured JSON payload with categories, severity, confidence, and flags fields—your harness must verify that every field is present, correctly typed, and within expected ranges before any downstream system acts on the classification. A missing confidence field or a severity value outside the defined enum is a harness-level failure, not a model behavior issue, and should trigger a retry or fallback.

Implement a validation layer immediately after the model response that checks: (1) the response is valid JSON, (2) all required keys exist (categories, severity, confidence, flags, rationale), (3) severity is one of none, low, medium, high, or critical, (4) confidence is a float between 0.0 and 1.0, and (5) categories is a non-empty array of strings from the defined taxonomy. If validation fails, retry once with the same prompt plus the validation error message appended as a [CORRECTION_REQUEST]. If the second attempt also fails, log the raw response and route to a human review queue. For high-risk applications, configure a confidence threshold (e.g., 0.85) below which classifications are automatically escalated regardless of severity. This prevents the system from acting on uncertain hate speech or harassment classifications that could generate false positives and unnecessary user sanctions.

Log every classification decision with the original input, the full model response, validation status, retry count, and final routing decision. This audit trail is essential for tuning thresholds, investigating false positives, and demonstrating due diligence to trust-and-safety reviewers. When deploying, choose a model with strong instruction-following and structured output capabilities—GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro are suitable starting points. Avoid smaller or older models that may struggle with the nuanced distinctions between hate speech, reclaimed language, and legitimate critique. For dialect and reclaimed-language edge cases identified in the eval framework, maintain a separate test suite that runs weekly against production prompts to catch regression. Never let the classification result directly trigger an automated user ban or content removal without a human-in-the-loop checkpoint for high and critical severity outputs.