Content Policy Violation Scoring Prompt Template

This prompt is designed for content moderation engineers and trust-and-safety teams who need to move beyond binary accept/reject decisions and implement structured, auditable violation assessments. The primary job-to-be-done is scoring user-generated content against granular, multi-category safety policies to produce per-category violation scores, an aggregate risk rating, and a recommended action with explicit policy citations. The ideal user is someone integrating this prompt into a moderation pipeline, an internal safety tool, or a human-review queue where downstream actions depend on the severity and category of a detected violation.

To use this prompt effectively, you must have defined policy categories with clear, machine-readable descriptions and severity definitions. The prompt assumes these definitions are provided as part of the [POLICY_DEFINITIONS] input. It is not a policy creation tool. Do not use this prompt for real-time blocking decisions on high-severity outputs without routing the result through a human review stage. It is also unsuitable when policy definitions are vague, contradictory, or untested, as the model will amplify that ambiguity into inconsistent scores. In such cases, invest in policy boundary definition prompts first.

The output is structured JSON, making it suitable for direct ingestion by downstream enforcement logic, logging systems, or audit databases. Before deploying, calibrate the scoring thresholds against a golden dataset of labeled examples and run inter-rater consistency evals to ensure the model's interpretation of your policies aligns with your enforcement team's expectations. The next step after reading this section is to review the prompt template and prepare your policy definitions and calibration examples.

This template is designed to be copied directly into your application or evaluation harness. Every placeholder in square brackets must be replaced with your specific policy definitions, content to evaluate, and output requirements before sending to the model. The prompt enforces structured JSON output, requires per-category scoring with evidence, and produces an aggregate risk rating that downstream systems can act on without additional parsing.

textCopy
You are a content policy enforcement classifier. Your task is to evaluate the provided content against each defined policy category and produce a structured violation assessment.

## POLICY CATEGORIES
[POLICY_CATEGORIES]

## CONTENT TO EVALUATE
[CONTENT]

## INSTRUCTIONS
1. For each policy category listed above, determine whether the content violates that category.
2. Assign a violation score from 0.0 to 1.0 where:
   - 0.0 = clearly compliant, no violation
   - 0.1-0.3 = minor concern but likely compliant
   - 0.4-0.6 = ambiguous, requires human review
   - 0.7-0.9 = likely violation
   - 1.0 = clear, unambiguous violation
3. For any category scored 0.4 or above, provide the specific excerpt from the content that triggered the concern.
4. Calculate an aggregate risk rating: LOW, MEDIUM, HIGH, or CRITICAL based on the highest individual category score.
5. Recommend an action: ALLOW, FLAG_FOR_REVIEW, BLOCK, or ESCALATE.

## OUTPUT SCHEMA
Return ONLY valid JSON matching this structure:
{
  "content_id": "[CONTENT_ID]",
  "evaluation_timestamp": "[TIMESTAMP]",
  "category_scores": [
    {
      "category": "string",
      "score": float,
      "triggering_excerpt": "string or null",
      "rationale": "string"
    }
  ],
  "aggregate_risk_rating": "LOW|MEDIUM|HIGH|CRITICAL",
  "recommended_action": "ALLOW|FLAG_FOR_REVIEW|BLOCK|ESCALATE",
  "requires_human_review": boolean,
  "review_priority": "ROUTINE|PRIORITY|URGENT or null"
}

## CONSTRAINTS
- Do not include any text outside the JSON object.
- If no categories are violated, return all scores as 0.0 and aggregate_risk_rating as LOW.
- If multiple categories are violated, include all of them in category_scores.
- triggering_excerpt must be a direct quote from the content, not a paraphrase.
- If the content is in a language you cannot evaluate confidently, set aggregate_risk_rating to HIGH, recommended_action to FLAG_FOR_REVIEW, and note the language limitation in each rationale.

To adapt this template, replace [POLICY_CATEGORIES] with your organization's specific policy definitions formatted as a numbered or bulleted list. Each category should include the policy name, a clear description of what constitutes a violation, and any edge-case guidance. Replace [CONTENT] with the user-generated text, image description, or transcript to evaluate. The [CONTENT_ID] and [TIMESTAMP] placeholders should be populated by your application layer before sending the prompt. For high-stakes moderation workflows, always route items scored 0.4 or above on any category to human review and log the full evaluation payload for audit purposes. Test this prompt against a golden dataset of known violations and benign content before deploying, and recalibrate score thresholds quarterly as your policy definitions evolve.

The Content Policy Violation Scoring prompt is designed to operate as a synchronous scoring service within a broader content moderation pipeline. It should be called after content ingestion and any necessary PII redaction, but before any automated enforcement action is taken. The prompt expects a single piece of content and a structured policy definition as input, and it returns a machine-readable scoring payload. The implementation harness must treat this prompt as a deterministic function with a probabilistic output, meaning the surrounding application code is responsible for enforcing thresholds, managing state, and deciding what happens next based on the scores returned.

To wire this into an application, wrap the prompt call in a function that accepts [CONTENT] and [POLICY_DEFINITION] as arguments. The function should construct the final prompt by injecting these variables into the template, call the model with a low temperature (0.0–0.2) to maximize scoring consistency, and parse the JSON response. Validation is critical: the application must verify that the returned JSON contains all expected policy categories, that each score is a number within the defined range, and that the aggregate_risk_rating field is present. If validation fails, implement a retry loop with a maximum of 2 attempts, appending the validation error to the next request. After 3 total failures, the content should be routed to a human review queue with a SCORING_FAILED flag rather than silently passed or blocked.

For high-volume production use, integrate this prompt into an asynchronous pipeline where content is queued, scored, and then acted upon by a downstream policy engine. The policy engine should compare per-category scores and the aggregate risk rating against configurable thresholds stored outside the prompt. This separation allows non-technical policy teams to adjust sensitivity without modifying the prompt itself. Log every scoring event with the content hash, model version, raw scores, and final action taken. This audit trail is essential for inter-rater reliability checks, threshold calibration, and defending enforcement decisions. Avoid using the model's action_recommendation field as the sole trigger for automated blocks; always cross-reference it with your own threshold rules to prevent model drift from silently changing enforcement behavior.