Child Safety Query Over-Refusal Reduction Prompt for Educators

Educational platforms face a persistent operational problem: generic safety classifiers block legitimate teacher and administrator queries about safeguarding, mandated reporting, and child protection policy because the surface vocabulary—terms like 'child,' 'abuse,' 'grooming,' 'reporting,' and 'neglect'—matches disallowed content patterns. This prompt playbook addresses that false-positive problem with a role-and-context classifier that evaluates three dimensions before deciding to respond or refuse: the user's professional role, the query's educational or administrative context, and the instructional intent behind the request. The classifier is designed to sit in the request pipeline after authentication and before the main model response, routing permitted queries through and escalating genuinely ambiguous cases for human review.

Deploy this prompt when your platform serves verified educators, school administrators, counselors, or safeguarding leads who need to discuss child safety topics as part of their professional duties. Concrete use cases include: a teacher preparing a mandated reporter training module who asks about indicators of neglect; a school safeguarding lead drafting policy language about online grooming risks; an administrator querying how to document and escalate a disclosure from a student; or a curriculum developer creating age-appropriate protective behaviors content. In each case, the query contains terms that would trigger a naive safety classifier, but the professional context and instructional intent make the query legitimate and necessary. The classifier uses structured output—a decision label, confidence score, and evidence markers—so your application can log decisions, track over-refusal rates, and audit borderline cases.

Do not use this prompt as a standalone content moderation system for unauthenticated users, public forums, or consumer chat products where requester role cannot be verified. The classifier depends on role context to make its distinction; without a verified educator or administrator identity, the safety calculus changes. Do not use this prompt to bypass child safety protections for non-professional queries, and do not treat a 'permit' classification as a guarantee that the subsequent model response will be safe—this is a pre-response gate, not a replacement for output monitoring. After implementing this classifier, instrument your pipeline to log every classification decision with the confidence score, track the false-positive rate on known-good educator queries, and set up a human review queue for any query classified as 'UNCERTAIN' or scoring below your configured confidence threshold. Run the provided test cases—mandated reporter training queries, safeguarding policy drafting, curriculum development scenarios—through your full pipeline before production deployment to verify that the classifier plus your main model produces safe, useful responses without over-refusal.

This prompt template is designed to be deployed as a system-level classifier that runs before your main conversational model. It evaluates whether a query from an authenticated educator falls within permitted professional discussion boundaries or should be refused. The template uses square-bracket placeholders that you must populate with your organization's specific policies, role definitions, and risk thresholds before deployment. The classifier returns structured JSON rather than free text, making it straightforward to wire into application logic for routing, logging, or escalation decisions.

textCopy
You are a child safety content classifier for an educational technology platform. Your role is to distinguish legitimate professional educator discussions about child safety from prohibited content that violates platform policies.

## CLASSIFICATION RULES

You must classify the user query into exactly one of these categories:

1. **PERMITTED_EDUCATOR_DISCUSSION**: The query involves professional educator activities such as:
   - Mandated reporter training and procedures
   - Safeguarding policy development and review
   - Curriculum development for child protection education
   - Professional development on recognizing abuse indicators
   - Discussing hypothetical scenarios for training purposes
   - Research and academic discussion of child safety topics
   - Compliance with [ORGANIZATION_SAFEGUARDING_POLICY_NAME]

2. **REFUSE_AND_REDIRECT**: The query requests or appears to request:
   - Descriptions or depictions of child exploitation or abuse
   - Instructions for harming or exploiting children
   - Content that sexualizes minors in any context
   - Methods to circumvent child safety protections
   - Personal opinions on specific real-world child abuse cases
   - Any content prohibited by [PLATFORM_CONTENT_POLICY_DOCUMENT]

3. **ESCALATE_FOR_REVIEW**: The query falls into an ambiguous boundary area where:
   - The educational context is unclear or unverifiable
   - The query contains concerning language patterns but plausible educational intent
   - The topic is permitted but the framing raises safety concerns
   - The user's role or authorization cannot be confirmed

## USER CONTEXT

The user has been authenticated with the following attributes:
- Role: [USER_ROLE]
- Institution: [USER_INSTITUTION]
- Authorization Level: [AUTHORIZATION_LEVEL]
- Session Risk Score: [SESSION_RISK_SCORE]

## INPUT QUERY
[USER_QUERY]

## OUTPUT SCHEMA

Return ONLY valid JSON with this exact structure:
{
  "classification": "PERMITTED_EDUCATOR_DISCUSSION" | "REFUSE_AND_REDIRECT" | "ESCALATE_FOR_REVIEW",
  "confidence": 0.0-1.0,
  "rationale": "Brief explanation of the classification decision",
  "risk_factors": ["List of specific risk indicators identified, empty array if none"],
  "recommended_action": "Specific next step for the application layer",
  "safe_alternative": "If REFUSE_AND_REDIRECT, provide a constructive redirection. Otherwise null."
}

## CONSTRAINTS
- Do not generate the response to the user query itself. Only classify.
- Do not include content that would violate child safety policies in your rationale.
- If the query contains [TRIGGER_PATTERNS], flag for escalation regardless of apparent intent.
- Default to ESCALATE_FOR_REVIEW when confidence is below [CONFIDENCE_THRESHOLD].
- Never assume educational intent without sufficient contextual evidence.

To adapt this template for your platform, start by replacing the bracketed placeholders with your actual policy documents, role definitions, and risk thresholds. The [TRIGGER_PATTERNS] placeholder should contain specific language patterns that your safety team has identified as high-risk even in educational contexts. Set [CONFIDENCE_THRESHOLD] based on your organization's risk tolerance—a lower threshold (e.g., 0.7) means more queries escalate for human review, while a higher threshold (e.g., 0.9) means more queries are auto-classified. The [SESSION_RISK_SCORE] should be populated by your upstream session monitoring system, not by this classifier. After deploying this prompt, you must run the eval suite described in the Testing and Evaluation section against your specific model version, as classification behavior varies significantly across models and even model versions. Never deploy this classifier without first testing against your golden dataset of known educator queries, edge cases, and adversarial examples.

This prompt is designed to sit between a user-facing chat or search interface and your content safety filter. It acts as a pre-classification layer that determines whether a query from an educator is a legitimate professional discussion about child safety or a policy violation. The implementation harness must treat this prompt as a decision gate, not a final answer. Its output should control routing: PERMIT sends the query to the main model for a substantive response, while REFUSE or REVIEW triggers alternative handling. Because the domain involves child safety, every decision must be auditable. Log the full prompt, the model's classification, the confidence score, and the final routing action for every request.

Integration pattern: Deploy this prompt as a lightweight, fast-inference call using a model optimized for classification (e.g., a smaller, cheaper model like GPT-4o-mini, Claude Haiku, or a fine-tuned classifier). The input variables are [QUERY] (the educator's raw text), [USER_ROLE] (verified from your auth system, e.g., teacher, administrator, counselor), and [INSTITUTION_CONTEXT] (e.g., K-12 public school, university). The output must be parsed as strict JSON with fields: classification (PERMIT, REFUSE, REVIEW), confidence (0.0–1.0), rationale (a short string), and policy_tags (an array of applicable policy categories). Implement a validation layer that rejects any response that fails to parse as valid JSON or contains a classification outside the allowed enum. On parse failure, retry once with a simplified prompt. On a second failure, default to REVIEW and queue for human evaluation.

Thresholds and routing: Define a confidence threshold in your application config (start at 0.85 for PERMIT). If classification is PERMIT and confidence is above the threshold, route the query to your main instructional model. If classification is REFUSE with high confidence, return a standard educational-safety refusal message. All REVIEW classifications and any classification below the confidence threshold should enter a human review queue. This queue is critical for calibration: review decisions feed back into your eval dataset to tune the prompt and threshold over time. For high-risk contexts, consider requiring a second classifier call with a different model before permitting a query. Never bypass human review for queries tagged with child_sexual_abuse_material or grooming policy tags, regardless of confidence score.

Observability and eval: Instrument every decision point. Log the raw query, user role, institution context, model response, parsed classification, confidence, and final routing action. Use these logs to build a safety regression suite: extract queries where the classifier and human reviewer disagreed, and add them to your golden test set. Run this test set against every prompt or model change before deployment. Monitor the REVIEW queue rate and the false-positive rate (queries flagged for review that were ultimately permitted) weekly. A rising false-positive rate signals over-refusal drift, which this prompt is explicitly designed to reduce. If the REVIEW queue exceeds 5% of total queries, investigate whether the prompt needs recalibration or the model has shifted behavior.