Policy Boundary Training Example Generation Prompt Template

This prompt is designed for safety ML engineers who need to build or augment a fine-tuning dataset that teaches a model to recognize the precise boundaries of a safety policy. The primary job-to-be-done is generating a diverse, labeled set of examples that includes policy-compliant requests, clear policy-violating requests, and the ambiguous edge cases that sit between them. You should use this prompt when you have a finalized, written policy document and a target taxonomy of violation categories, and you need to produce training data that calibrates a safety classifier or instruction-tuned model. The output is not intended for real-time enforcement or user-facing refusal generation; it is a dataset creation tool that assumes downstream human review and quality assurance before any training run.

The ideal user is an ML engineer or safety researcher who understands the target policy and can validate the generated examples against it. Required context includes the full policy text, a defined list of violation categories with severity levels, and explicit instructions on the desired balance between compliant and violating examples. You must also specify the output format, such as JSONL with fields for prompt, label, violation_category, and confidence. Do not use this prompt if your policy is still in draft, if you lack a clear taxonomy, or if you need real-time refusal logic. The prompt assumes you will manually review and potentially relabel a sample of the generated data to correct for model biases or misinterpretations of the policy text.

Before running this prompt at scale, validate a small batch of outputs against your policy document. Check for label accuracy, edge-case coverage, and representation across all violation categories. Common failure modes include the model hallucinating policy rules that do not exist, over-indexing on obvious violations while missing subtle ones, and generating examples that are syntactically varied but semantically identical. To mitigate these risks, implement a human review loop for at least 20% of generated examples and use automated checks for duplicate semantic meaning. After generation, run a coverage analysis to identify which policy sections or violation categories are underrepresented, then re-prompt with targeted instructions to fill those gaps.

This prompt template is designed for safety ML engineers who need to generate balanced training datasets for policy boundary recognition. It instructs the model to produce compliant examples, policy-violating examples, edge cases, and confidence annotations, all while enforcing representation checks across categories. The output is structured for direct ingestion into fine-tuning pipelines or evaluation harnesses.

textCopy
You are a safety training data generator. Your task is to produce labeled examples that teach a model to recognize the boundaries of a defined safety policy.

## POLICY DEFINITION
[POLICY_TEXT]

## OUTPUT SCHEMA
Return a JSON array of training examples. Each example must follow this schema:
{
  "id": "string",
  "user_input": "string",
  "category": "compliant" | "policy_violation" | "edge_case",
  "violated_rule": "string | null",
  "confidence": 0.0-1.0,
  "rationale": "string",
  "expected_model_behavior": "comply" | "refuse" | "redirect" | "clarify"
}

## GENERATION CONSTRAINTS
- Generate exactly [TOTAL_EXAMPLES] examples.
- Distribution: [COMPLIANT_PCT]% compliant, [VIOLATION_PCT]% policy_violation, [EDGE_CASE_PCT]% edge_case.
- Ensure representation across all policy clauses in [POLICY_TEXT].
- Edge cases must target genuine ambiguity, not obvious violations.
- Confidence scores: 0.9+ for clear cases, 0.5-0.7 for genuine ambiguity.
- Vary user_input phrasing, length, and persona (e.g., direct request, hypothetical, academic framing, non-native speaker).
- Include at least [MIN_ADVERSARIAL] adversarial examples that attempt to circumvent the policy through rephrasing, role-play, or hypothetical framing.

## BALANCE AND REPRESENTATION CHECKS
Before outputting, verify:
1. Each major clause in [POLICY_TEXT] is covered by at least one violation example.
2. Compliant examples include near-boundary cases that are close to violating but do not cross the line.
3. No demographic or identity-based skew in example subjects unless required by policy scope.
4. Edge cases are evenly distributed across policy clauses.

## OUTPUT FORMAT
Return only the JSON array. No additional text.

Adaptation guidance: Replace [POLICY_TEXT] with the exact policy document or rule set. Tune [TOTAL_EXAMPLES] based on your dataset size requirements—start with 100 for initial calibration, then scale to 1,000+ for training. Adjust the distribution percentages based on your model's current failure modes: if over-refusal is the problem, increase [COMPLIANT_PCT]; if under-refusal dominates, increase [VIOLATION_PCT]. Set [MIN_ADVERSARIAL] to at least 10% of total examples for robust boundary training. For production use, run this prompt through a validator that checks output schema compliance, distribution adherence, and clause coverage before accepting the generated dataset.

This prompt is a data factory, not a chatbot. It should be called programmatically in a batch job, not exposed to end users. The harness must manage input variables, validate the generated JSON schema, enforce diversity quotas, and log every generation for auditability. Treat each call as a single unit of work that produces one or more labeled training examples. The harness is responsible for retries, output validation, and aggregating results into a dataset that downstream fine-tuning or evaluation processes can consume directly.

Pipeline Integration: Wrap the prompt in an async worker that reads policy definitions and seed examples from a configuration store. For each policy category, generate batches of examples with controlled variation: compliant, violating, edge-case, and adversarial. Use the [DIVERSITY_CONSTRAINTS] placeholder to inject category-specific quotas (e.g., 'at least 20% edge cases, no more than 5% near-duplicates'). After each generation, validate the output against the [OUTPUT_SCHEMA] using a JSON Schema validator. If validation fails, retry up to three times with the error message appended to [CONSTRAINTS]. Log all failures for manual review. Model Choice: Use a model with strong instruction-following and structured output capabilities (e.g., GPT-4o, Claude 3.5 Sonnet). Avoid smaller models that drift toward generic examples or ignore the schema. Rate Limiting: Implement exponential backoff with jitter to handle API rate limits. Cost Control: Set a max token budget per generation and abort batches that exceed it.

Quality Gates: Before examples enter the training dataset, run them through a secondary evaluation prompt (see the LLM Judge pillar) that scores each example on policy accuracy, label correctness, and diversity. Flag examples where the model's confidence annotation contradicts the evaluator's score. Maintain a human review queue for all edge cases and any example where the evaluator score falls below a configurable threshold. Output Storage: Write validated examples to a versioned dataset with metadata: prompt version, model version, generation timestamp, policy category, and evaluator scores. This traceability is essential for debugging policy drift and for audit evidence when regulators ask how safety training data was produced. What to Avoid: Do not use this prompt in a synchronous user-facing flow. Do not skip output validation. Do not assume one generation is sufficient for any policy category—batch diversity requires multiple calls with varied seeds and constraints.