Example Validation Prompt Before Insertion

This prompt is designed for prompt library maintainers and AI operations teams who manage collections of few-shot examples used in production prompts. Before an example is inserted into a live prompt template, it must be validated for correctness, format compliance, and alignment with the current system instructions. This validation prompt acts as a preflight check that catches silent example degradation, instruction mismatch, format drift, and edge-case gaps before they corrupt model behavior in production.

Use this prompt when you are adding new examples to a library, rotating stale examples, reviewing examples after a prompt version change, or auditing an existing example set for quality. The output is a structured validation report with pass/fail flags, specific issues, and actionable fix recommendations. This prompt belongs in the CI/CD pipeline for prompt changes, not as a one-off manual review tool. Wire it into your pull request checks so that any change to an example file triggers validation against the current system prompt and output schema before merge.

Do not use this prompt for runtime example selection, embedding-based retrieval, or dynamic few-shot assembly. Those workflows require different prompts focused on relevance ranking, context budget allocation, or semantic similarity. This prompt is strictly a pre-insertion quality gate. If your examples are already in production and you suspect drift, use the Example Drift Detection Prompt Template instead. For teams operating in regulated domains, always pair automated validation with human review of any example that touches compliance-sensitive instructions or refusal boundaries.

This prompt template validates candidate few-shot examples against your current task instructions, output schema, and quality standards before they are inserted into production prompts. It produces a structured validation report that flags correctness issues, format violations, instruction misalignment, and silent degradation risks. Use this as a preflight check whenever examples are added, rotated, or regenerated in your prompt library.

textCopy
You are an example quality validator for a production prompt library. Your job is to inspect candidate few-shot examples and determine whether they are safe to insert into a prompt that teaches a model to perform the following task.

TASK INSTRUCTIONS:
[INSTRUCTIONS]

OUTPUT SCHEMA (if applicable):
[OUTPUT_SCHEMA]

CONSTRAINTS AND POLICIES:
[CONSTRAINTS]

CANDIDATE EXAMPLES TO VALIDATE:
[EXAMPLES]

For each candidate example, produce a validation entry with these fields:
- example_id: the identifier from the candidate list
- passes: true if the example is safe to insert, false otherwise
- correctness_score: 1-5 rating of whether the example output is factually correct given the input
- format_compliance: true if the example output matches the required output schema or format conventions
- instruction_alignment: 1-5 rating of how well the example demonstrates the behavior described in TASK INSTRUCTIONS
- constraint_violations: list any constraints or policies the example violates, empty array if none
- degradation_risk: "low", "medium", or "high" risk that this example will silently degrade model behavior over time
- issues: human-readable description of any problems found, empty string if none
- recommendation: "insert", "fix", or "reject"

Before scoring, reason step by step:
1. Check whether the example input is well-formed and representative of real inputs.
2. Verify the example output against the task instructions and output schema.
3. Check for contradiction with constraints or policies.
4. Assess whether the example teaches a pattern that could become stale or harmful.
5. Assign scores and recommendation.

Return a JSON object with a single key "validations" containing an array of validation entries, one per candidate example.

Adapt this template by replacing [INSTRUCTIONS] with the exact system or task instructions the production prompt uses. Replace [OUTPUT_SCHEMA] with your expected JSON schema, field descriptions, or format rules. Replace [CONSTRAINTS] with any refusal policies, tone rules, or domain-specific boundaries. Replace [EXAMPLES] with the candidate few-shot examples you want to validate, each with a unique example_id. Run this validation before inserting examples into production prompts, and route any example scored recommendation: "fix" to a human reviewer. For high-risk domains, require human sign-off on all examples scored below correctness_score: 5 or flagged with degradation_risk: "high".

The example validation prompt is not a one-off review tool—it is a pre-insertion gate that should run every time an example is added, modified, or rotated in your prompt library. The harness wraps the validation prompt in a lightweight application layer that handles input assembly, output parsing, retry logic, and logging. The goal is to prevent a bad example from silently degrading production prompt behavior. Without this harness, example quality drifts over time as instructions evolve, schemas change, and edge cases accumulate.

Wire the prompt into a validation step that runs before any example reaches a production prompt template. The harness should: (1) assemble the current system instructions, output schema, and the candidate example into the prompt template's [INSTRUCTIONS], [OUTPUT_SCHEMA], and [CANDIDATE_EXAMPLE] placeholders; (2) call the model with a low temperature (0.0–0.2) to get deterministic validation results; (3) parse the structured validation report, checking the pass boolean and scanning for CRITICAL severity findings; (4) block insertion if pass is false or any critical finding exists; (5) log the full validation report with example ID, timestamp, and model version for auditability. For high-reliability systems, run the validation prompt against two different models (e.g., GPT-4o and Claude Sonnet) and require both to pass—this catches model-specific blind spots where one model accepts an example that confuses another.

After the validation gate, store the approved example with its validation report hash in your example registry. If you later update system instructions or output schemas, re-validate all existing examples against the new instructions. A previously valid example can become misleading when instructions change. Set up a scheduled re-validation job (daily or per-deployment) that flags stale examples before they cause production regressions. Avoid the temptation to skip validation for 'obviously correct' examples—the most damaging example drift often comes from small, incremental changes that accumulate across many library updates.