Negative Example Selection Strategy Prompt

This prompt is for prompt designers and AI engineers who need to curate high-signal negative examples for few-shot learning. Instead of guessing which counterexamples will teach the model boundaries, this prompt analyzes a candidate pool of negative examples and selects the subset that maximizes learning signal. Use it when you have a collection of potential negative examples but need to identify near-miss cases, common failure modes, and boundary violations that prevent over-generalization and reduce false positives in production.

The ideal user already has a labeled dataset or log of model errors and needs a systematic selection strategy. This prompt is not for generating new negative examples from scratch. It assumes you have existing data—production error logs, validation failures, human-annotated rejections, or synthetic edge cases—and need to filter for the examples that will actually shift model behavior. The prompt evaluates each candidate against criteria like boundary proximity, representativeness of failure clusters, and contrast with positive examples. It outputs a ranked selection with justification, coverage analysis, and signal-to-noise evaluation.

Do not use this prompt when you lack a candidate pool or need to create negative examples from scratch. It is also inappropriate when your few-shot budget is extremely tight (1-2 examples) and you need manual expert curation rather than algorithmic selection. For high-risk domains like healthcare, legal, or safety-critical systems, always pair the selected examples with human review before deployment. The prompt identifies high-signal candidates, but domain experts must validate that the selected counterexamples teach the correct boundaries without introducing unintended refusal patterns or bias.

This prompt template helps you curate a set of negative examples that maximize learning signal while minimizing token waste. Unlike random sampling, this strategy identifies near-miss cases, common failure modes, and boundary violations that teach the model what not to do. Replace each square-bracket placeholder with your specific data before sending to the model. The output is a ranked list of candidate negative examples with coverage analysis and signal-to-noise evaluation, ready for inclusion in your few-shot prompt or evaluation harness.

textCopy
You are an expert example curator for AI prompt engineering. Your task is to select high-signal negative examples from a pool of candidates. A high-signal negative example is one that teaches the model a clear boundary: it is close to a valid input or output but violates a specific constraint, rule, or expected behavior.

## INPUT DATA
- Positive examples (what the model SHOULD do):
[POSITIVE_EXAMPLES]

- Candidate negative examples (potential counterexamples to evaluate):
[CANDIDATE_NEGATIVE_EXAMPLES]

- Task description and constraints:
[TASK_DESCRIPTION]

- Target output schema or format:
[OUTPUT_SCHEMA]

- Maximum number of negative examples to select:
[MAX_COUNT]

- Risk tolerance (low, medium, high):
[RISK_LEVEL]

## SELECTION CRITERIA
For each candidate negative example, evaluate:
1. **Near-miss score** (1-5): How close is this to a valid case? Near-misses teach sharper boundaries.
2. **Failure mode coverage**: Which failure category does this represent? (e.g., format violation, hallucination, over-refusal, constraint violation, ambiguity mishandling, domain boundary violation)
3. **Signal-to-noise ratio** (1-5): Does this example clearly demonstrate ONE specific mistake, or is it confusing?
4. **Diversity contribution**: Does this example cover a failure mode not already covered by other selected examples?
5. **Production relevance**: How likely is this failure mode to occur in real inputs?

## OUTPUT FORMAT
Return a JSON object with the following structure:
{
  "selected_negative_examples": [
    {
      "index": 0,
      "example": "<the negative example text>",
      "near_miss_score": 4,
      "failure_mode": "format_violation",
      "signal_to_noise": 5,
      "rationale": "This example uses valid content but nests JSON incorrectly, a common production error."
    }
  ],
  "coverage_summary": {
    "failure_modes_covered": ["format_violation", "constraint_violation"],
    "failure_modes_missed": ["ambiguity_mishandling"],
    "diversity_score": 4
  },
  "excluded_examples": [
    {
      "index": 2,
      "reason": "Redundant with selected example 0; same failure mode with lower signal-to-noise."
    }
  ],
  "recommendations": "Add at least one ambiguity mishandling example before deployment. Current coverage is strong on format but weak on semantic boundary cases."
}

## CONSTRAINTS
- Do not select examples that are confusing or teach multiple failure modes simultaneously.
- Prioritize examples that are close to valid cases (near-misses) over obviously invalid inputs.
- Ensure coverage across at least [MIN_FAILURE_MODES] distinct failure modes.
- If [RISK_LEVEL] is high, flag any uncovered failure modes as critical gaps.
- Return ONLY valid JSON. No markdown fences, no commentary outside the JSON object.

After copying this template, replace each placeholder with your specific data. Critical adaptation steps: (1) Populate [POSITIVE_EXAMPLES] with 3-5 representative correct outputs that establish the desired behavior baseline. (2) Populate [CANDIDATE_NEGATIVE_EXAMPLES] with at least 10-20 candidates drawn from production logs, edge-case test suites, or synthetic generation. (3) Set [MAX_COUNT] based on your token budget—typically 3-7 negative examples for few-shot prompts. (4) Set [RISK_LEVEL] to high for safety-critical or compliance workflows, which will trigger stricter coverage requirements. (5) Validate the model's JSON output against the schema before injecting selected examples into your production prompt. For high-risk deployments, have a human reviewer confirm that selected examples do not inadvertently teach the model to refuse legitimate requests.

This prompt is designed to be called programmatically, not used in a one-off chat interface. It expects a structured input payload containing candidate examples, a target task description, and a set of evaluation criteria. The primary integration point is within a CI/CD pipeline for prompt evaluation or a data preparation workflow for fine-tuning. The harness should treat this prompt as a scoring and selection function: it receives a batch of candidates and returns a ranked, annotated subset of high-signal negative examples. Because the output is a structured JSON object, the integration layer must validate the schema before downstream consumption.

Wire the prompt into an application by building a thin service wrapper that: (1) loads candidate examples from a database, vector store, or static test suite; (2) constructs the prompt payload with the required [TASK_DESCRIPTION], [CANDIDATE_EXAMPLES], and [EVALUATION_CRITERIA] placeholders; (3) calls the model with response_format set to json_object and a low temperature (0.0–0.2) to enforce deterministic selection; (4) validates the returned JSON against the expected [OUTPUT_SCHEMA] using a library like Pydantic or Zod; and (5) on validation failure, retries once with the validation error message appended to the prompt as a correction hint. Log every selection run with the model version, input hash, selected example IDs, and coverage metrics. For high-stakes safety or compliance workflows, route the selected negative examples to a human review queue before they are injected into a production prompt or training dataset.

The most common production failure mode is schema non-conformance in the output, especially when the model returns a truncated JSON array or omits the required rationale field. Mitigate this by setting a sufficiently high max_tokens value and implementing a strict post-processing validator that rejects malformed responses. A second failure mode is selection homogeneity, where the model picks negative examples that are all variations of the same failure pattern, reducing coverage. Address this by including a diversity_weight parameter in the [EVALUATION_CRITERIA] and post-hoc checking the selected set's coverage against your known failure taxonomy. Do not use this prompt to select negative examples for safety-critical refusal boundaries without human review of the final example set.