Boundary Example Selection Prompt for Ambiguous Inputs

This prompt is designed for QA engineers and ML teams who are hardening a production classifier and need to build a high-signal boundary test set. Its job is not to classify inputs, but to select the most ambiguous examples from a candidate pool—those sitting near the decision boundaries between categories where the model is most likely to fail. The ideal user has an existing classifier (or a candidate model) and a pool of unlabeled or partially labeled inputs, and they need to surface the cases that will stress-test the model's limits before release. This is a pre-release evaluation tool, not a runtime classification prompt.

Use this prompt when you have a large volume of candidate inputs and need to prioritize which ones to label and include in a regression test suite. It is particularly valuable when you suspect class overlap, when your production data contains edge cases that don't fit neatly into a single category, or when you need to measure classifier precision near decision boundaries. The prompt requires you to provide the candidate inputs, the target categories, and optionally the model's current confidence scores or predicted labels. It works best when you can supply at least 50–100 candidates per category to give the selection logic enough material to find true boundary cases. Do not use this prompt if you have fewer than 10 candidates per category, if your categories are mutually exclusive with no ambiguity, or if you need a statistically representative sample rather than a boundary-focused adversarial set.

The output is a curated list of ambiguous inputs with boundary annotations, expected behavior labels, and confidence thresholds—not a trained model or a classification result. After running this prompt, you should manually review the selected examples to confirm they genuinely represent decision-boundary cases, then integrate them into your classifier's golden test set. Avoid using this prompt as a substitute for real production traffic sampling; it selects for difficulty, not representativeness. Pair it with a separate coverage-optimized example selection prompt to ensure your test set includes both boundary cases and typical inputs.

This prompt template is designed to be wired into a script that iterates over a candidate pool of inputs and selects those most likely to cause classification errors. It forces the model to identify examples near decision boundaries—where small changes in phrasing, context, or features flip the predicted label. The output is a curated boundary test set with expected behavior labels and confidence thresholds, ready for integration into a regression testing harness or QA pipeline. Use this when you have an existing classifier (rule-based, ML, or LLM-driven) and need to surface its brittleness before production deployment.

textCopy
You are a QA engineer hardening a production classifier. Your task is to select boundary examples from a candidate pool that will expose classification weaknesses near decision boundaries.

## INPUT
Candidate pool of inputs with current model predictions and confidence scores:
[CANDIDATE_POOL]

## CLASSIFIER CONTEXT
- Task description: [CLASSIFIER_TASK]
- Target categories: [CATEGORIES]
- Current model type: [MODEL_TYPE]
- Known failure patterns: [KNOWN_FAILURES]

## SELECTION CRITERIA
1. Select inputs where the model's confidence score falls between [CONFIDENCE_LOWER_BOUND] and [CONFIDENCE_UPPER_BOUND].
2. Prioritize inputs where small perturbations (word substitution, negation, reordering) would flip the predicted label.
3. Include at least one example per category pair boundary (e.g., Category A vs Category B, Category B vs Category C).
4. Ensure coverage of [EDGE_CASE_TYPES] including null inputs, maximum-length inputs, and inputs with conflicting signals.
5. Exclude inputs that are trivially classified with confidence above [HIGH_CONFIDENCE_THRESHOLD] unless they represent a known blind spot.

## OUTPUT FORMAT
Return a JSON object with the following structure:
{
  "boundary_examples": [
    {
      "input": "<selected input text>",
      "current_prediction": "<current model label>",
      "current_confidence": <float 0-1>,
      "expected_behavior": "<correct label or 'ambiguous'>",
      "boundary_pair": ["<category_a>", "<category_b>"],
      "failure_rationale": "<why this input is near a decision boundary>",
      "perturbation_sensitivity": "<how small changes affect classification>"
    }
  ],
  "coverage_summary": {
    "total_selected": <int>,
    "category_pairs_covered": <int>,
    "edge_case_types_covered": ["<type1>", "<type2>"]
  },
  "selection_notes": "<any observations about gaps or biases in the candidate pool>"
}

## CONSTRAINTS
- Select no more than [MAX_EXAMPLES] examples.
- If the candidate pool lacks sufficient boundary examples, flag this in selection_notes rather than forcing low-quality selections.
- For regulated or high-risk domains, mark any example where the correct label is genuinely ambiguous as requiring human review.
- Do not fabricate inputs; only select from the provided candidate pool.

To adapt this template, replace each square-bracket placeholder with concrete values from your classification system. The [CANDIDATE_POOL] should contain at least 50–100 inputs with model predictions and confidence scores to give the selection algorithm enough material. Set [CONFIDENCE_LOWER_BOUND] and [CONFIDENCE_UPPER_BOUND] to bracket the uncertainty zone—typically 0.3 to 0.7 for binary classifiers, adjusted based on your model's calibration curve. Wire this prompt into a test harness that logs selections, runs them against your classifier, and compares actual behavior to expected behavior. For high-risk domains such as healthcare triage or financial fraud detection, route any example marked as genuinely ambiguous to a human reviewer before accepting it as a test case. Validate the output JSON against the schema before storing it in your test suite to prevent downstream parsing failures.

The Boundary Example Selection Prompt is designed to be called programmatically as part of a classifier hardening workflow, not as a one-off manual prompt. The typical integration pattern is a Python script or microservice that loads a candidate example pool from a database or file, injects it into the prompt template along with the target classifier's description and known failure categories, and then parses the structured output to populate a boundary test set. The prompt expects a JSON output schema, so your harness must validate that the model returned valid JSON, that all required fields are present, and that the selected examples actually exist in the input pool before accepting the result.

Validation and retry logic is critical here because the model may occasionally return malformed JSON, invent example IDs not present in the input, or produce confidence thresholds outside the 0.0-1.0 range. Implement a validation layer that checks: (1) the output is parseable JSON matching the expected schema, (2) every example_id in the output exists in the input pool, (3) confidence_threshold values are floats between 0 and 1, and (4) expected_behavior labels match the classifier's output categories. On validation failure, retry up to two times with the same prompt, appending the specific validation error message to the retry request. If all retries fail, log the failure and route to a human review queue rather than silently accepting a partial or incorrect boundary set.

Model choice and temperature settings matter for this workflow. Use a model with strong JSON-following capabilities (GPT-4o, Claude 3.5 Sonnet, or equivalent) and set temperature to 0 or very low (0.0-0.1) to maximize deterministic example selection. The prompt is performing a curation and analysis task, not creative generation, so higher temperatures introduce unnecessary variance in which examples are selected and how confidence thresholds are assigned. If you're processing large candidate pools that exceed context windows, implement a pre-filtering step that scores examples by predicted ambiguity using a cheaper model before passing the top-N candidates to this prompt for final boundary selection.

Logging and observability should capture the full prompt input, the raw model output, the validated output, and any validation failures. Store these in a versioned evaluation database so you can track how boundary test sets evolve as your classifier changes. Include metadata: the classifier version, the candidate pool source, the model used, the timestamp, and the validation pass/fail status. This audit trail is essential when debugging why a classifier's production accuracy degraded—you can check whether the boundary test set still reflects current decision boundaries or whether the example selection itself introduced blind spots.

Human review gates are recommended before the boundary test set is used to gate a classifier release. Even with validation passing, a QA engineer should spot-check the selected examples to confirm they genuinely represent ambiguous cases near the decision boundary, not obvious misclassifications or trivial examples. The prompt's output includes a selection_rationale field for each example—use this to speed up human review by highlighting why the model believes each example is boundary-relevant. If more than 20% of selected examples are rejected during human review, consider refining the prompt's boundary criteria or increasing the diversity of the candidate pool before re-running the selection.

Integration with CI/CD is the final step. Once the boundary test set is validated and reviewed, store it as a versioned artifact in your test suite. Your classifier evaluation pipeline should run against this boundary set on every pull request that modifies the classifier or its prompt, failing the build if boundary accuracy drops below a defined threshold. This closes the loop: the Boundary Example Selection Prompt produces the test set, and your CI/CD system enforces that the classifier doesn't regress on the cases where it's most likely to fail.