Edge-Case Example Selection Prompt for Rare Inputs

This prompt is for production teams who have already built a working prompt for a classification, extraction, or generation task but are seeing failures on rare, ambiguous, or malformed inputs. The job-to-be-done is hardening: you need a curated set of few-shot examples that specifically teach the model how to handle long-tail edge cases without degrading performance on common inputs. The ideal user is an AI engineer or prompt operator who has access to production logs, failure traces, or a labeled set of tricky inputs, and who needs to convert those failures into reusable teaching examples. You should not use this prompt during initial prototyping when you don't yet have a stable base prompt or a clear picture of failure modes—start with a standard few-shot selection prompt first.

This prompt requires several concrete inputs to be useful. You must provide a set of candidate edge-case inputs (drawn from production logs, error traces, or manual QA), the current system prompt or task instructions that the model already follows, and a description of the desired output behavior for each edge case. The prompt works by analyzing each candidate input against the existing instructions, identifying where the model is likely to fail, and generating a compact example that demonstrates the correct handling pattern. It also flags examples that might conflict with existing instructions or introduce new failure modes. Run this prompt before you insert examples into your production prompt, and always validate the output against a held-out set of edge cases to confirm the examples actually improve behavior.

After generating edge-case examples, you must test them in your actual prompt harness with eval checks for coverage gaps, example staleness, and unintended regression on common inputs. Do not treat this as a one-time fix—edge cases evolve as your product changes, and examples that were once rare can become common. Schedule periodic re-runs of this prompt against fresh production logs, and maintain a versioned example library so you can roll back if new examples cause unexpected behavior. If your task involves regulated or high-risk decisions, require human review of generated examples before they enter production prompts.

This template is designed to be copied directly into your prompt engineering workflow. It instructs a model to act as an adversarial example generator, producing a curated set of few-shot demonstrations that target the specific failure modes, rare inputs, and boundary conditions most likely to break your production prompt. The output is a structured set of examples you can insert into your system prompt or dynamic few-shot assembly logic. Before using this template, you must have a clear understanding of your production prompt's intended behavior, its known failure modes, and the input schema it expects. This prompt is not a substitute for production monitoring; it is a pre-deployment hardening tool.

textCopy
You are an adversarial prompt engineer tasked with hardening a production prompt against edge-case failures. Your goal is to generate a curated set of few-shot examples that teach the model to handle rare, ambiguous, or malformed inputs correctly.

## PRODUCTION PROMPT TO HARDEN
[PRODUCTION_PROMPT]

## TASK DESCRIPTION
[PRODUCTION_TASK_DESCRIPTION]

## INPUT SCHEMA
[INPUT_SCHEMA]

## EXPECTED OUTPUT SCHEMA
[OUTPUT_SCHEMA]

## KNOWN FAILURE MODES
[KNOWN_FAILURE_MODES]

## CONSTRAINTS
- Generate exactly [NUM_EXAMPLES] input-output example pairs.
- Each example must target a distinct edge case from the known failure modes or input schema boundaries.
- Examples must be realistic and plausible, not contrived nonsense.
- Output format must be a JSON array of objects, each with "input" and "expected_output" fields.
- For each example, include a "targeted_failure_mode" field explaining which edge case it addresses.
- If the production task requires abstention or refusal for certain inputs, include examples demonstrating correct refusal behavior.
- Do not generate examples that are already covered by the existing few-shot examples provided below.

## EXISTING FEW-SHOT EXAMPLES (DO NOT DUPLICATE)
[EXISTING_EXAMPLES]

## OUTPUT FORMAT
Return a JSON object with a single key "edge_case_examples" containing an array of example objects. Each object must have the following fields:
- "input": The edge-case input to the production prompt.
- "expected_output": The correct model response for this input.
- "targeted_failure_mode": A brief description of the edge case or failure mode this example addresses.
- "severity": One of "critical", "high", "medium", or "low".

To adapt this template, replace each square-bracket placeholder with concrete values from your production system. The [PRODUCTION_PROMPT] placeholder should contain the exact system and user prompt template you are hardening. The [KNOWN_FAILURE_MODES] field is critical: populate it with specific failure patterns observed in production logs, eval runs, or red-team exercises. If you lack production data, use the failure modes you hypothesize based on input schema boundaries—empty strings, maximum-length inputs, special characters, conflicting instructions, and ambiguous intent. The [EXISTING_EXAMPLES] field prevents the generator from wasting tokens on examples you already have. After generating examples, always validate them against your eval suite before inserting them into production prompts. A generated example that teaches incorrect behavior is worse than no example at all.

The Edge-Case Example Selection Prompt is not a one-off script; it is a component in a continuous example curation pipeline. In production, this prompt should be triggered by a scheduled job or a webhook that fires when new failure logs, support tickets, or eval reports indicate coverage gaps. The harness must supply the prompt with a structured [INPUT] containing recent production failures, a [CONTEXT] block with the current few-shot example library and its performance metrics, and an [OUTPUT_SCHEMA] that defines the expected JSON shape for the curated edge-case set. Without this harness, the prompt becomes a manual debugging tool rather than an automated safety net.

Wire the prompt into an application by wrapping it in a function that validates the output against the declared schema before the examples enter any production prompt. After the model returns the proposed edge-case set, run a validation layer that checks: (1) every example has a non-empty input and expected_output field, (2) no example duplicates an existing library entry by normalized text similarity above a configurable threshold (e.g., 0.92 cosine), and (3) the total token count of the new examples does not exceed the allocated context budget. If validation fails, log the failure with the raw model output and the specific validation error, then retry once with the error message appended to the prompt as a correction hint. After a second failure, escalate to a human review queue rather than silently inserting bad examples. For high-risk domains such as healthcare or finance, require human approval on all edge-case additions regardless of validation status.

Model choice matters here. Use a model with strong instruction-following and structured output capabilities, such as gpt-4o or claude-3-5-sonnet, because the task requires precise adherence to the output schema and careful reasoning about edge-case coverage. Avoid smaller or older models that may hallucinate example fields or fail to distinguish between genuine edge cases and common inputs. Log every invocation with a trace ID, the prompt version, the input failure samples, the model used, the output example set, validation results, and the final disposition (inserted, rejected, or escalated). This audit trail is essential for debugging example drift and for demonstrating governance to compliance reviewers. Finally, integrate the validated examples into your example library via an atomic update that preserves the previous version, enabling rollback if the new edge-case examples cause a regression in downstream eval scores.