Edge-Case Discovery Prompt for Regression Suites

This prompt is for QA engineers and prompt engineering teams who need to harden their regression test suites. It takes an existing set of test cases and systematically generates edge-case examples that stress the behavioral limits of a prompt under test. Use it when your golden test set feels too narrow, when you have observed production failures on boundary inputs, or when you are preparing a prompt for a release gate and need to demonstrate coverage of null inputs, conflicting constraints, maximum-length inputs, and adversarial phrasings. The core job-to-be-done is coverage expansion: turning a known set of passing cases into a catalog of challenging variants that reveal where the prompt under test is brittle, over-confident, or silently non-compliant.

The ideal user is someone who already has a working regression suite with at least 10–20 golden test cases and a defined expected output contract per case. You should have observed the prompt under test in production or staging long enough to know its common failure modes. This prompt is not a starting point for teams with no existing test cases; it amplifies an existing signal, it does not invent one from scratch. Required context includes the prompt under test, the golden test cases with their expected outputs, and any known production failure categories you want the edge-case generator to prioritize. You must also supply a risk tolerance level and any domain-specific constraints that bound acceptable behavior.

Do not use this prompt when your golden test set is unvalidated or when you lack a clear expected output contract for each case. If your existing tests are ambiguous or contradictory, the edge-case generator will amplify that noise rather than producing useful stress tests. This prompt also does not evaluate the prompt under test—it produces new test cases that you then feed into your regression harness. If you need a pass/fail judgment on a prompt version, use the Regression Test Harness Prompt instead. Finally, avoid this prompt for safety-critical refusal testing without human review of the generated edge cases; adversarial examples that trigger unsafe outputs must be screened before they enter an automated harness.

This prompt template is designed to be dropped directly into your AI workspace. It instructs the model to act as a QA engineer, analyzing a provided set of existing test cases to identify coverage gaps. The model will then generate new, targeted edge-case examples—including boundary conditions, null inputs, conflicting constraints, and adversarial inputs—that are specifically designed to stress the behavioral limits of the prompt under test. The output is a structured JSON array, ready for ingestion into your automated regression suite.

textCopy
You are a senior QA engineer specializing in prompt robustness. Your task is to analyze a set of existing test cases for an AI prompt and generate new edge-case examples to expand the regression suite.

First, review the provided [EXISTING_TEST_CASES] and the [TARGET_PROMPT_DESCRIPTION] to understand the prompt's intended behavior and current test coverage.

Next, identify coverage gaps across the following dimensions:
- Boundary conditions (max/min length, numeric limits, date ranges)
- Null, empty, or missing inputs
- Conflicting or mutually exclusive constraints
- Adversarial inputs designed to trigger over-refusal or unsafe compliance
- Ambiguous or underspecified requests
- High-risk or regulated-domain scenarios

Generate a JSON array of new test cases. Each object in the array must have the following structure:
{
  "test_id": "A unique string identifier, e.g., 'EDGE-001'",
  "test_name": "A short, descriptive name for the test case",
  "input": "The exact input to be fed to the target prompt",
  "expected_behavior": "A clear description of the correct model response, including refusal conditions, format requirements, or specific content that must or must not be present.",
  "failure_mode": "The specific vulnerability or gap this test is designed to catch (e.g., 'null input crash', 'over-refusal on legitimate request')",
  "risk_level": "One of: 'low', 'medium', 'high', 'critical'"
}

Apply these behavioral constraints:
- [CONSTRAINTS]

Here are the existing test cases to analyze:
[EXISTING_TEST_CASES]

Here is a description of the target prompt being tested:
[TARGET_PROMPT_DESCRIPTION]

Return ONLY the valid JSON array. Do not include any explanatory text outside the JSON.

To adapt this template, replace the square-bracket placeholders with your specific context. [EXISTING_TEST_CASES] should contain your current test suite, ideally in a structured format like JSONL or a markdown table. [TARGET_PROMPT_DESCRIPTION] is a concise summary of the prompt you are testing, including its role, output format, and key behavioral rules. The [CONSTRAINTS] placeholder is critical for high-risk domains; use it to inject specific safety policies, regulatory requirements, or refusal rules that the generated edge cases must respect. For example, a healthcare constraint might be: "Generated test cases must not use real patient data and must test for PHI leakage." After running this prompt, validate the output JSON against the defined schema before adding the new cases to your regression suite. A failed generation should trigger a retry or a manual review of the gaps identified.

The edge-case discovery prompt is not a one-off brainstorming tool; it is a programmatic step in a test suite hardening workflow. The typical integration pattern is a scheduled or trigger-based job that reads the current golden test set, invokes the prompt to generate candidate edge cases, validates the generated examples against a schema, deduplicates them against the existing suite, and appends approved cases to a staging area for human or automated review before promotion to the active regression suite.

Wire the prompt into a Python or TypeScript harness that manages the full lifecycle. The harness should: (1) load the existing test suite from a version-controlled JSON or YAML file; (2) serialize the suite into the [EXISTING_TEST_SUITE] placeholder, including input, expected output, and difficulty tags for each case; (3) call the model with a low temperature (0.1–0.3) to prioritize coverage over creativity; (4) parse the structured JSON output and validate each generated case against a schema that requires input, expected_output, edge_case_category, and rationale fields; (5) run a semantic deduplication check against the existing suite using embedding similarity to avoid near-duplicate cases; (6) write accepted candidates to a staging/ directory with a timestamp and the generating prompt version for auditability. For high-risk domains, insert a human review step before staging cases are merged into the active suite. Log every generation run with the prompt version, model used, number of cases generated, number accepted, and any validation failures.

Model choice matters. Use a model with strong instruction-following and structured output capabilities for the generation step. If the prompt produces malformed JSON, implement a retry with the Output Repair and Validation Prompts pillar patterns rather than silently dropping cases. For cost control, consider running edge-case discovery on a weekly cadence or triggered by a detected drift in production input distributions. Avoid wiring this prompt directly into a CI/CD gate without human review unless the domain is low-risk and the generated cases are purely additive. The output of this harness feeds directly into the Regression Test Harness Prompt and the Golden Dataset Drift Detection Prompt, forming a closed-loop test maintenance system.