Prompt Boundary Probing Prompt

This prompt is for prompt engineers and QA leads who need to systematically map the edges of a prompt's intended behavior before it ships. The job-to-be-done is producing a boundary test suite—a set of edge-case inputs, expected behaviors at limits, and confidence degradation curves—that documents the safe operating envelope of a prompt. Use it when you have a stable prompt candidate and need to understand where it breaks, degrades, or produces unexpected outputs under stress. The ideal user has a prompt under test, knows its primary success criteria, and needs a repeatable method to discover failure modes that unit tests and golden datasets might miss.

Do not use this prompt for initial prompt authoring, for evaluating model quality independent of a specific prompt, or for generating standard regression test cases from known examples. It is not a replacement for golden-dataset evaluation or semantic drift detection. The prompt assumes you already have a working prompt and need to stress its boundaries. Required context includes the prompt under test, its intended behavior specification, known constraints, and the risk tolerance of the deployment environment. If the prompt handles regulated or high-risk domains, pair the output with human review and evidence grounding before accepting any boundary as safe.

The output is a structured boundary test suite, not a pass/fail report. Expect edge-case input descriptions, expected behavior at each boundary, confidence degradation indicators, and a documented safe-operating envelope. Wire this into your QA pipeline as a pre-release discovery step—run it before final regression gates to catch brittle instructions early. Avoid treating the generated boundaries as exhaustive; they are a starting point for adversarial thinking. After generating the suite, prioritize test cases by risk severity and integrate the highest-risk boundaries into your automated regression harness.

This prompt template is designed to be dropped into your test generation workflow. It instructs the model to act as a boundary-test designer, producing a structured suite of edge-case inputs, expected behaviors at each limit, and confidence degradation notes. The output is a machine-readable test plan you can feed directly into your regression harness.

textCopy
You are a prompt boundary-test designer. Your task is to analyze a target prompt and produce a boundary test suite that maps the edges of its intended behavior.

## TARGET PROMPT
[PROMPT_UNDER_TEST]

## OUTPUT SCHEMA
Return a JSON object with the following structure:
{
  "safe_operating_envelope": "string describing the input space where the prompt is expected to behave correctly",
  "boundary_cases": [
    {
      "case_id": "string",
      "boundary_type": "one of: input_length | input_format | semantic_edge | role_boundary | tool_boundary | policy_edge | ambiguity | contradiction | noise_tolerance",
      "input": "the exact input to test",
      "expected_behavior": "what the prompt should do at this boundary",
      "degradation_risk": "low | medium | high",
      "confidence_should_drop": true | false,
      "failure_mode_if_broken": "what happens if the prompt fails at this boundary"
    }
  ],
  "confidence_degradation_curve": {
    "description": "how confidence should degrade as inputs move from center to edge",
    "stages": [
      {
        "distance_from_center": "in_center | near_edge | at_boundary | beyond_boundary",
        "expected_confidence": "high | medium | low | should_refuse",
        "behavior_change": "description"
      }
    ]
  },
  "coverage_gaps": ["boundary types not covered by this suite"],
  "test_priority_order": ["case_ids in recommended execution order"]
}

## CONSTRAINTS
- Generate at least [MIN_CASES] boundary cases covering at least [MIN_BOUNDARY_TYPES] distinct boundary types.
- Every case must include a concrete, executable input string.
- Do not invent behaviors the target prompt does not specify.
- If the target prompt has explicit refusal or safety rules, include policy-edge cases.
- Mark any case where the expected behavior is ambiguous as degradation_risk: "high".
- For [RISK_LEVEL] risk prompts, include human-review flags on cases where model behavior is uncertain.

## EXAMPLES OF BOUNDARY TYPES
- input_length: very short inputs, very long inputs, empty inputs
- input_format: unexpected formats, markdown, JSON, code blocks, mixed languages
- semantic_edge: topics adjacent to the prompt's domain but outside it
- role_boundary: inputs that try to change the assistant's role or instructions
- tool_boundary: inputs that misuse or over-call available tools
- policy_edge: inputs near refusal boundaries
- ambiguity: inputs with multiple valid interpretations
- contradiction: inputs that conflict with themselves or prior context
- noise_tolerance: inputs with typos, OCR errors, or garbled text

Adapt this template by adjusting the boundary types list to match your prompt's domain. For prompts with tool access, expand tool_boundary cases to cover argument edge cases, missing required parameters, and tool-call loops. For customer-facing prompts, add tone and brand-safety boundaries. The output JSON is designed to be consumed by an automated test runner—each case_id maps to a test execution record, and the test_priority_order field lets you run high-risk cases first in CI/CD. Always version this boundary suite alongside the prompt it tests so you can detect when a prompt change shrinks the safe operating envelope.

The Prompt Boundary Probing Prompt is designed to be executed as a pre-release gate in a CI/CD pipeline, not as a one-off manual test. The harness should take the prompt under test as an input variable, inject it into the probing template, and execute the generated boundary test suite against the prompt. This means the harness must be able to programmatically substitute the [PROMPT_UNDER_TEST] placeholder, run the model to generate the test plan, parse the structured output, and then execute each generated test case against the same prompt in a controlled loop. The primary integration point is your existing evaluation runner—this prompt generates the test cases that runner will execute.

The implementation should follow a three-stage pipeline. Stage 1: Generation. Call the boundary probing prompt with the prompt-under-test and a strict output schema. Validate that the returned JSON contains the required fields (test_suite_name, safe_operating_envelope, boundary_test_cases) and that each test case has a non-empty input, expected_behavior, and boundary_category. If validation fails, retry once with a repair prompt that includes the validation error. Stage 2: Execution. For each generated test case, send the input to the prompt-under-test in a fresh context (no conversation history leakage). Collect the raw output, latency, and token usage. Stage 3: Evaluation. For each output, run the eval_criteria checks defined in the test case. This typically includes an LLM-as-judge call to verify expected_behavior adherence, plus structural checks like schema validity or refusal keyword detection. Log all results with the test case ID, prompt version, model version, and timestamp for traceability.

For high-risk prompts (those governing safety policies, financial decisions, or clinical workflows), add a human review gate between Stage 1 and Stage 2. The generated boundary test suite should be reviewed by a prompt engineer before execution to confirm the edge cases are relevant and the expected behaviors are correct. Store the approved test suite as a versioned artifact in your prompt registry. In CI/CD, the gate should block promotion if any critical severity test case fails or if the overall pass rate drops below a configurable threshold (e.g., 95%). Wire the results into your observability stack so that boundary failures trigger alerts and are correlated with the specific prompt version that degraded. Avoid running boundary probes in production traffic; this is a pre-release testing workflow that should use dedicated evaluation infrastructure.