Risk-Based Test Case Prioritization Prompt

This prompt is built for QA managers and release leads who face a hard constraint: more test cases than execution capacity. Instead of running tests in an arbitrary order or skipping them based on intuition, you feed the model a flat list of test cases—each with metadata like failure history, business impact, and recent code changes—and it returns a risk-ranked execution order with explicit scores and justifications. The output is a defensible, evidence-based prioritization that you can share with stakeholders, attach to release notes, or feed into a test management system. The prompt does not generate new test cases; it assumes you already have a test case inventory with enough metadata to assess probability and impact.

Use this prompt when you need to make a trade-off between coverage and time, and you want that trade-off to be transparent. Ideal inputs include test case IDs, titles, historical pass/fail rates, associated requirement or risk IDs, business criticality labels (e.g., 'revenue-blocking', 'PII-exposed'), and pointers to recent code diffs or changed modules. The prompt works best when you provide a consistent risk taxonomy—such as a 1–5 scale for both probability and impact—so the model can apply it uniformly. If your test cases lack failure history or impact metadata, the model will still produce a ranking, but the justifications will be weaker and you should flag the output as 'low-confidence' in your test plan.

Do not use this prompt when you need to generate new test cases, when you are performing root cause analysis on a specific failure, or when you are designing a test automation strategy from scratch. It is also not a substitute for a formal risk assessment workshop with product and engineering stakeholders; the model ranks what you give it, but it cannot discover risks you haven't documented. If your test case inventory is incomplete or your metadata is stale, fix that first. After running the prompt, always review the top-ranked items for face validity—if a known high-risk area is missing from the top tier, your input metadata may be incomplete or your risk taxonomy may need adjustment.

This prompt template is designed to be pasted directly into your AI harness. It instructs the model to act as a QA architect, consuming a list of test cases alongside risk context and producing a prioritized execution order. The model is forced to reason about both the likelihood of failure and the severity of impact before assigning a final risk score. All placeholders are enclosed in square brackets—replace them with real data from your test management system or risk register before execution.

textCopy
You are a QA architect specializing in risk-based testing. Your task is to analyze a list of test cases and prioritize them for execution based on failure probability and business impact.

## INPUT
[TEST_CASE_LIST]

## RISK CONTEXT
- Feature Area: [FEATURE_AREA]
- Recent Code Changes: [RECENT_CODE_CHANGES]
- Historical Defect Data: [HISTORICAL_DEFECT_DATA]
- Business Criticality: [BUSINESS_CRITICALITY]

## CONSTRAINTS
- Assign a Failure Probability score (1-5, where 5 is almost certain to fail).
- Assign a Business Impact score (1-5, where 5 is a critical revenue or compliance outage).
- Calculate a Risk Score by multiplying Failure Probability by Business Impact.
- Justify each score with a specific, evidence-based reason tied to the provided context.
- Do not invent failure modes not supported by the input data.
- Identify any high-impact scenarios that appear to be missing from the test case list.

## OUTPUT_SCHEMA
Return a JSON object with a single key "prioritized_test_cases" containing an array of objects. Each object must have the following fields:
- "test_case_id": string
- "test_case_title": string
- "failure_probability": integer (1-5)
- "business_impact": integer (1-5)
- "risk_score": integer (1-25)
- "justification": string
- "execution_order": integer

Also include a "missing_coverage_notes" array of strings at the top level.

## OUTPUT

To adapt this prompt, replace [TEST_CASE_LIST] with a structured dump of your test cases, including their IDs, titles, and steps. The [RISK_CONTEXT] fields are critical for the model to produce grounded justifications; if you omit them, the model will hallucinate plausible-sounding but unsupported risk factors. For high-stakes releases, add a [HUMAN_REVIEW_FLAG] boolean to the output schema and route any test case with a risk score above 15 to a manual review queue before execution. Always validate the output JSON against the schema before ingesting it into your test runner to catch format drift or missing fields.

This prompt is designed to be called programmatically as part of a release readiness or sprint planning workflow, not as a one-off chat interaction. The application layer is responsible for assembling the [FEATURE_SPEC], [RISK_MATRIX], [TEST_CATALOG], and [BUSINESS_CONTEXT] inputs from your test management system (e.g., TestRail, Zephyr, Xray) and risk register before invoking the model. The prompt returns a structured JSON payload containing a ranked test suite with risk scores, justifications, and a recommended execution order. Your harness must validate this output against a defined schema before accepting it into downstream systems.

Validation and Schema Enforcement: Define a strict JSON schema for the expected output, including required fields like testId, riskScore (integer 1-100), probability, impact, justification, and executionOrder. Use a validation library (e.g., ajv for JavaScript, pydantic for Python) to reject malformed responses. If validation fails, implement a retry loop (maximum 2 retries) that feeds the validation errors back into the prompt as additional [CONSTRAINTS]. After the final retry, if the output still fails validation, log the failure and escalate to a human QA lead for manual prioritization. Eval Checks: Run automated evals on every response to verify risk factor consistency—for example, a test case tagged as 'low probability' should not receive a risk score above 40. Also check that at least one test case from each high-impact area in the [RISK_MATRIX] appears in the top quartile of the ranked list. Flag any response that omits a high-impact scenario entirely.

Model Choice and Tool Use: This prompt works best with models that have strong reasoning capabilities (e.g., GPT-4o, Claude 3.5 Sonnet). Avoid smaller or older models that may struggle with the multi-factor ranking logic. The prompt does not require external tools or RAG retrieval—all necessary context is provided in the input placeholders. However, you should log every invocation with the full prompt, model response, validation result, and eval scores to an observability platform (e.g., LangSmith, Braintrust, or your existing monitoring stack). This trace data is essential for auditing prioritization decisions during post-release reviews and for detecting drift in model behavior over time. Human Review Gate: For any release with a [RISK_LEVEL] of 'critical' or 'high', route the final ranked test suite to a QA manager for approval before execution begins. The approval UI should display the risk scores, justifications, and any eval warnings side by side.