Test Execution Time vs Risk Trade-Off Decision Prompt

This prompt is designed for the moment a release coordinator, QA lead, or SDET faces a hard constraint: the full regression suite requires 12 hours, but the release window allows only 4. The job-to-be-done is not just to pick tests, but to produce a defensible, auditable test plan that maximizes risk coverage under a strict time budget. The ideal user has access to a list of test cases with historical execution times, module-to-test mappings, and a risk assessment of the changes in the release—typically derived from a code diff, dependency graph analysis, or a feature risk matrix. Without this context, the prompt cannot perform the required trade-off analysis and will produce a generic, ungrounded list.

Use this prompt when the decision must survive a post-release review. The output includes a Pareto analysis showing the cumulative risk coverage gained per minute of execution, which allows stakeholders to understand the 'why' behind every skipped test. This is critical in regulated environments (finance, healthcare) or high-reliability systems where a missed regression test can become an audit finding. The prompt is also appropriate for CI/CD pipelines with variable time budgets, such as a PR build that must complete in under 15 minutes, where the test selection logic must be transparent to the developer who owns the change.

Do not use this prompt when you have no execution time data for your tests, no risk assessment of the changes, or no ability to map tests to the modules affected by the release. In those cases, the prompt will hallucinate plausible-sounding but ungrounded trade-offs. This prompt is also not a substitute for a permanent, optimized test suite; it is a tactical decision tool for a specific release under a specific time constraint. If you find yourself using it for every release, the underlying problem is test suite bloat, and you should instead apply the Redundant Test Detection and Retirement Recommendation Prompt to permanently reduce execution time. Finally, do not use this prompt for safety-critical systems where any skipped test is unacceptable—in those contexts, the only correct answer is to extend the release window or parallelize execution, not to accept residual risk.

This prompt template is designed to be dropped directly into your AI orchestration layer, test harness, or CI/CD pipeline. It accepts a fixed execution window, a list of candidate tests with metadata, and a risk profile, then instructs the model to produce a constrained test selection that maximizes risk coverage under the time budget. The output includes a Pareto analysis showing coverage gained per minute of execution, explicit skip rationale for every excluded test, and a residual risk summary. Use this when you have a hard release deadline and cannot run the full suite.

textCopy
You are a test architect for a critical software release. Your task is to select a subset of regression tests that maximizes risk coverage within a strict execution time budget.

## INPUTS
- Test Catalog: [TEST_CATALOG_JSON]
  Each test object: { id, name, estimated_minutes, risk_score (1-10), module, tags, last_failure_date, flaky_score (0-1) }
- Time Budget (minutes): [TIME_BUDGET_MINUTES]
- Release Context: [RELEASE_CONTEXT]
  Include: changed modules, diff summary, known risky areas, any compliance requirements
- Constraints: [CONSTRAINTS]
  e.g., mandatory tests, excluded tags, maximum flaky score threshold

## OUTPUT SCHEMA
Return a JSON object with this exact structure:
{
  "selected_tests": [
    { "id": "string", "name": "string", "minutes": number, "risk_score": number, "selection_reason": "string" }
  ],
  "skipped_tests": [
    { "id": "string", "name": "string", "minutes": number, "risk_score": number, "skip_reason": "string", "risk_accepted": "string" }
  ],
  "total_selected_minutes": number,
  "budget_remaining_minutes": number,
  "coverage_summary": {
    "total_risk_covered": number,
    "total_risk_available": number,
    "risk_coverage_pct": number,
    "pareto_analysis": [
      { "cumulative_minutes": number, "cumulative_risk_pct": number, "marginal_risk_per_minute": number }
    ]
  },
  "residual_risk_statement": "string",
  "assumptions": ["string"]
}

## INSTRUCTIONS
1. Sort all tests by risk_score descending.
2. Select tests greedily by risk_score until the cumulative estimated_minutes reaches [TIME_BUDGET_MINUTES].
3. If a test would exceed the budget, skip it unless it is flagged as mandatory in [CONSTRAINTS].
4. For every skipped test, provide a specific skip_reason and describe the risk_accepted.
5. Exclude any test with flaky_score above the threshold in [CONSTRAINTS].
6. Generate the pareto_analysis array with at least 5 data points showing cumulative risk coverage vs. cumulative time.
7. Write the residual_risk_statement summarizing what risks remain untested and their potential impact on the release.
8. List all assumptions you made about test dependencies, environment stability, or risk score accuracy.

## EVALUATION CRITERIA
- Every selected test must have a risk_score >= 5 unless mandatory.
- No skipped test may lack a skip_reason.
- The total_selected_minutes must not exceed [TIME_BUDGET_MINUTES].
- The pareto_analysis must be internally consistent (monotonically increasing cumulative values).

Before wiring this into production, validate the output against the evaluation criteria listed in the prompt itself. Add a post-processing step that checks total_selected_minutes <= [TIME_BUDGET_MINUTES] and confirms every skipped test has a non-empty skip_reason. For high-risk releases, route the residual_risk_statement to a human release coordinator for sign-off before the test run begins. If your test catalog exceeds 500 entries, split it into module-based batches and merge results, or use a model with a larger context window.

This prompt is designed to be called programmatically during a release window when a fixed execution budget (e.g., 45 minutes) forces hard trade-offs. The implementation harness should treat the prompt as a decision-support step, not an autonomous executor. The model receives a ranked list of test suites with historical durations, risk scores, and coverage metadata, and returns a Pareto-optimal selection that maximizes risk coverage under the time constraint. The output must be validated for budget compliance, coverage completeness, and selection rationale before any test runner consumes it.

Wire the prompt into your CI/CD or release orchestration system as a pre-execution gate. The caller must supply: a JSON array of candidate test suites with suite_id, estimated_duration_minutes, risk_score (0-100), coverage_areas (list of tags), and last_failure_rate; a time_budget_minutes integer; and an optional mandatory_suite_ids list for tests that must run regardless of budget. The prompt returns a JSON object with selected_suites, skipped_suites, total_estimated_minutes, coverage_gaps (risk areas with zero selected tests), and a pareto_rationale string. Validate the response with these checks before forwarding to the test runner: (1) total_estimated_minutes must not exceed time_budget_minutes; (2) all mandatory_suite_ids must appear in selected_suites; (3) every skipped suite must have a non-empty rationale entry; (4) coverage_gaps must not include any area tagged as [CRITICAL_COVERAGE_AREA] without triggering a human review flag. If validation fails, retry once with the validation errors appended to the prompt as [CONSTRAINTS] feedback. After a second failure, escalate to the release coordinator with the partial output and validation log. Log the full prompt, response, validation results, and final test execution outcomes for auditability. This trace is essential for post-release reviews and for calibrating risk scores over time.

Model choice matters here. Use a model with strong structured output and reasoning capabilities (GPT-4o, Claude 3.5 Sonnet, or equivalent). Set temperature=0 to maximize deterministic selection behavior. The prompt's [OUTPUT_SCHEMA] placeholder should be replaced with a strict JSON schema definition matching your test management system's expected format. Avoid using this prompt for real-time CI triggers on every commit; it is designed for scheduled release gates where the time budget is known and the suite list is stable. For per-commit optimization, use the lighter-weight Pull Request Test Suite Optimization Prompt instead. Always pair this harness with a post-execution feedback loop: compare actual runtimes against estimates, track whether skipped suites later detected escaped defects, and feed that data back into the risk score model that populates the prompt's input fields.