Pull Request Test Suite Optimization Prompt Template

This prompt is designed for CI/CD pipeline integration at pull request creation time. It analyzes changed file paths, module ownership, static analysis hints, and a runtime budget to produce a minimal yet sufficient test subset. The output includes a confidence score for every skipped test, making the selection auditable for release managers and SDETs who need to defend why certain tests were not executed. Use this when your full regression suite exceeds your PR pipeline time budget and you need a data-driven, repeatable method for test selection that does not rely on a single engineer's intuition.

The prompt requires structured inputs: a list of changed file paths from the git diff, a module ownership map linking paths to teams, any available static analysis results (lint warnings, complexity scores, security scan findings), and a hard runtime budget in minutes. Without these inputs, the model cannot produce a defensible selection. The output schema must be strictly enforced—expect a JSON array of test suites to execute, each with a rationale, and a separate array of skipped suites with confidence scores and risk justifications. Wire this into your CI system so that the prompt runs before test execution, and the output directly feeds your test runner's filter or tag mechanism.

Do not use this prompt for hotfixes that bypass normal PR pipelines, for repositories with fewer than 50 test suites where full runs are cheap, or when the runtime budget is so tight that even a minimal selection cannot complete. The prompt is not a substitute for flaky test quarantine—combine it with a flaky test exclusion list if your suite has known instability. Always log the full prompt input and output as a build artifact so release managers can audit selection decisions post-hoc. If a skipped test's confidence score falls below your team's threshold, configure the pipeline to escalate for human review rather than silently omitting it.

This prompt template is designed to be pasted directly into your CI/CD pipeline configuration, an AI gateway, or a test orchestration script. It instructs the model to act as a test architect, analyzing a pull request's diff, file paths, and ownership metadata to produce a JSON output that specifies exactly which tests to run, which to skip, and why. The template uses square-bracket placeholders for all dynamic inputs, ensuring you can replace them with data from your version control system, test management platform, and static analysis tools before each invocation.

markdownCopy
You are a senior test architect optimizing a regression test suite for a pull request. Your goal is to minimize execution time while maintaining acceptable risk coverage.

## Input Data
- **PR Diff:** [DIFF]
- **Changed File Paths:** [CHANGED_FILES]
- **Module Dependency Graph (Mermaid or JSON):** [DEPENDENCY_GRAPH]
- **Test Suite Catalog (Test Name, Path, Tags, Avg Runtime, Owner):** [TEST_CATALOG]
- **Historical Failure Data (Test Name, Failure Rate, Last Failed Commit):** [FAILURE_DATA]
- **Code Ownership Map (Path Pattern -> Team):** [OWNERSHIP_MAP]

## Constraints
- **Runtime Budget:** [RUNTIME_BUDGET_MINUTES] minutes.
- **Mandatory Tests (must always run):** [MANDATORY_TESTS]
- **Flaky Test Quarantine List (do not select):** [QUARANTINE_LIST]
- **Risk Threshold:** Acceptable risk of a missed regression is [RISK_THRESHOLD]%.

## Task
1. Analyze the changed files and dependency graph to identify all directly and transitively impacted modules.
2. Cross-reference impacted modules with the test suite catalog to find candidate tests.
3. Prioritize candidates using a risk score derived from: code churn size, historical failure rate of the test, and criticality of the owning team's module.
4. Select a subset of tests that fits within the [RUNTIME_BUDGET_MINUTES]-minute budget, maximizing risk coverage.
5. For every test NOT selected, provide a specific, auditable reason for skipping it.

## Output Schema
Return a single valid JSON object with this exact structure:
{
  "analysis_summary": {
    "impacted_modules": ["string"],
    "total_candidate_tests": "integer",
    "estimated_runtime_all_candidates_minutes": "float"
  },
  "selected_tests": [
    {
      "test_name": "string",
      "risk_score": "float (0-100)",
      "selection_reason": "string"
    }
  ],
  "skipped_tests": [
    {
      "test_name": "string",
      "skip_reason": "string",
      "risk_if_skipped": "low | medium | high",
      "confidence_in_skip": "float (0-100)"
    }
  ],
  "total_estimated_runtime_minutes": "float",
  "coverage_risk_assessment": "string"
}

To adapt this template, replace each [PLACEHOLDER] with data from your ecosystem. The [DIFF] can be a unified diff from git diff main...HEAD. The [DEPENDENCY_GRAPH] can be generated by tools like madge or your build system. The [TEST_CATALOG] should be a JSON array exported from your test management system. Crucially, the [RUNTIME_BUDGET_MINUTES] and [RISK_THRESHOLD] are policy decisions you must supply. After replacing the variables, send the prompt to a capable model (like GPT-4o or Claude 3.5 Sonnet) and validate the JSON output against the schema before using it to orchestrate your test runner. For high-risk deployments, route the skipped_tests list with a high risk rating to a human reviewer for final approval before the pipeline proceeds.

This prompt is designed to be called programmatically during pull request creation or update events. The implementation harness should treat the prompt as a deterministic component of a larger pipeline: it receives structured inputs from the PR event, produces a structured JSON output, and that output must be validated before any test-skipping decisions are executed. The harness is responsible for providing the prompt with the correct [CHANGED_PATHS], [DEPENDENCY_GRAPH], [TEST_CATALOG], [HISTORICAL_FAILURE_DATA], and [RUNTIME_BUDGET] placeholders. These inputs should be assembled from your version control system (e.g., git diff --name-only origin/main...HEAD), your test management database, your CI flakiness tracker, and your pipeline configuration respectively.

The harness must enforce a validation gate on the model's output before acting on it. Parse the JSON response and check that every recommended test exists in your test catalog, that the sum of estimated runtimes does not exceed the [RUNTIME_BUDGET], and that the skipped_tests array includes a non-empty rationale and confidence_score for each entry. If validation fails, retry the prompt once with the validation errors appended to the [CONSTRAINTS] input. If the retry also fails, fall back to running the full test suite and log the incident for prompt improvement. Never skip a test based on an unvalidated or low-confidence recommendation (e.g., confidence_score < 0.85) without human review.

For model choice, use a model with strong JSON mode and reasoning capabilities. Temperature should be set to 0 or near-zero to maximize output determinism. Implement structured logging that captures the PR identifier, input change set, raw prompt, raw response, validation result, and final test execution decision. This audit trail is critical for post-incident review and for tuning the prompt's risk thresholds over time. Wire the harness into your CI platform (GitHub Actions, GitLab CI, Jenkins) as a gate that runs before the test matrix is dispatched, and ensure the test orchestrator can dynamically include or exclude tests based on the validated, approved output of this prompt.