Semantic Versioning Impact Regression Test Recommendation Prompt

This prompt is designed for library, API, and platform maintainers who are preparing a new release and need to understand the full testing impact of their version change. The core job is to take a semantic version bump (major, minor, or patch) and a diff of the public API surface, and produce a differentiated, risk-based regression test plan. The ideal user is an SDET, release manager, or senior engineer who understands their public API contracts and needs a structured, auditable recommendation for what to test, what to skip, and why. The prompt requires a clear public API surface diff, the current and proposed version strings, and optionally a changelog or commit history for additional context.

Use this prompt when the release has a well-defined public API and the team follows semantic versioning strictly. It is most effective when the API surface is machine-readable (e.g., OpenAPI schemas, protobuf definitions, or exported function signatures) and a diff can be generated programmatically. The prompt excels at identifying breaking changes that require exhaustive backward-compatibility testing, new functionality that needs fresh test coverage, and internal refactors that should be verified with existing regression suites. It also flags downstream consumer impact, such as deprecated endpoints or changed error signatures, that might be missed by a simple code-coverage analysis.

Do not use this prompt for internal-only modules without a defined public contract, or for repositories that do not follow semantic versioning. It is not a replacement for a full test impact analysis tool that traces code paths; it reasons about the contract and intent of changes, not their implementation details. Avoid this prompt when the API diff is incomplete or manually estimated, as the output quality depends entirely on the accuracy of the input diff. For hotfixes or emergency patches, use the 'Hotfix Regression Test Scope Prompt' instead. After generating the test plan, always have a domain expert review the backward-compatibility assertions and the 'skipped test' rationale before finalizing the release gate.

This prompt template is designed for library and API maintainers preparing a release. It takes a semantic versioning level (major, minor, patch), a public API surface diff, and a changelog as inputs, then produces a test scope differentiated by version impact. The output includes backward-compatibility test requirements specific to each version level, a risk-weighted test selection, and explicit rationale for auditability. The template uses square-bracket placeholders that you replace with your actual data before sending the prompt to the model.

textCopy
You are a release test strategist for a software library or API. Your task is to recommend a regression test scope based on a semantic versioning change.

## INPUTS
- Semantic Version Level: [VERSION_LEVEL]
- Public API Surface Diff: [API_SURFACE_DIFF]
- Changelog Summary: [CHANGELOG_SUMMARY]
- Existing Test Suite Inventory: [TEST_SUITE_INVENTORY]
- Historical Defect Data (optional): [HISTORICAL_DEFECT_DATA]

## CONSTRAINTS
- [CONSTRAINTS]

## OUTPUT SCHEMA
Return a JSON object with the following structure:
{
  "version_analysis": {
    "level": "major|minor|patch",
    "breaking_changes_detected": boolean,
    "backward_compatibility_risk": "high|medium|low|none"
  },
  "test_recommendations": [
    {
      "test_id": "string",
      "test_name": "string",
      "priority": "critical|high|medium|low",
      "selection_rationale": "string explaining why this test is selected",
      "version_impact_category": "breaking_change|deprecation|new_feature|behavior_change|regression_risk|compatibility",
      "estimated_runtime_seconds": number,
      "backward_compatibility_check": boolean
    }
  ],
  "backward_compatibility_test_plan": {
    "required_checks": ["string describing each mandatory compatibility check"],
    "deprecated_api_tests": ["test_id for deprecated surface tests"],
    "consumer_impact_assessment": "string summarizing downstream consumer risk"
  },
  "skipped_tests": [
    {
      "test_id": "string",
      "skip_rationale": "string explaining why this test can be safely skipped",
      "risk_of_skip": "low|medium|high"
    }
  ],
  "total_estimated_runtime_seconds": number,
  "coverage_gaps": ["string describing any identified gaps in existing test coverage"]
}

## INSTRUCTIONS
1. Classify every public API change as breaking, additive, deprecating, or behavioral.
2. For MAJOR version changes, require all backward-compatibility checks and flag every consumer-facing contract test.
3. For MINOR version changes, focus on new feature tests, deprecation warnings, and backward-compatible behavior verification.
4. For PATCH version changes, limit scope to the specific fix area, adjacent modules, and regression checks for previously escaped defects.
5. If [HISTORICAL_DEFECT_DATA] is provided, weight test selection toward modules with high defect density.
6. For each skipped test, provide a clear, auditable rationale that a release manager can review.
7. Flag any coverage gaps where the existing test suite does not cover a changed API surface.
8. If [CONSTRAINTS] includes a runtime budget, prioritize tests to maximize risk coverage within that budget.

## EXAMPLES
[EXAMPLES]

## RISK LEVEL
[RISK_LEVEL]

To adapt this template, replace each square-bracket placeholder with your actual data. The [API_SURFACE_DIFF] should be a structured diff of your public API—method signatures, type changes, removed endpoints, or modified contracts. The [TEST_SUITE_INVENTORY] should list available test identifiers with brief descriptions and typical runtimes. The [CONSTRAINTS] placeholder accepts runtime budgets, required test inclusions, or environment limitations. The [EXAMPLES] placeholder should contain one or two few-shot examples showing the expected output shape for each version level. The [RISK_LEVEL] placeholder should specify the deployment context—such as "financial services, high-availability requirement" or "internal tool, low blast radius"—to calibrate the model's risk tolerance.

Before using this prompt in a production release pipeline, validate the output against a golden dataset of known version changes and expected test selections. For high-risk releases, always require human review of the backward-compatibility test plan and any skipped tests with a risk_of_skip rating of "medium" or higher. Do not use this prompt as the sole gate for a production release without a human-in-the-loop approval step.

This prompt is designed to sit between your version control system and your test orchestrator. The primary integration point is a release pipeline stage that fires after a version tag is created but before the full regression suite is scheduled. The application layer must supply three structured inputs: the public API surface diff (generated by a tool like openapi-diff or a custom AST comparator), the full changelog or commit log since the last tagged release, and the current test catalog with metadata (names, tags, historical pass/fail rates, and module ownership). The prompt itself is stateless, so the harness is responsible for assembling these inputs, enforcing token budgets, and validating the output before it reaches the test runner.

The output schema must be enforced with a strict JSON validator before any test execution begins. The prompt returns a JSON object with three arrays: major_tests, minor_tests, and patch_tests, each containing test identifiers, a backward_compat_required boolean, and a rationale string. Your harness should validate that every returned test ID exists in the current test catalog and reject any hallucinated identifiers. Implement a retry loop with a maximum of two attempts: if validation fails, feed the validation errors back into the prompt as additional context and request a corrected output. If the second attempt also fails, escalate to the release manager with the partial results and a warning that manual test selection is required. Log every prompt invocation, the input diff, the raw output, and the validation result for auditability. For model choice, a model with strong structured output capabilities and a 128K+ context window is recommended because the API diff and test catalog can be large; GPT-4o or Claude 3.5 Sonnet are suitable defaults, but test with your actual catalog size before committing to a model.

The highest-risk failure mode is a false negative: the prompt recommends skipping a test that would have caught a breaking change. To mitigate this, always run the backward_compat_required: true tests regardless of the version bump level, and consider running the full major_tests array even for minor releases if the API diff touches a deprecated endpoint. A secondary harness check should compare the recommended test list against a static 'safety net' test set that you maintain for your most critical API contracts; if any safety-net test is missing from the recommendation, append it automatically and log the override. Do not use this prompt as the sole gate for a production release without a human review step for major version bumps. For minor and patch releases, you can automate the test selection but should still surface the rationale strings in the CI summary so the on-call engineer can spot-check the reasoning.