Breaking Change Detection Prompt

This prompt is designed for API and library maintainers who need to automatically assess whether a pull request introduces breaking changes to public contracts, schemas, or documented behaviors. The core job-to-be-done is to transform a raw code diff into a structured risk assessment that flags backward-incompatible modifications—such as removed endpoints, changed field types, tightened validation, or altered default values—before they reach production. The ideal user is a platform engineer, tech lead, or CI/CD pipeline operator who owns API stability and needs consistent, evidence-backed analysis that integrates directly into code review tools or release gates.

Use this prompt when you have a concrete diff against a known public API surface and you need a machine-readable report that maps each finding to a specific line, severity level, and migration note. The prompt assumes you can provide the diff content, a description of the public contract (e.g., an OpenAPI spec, interface definitions, or documented endpoints), and any versioning policy constraints. It is not suitable for reviewing internal-only refactors with no public surface, for assessing performance regressions without API signature changes, or for security-focused vulnerability scanning—those require separate specialized prompts. The prompt also does not replace human judgment for critical path decisions; its output should be treated as a review aid that flags candidates for manual verification.

Before wiring this into an automated pipeline, you must establish a ground-truth dataset of known breaking and non-breaking changes to measure false positive and false negative rates. The evaluation criteria should include precision (are flagged changes truly breaking?), recall (did the prompt miss any known breaking changes from your changelog?), and severity classification accuracy. If your API follows semantic versioning, the prompt's output should align with your MAJOR/MINOR/PATCH bump rules. Start by running this prompt against historical PRs with documented breaking changes to calibrate expectations, and always route high-severity findings to a human reviewer before blocking a merge or deployment.

The prompt below is designed to be copied directly into your application or evaluation harness. It accepts a code diff, an optional public API specification, and a set of constraints that define what constitutes a breaking change for your project. The placeholders let you inject project-specific definitions without rewriting the core logic. Use this template as the foundation for a CI/CD step that flags risky changes before they reach production.

textCopy
You are an API compatibility reviewer. Your task is to analyze the provided code diff and identify any breaking changes to public APIs, schemas, or contracts.

## INPUT
[DIFF]

## OPTIONAL CONTEXT
Public API specification or contract document:
[API_SPEC]

## DEFINITIONS
A breaking change is any modification that would cause existing, correctly-functioning consumers to fail or behave incorrectly. Use the following project-specific definitions to guide your analysis:
[BREAKING_CHANGE_DEFINITIONS]

## CONSTRAINTS
- Only report changes that affect public, documented, or exported interfaces.
- Ignore internal refactors, private method changes, and implementation details unless they alter public behavior.
- If a change is additive (new optional field, new endpoint), classify it as NON_BREAKING.
- If a change is subtractive or modifies existing behavior, classify it as BREAKING.
- For each finding, cite the specific file and line range from the diff.
- If no breaking changes are detected, explicitly state that.

## OUTPUT SCHEMA
Return a JSON object with the following structure:
{
  "summary": "A one-sentence summary of the overall risk level.",
  "risk_level": "HIGH" | "MEDIUM" | "LOW" | "NONE",
  "breaking_changes": [
    {
      "severity": "CRITICAL" | "MAJOR" | "MINOR",
      "category": "SIGNATURE_CHANGE" | "BEHAVIOR_CHANGE" | "SCHEMA_CHANGE" | "REMOVAL" | "RENAMING" | "CONTRACT_VIOLATION",
      "location": "file:line_range",
      "description": "What changed and why it breaks consumers.",
      "migration_guidance": "What consumers must do to adapt, or 'N/A' if removal is intentional."
    }
  ],
  "non_breaking_changes": [
    {
      "category": "ADDITION" | "DEPRECATION" | "INTERNAL_REFACTOR",
      "location": "file:line_range",
      "description": "What changed and why it is safe."
    }
  ]
}

## EXAMPLES
[EXAMPLES]

## INSTRUCTIONS
1. Parse the diff to identify all public-facing changes.
2. Compare each change against the API specification if provided.
3. Classify each change as BREAKING or NON_BREAKING using the definitions above.
4. Produce the JSON output. Do not include any text outside the JSON object.

To adapt this template, replace the placeholders with your project's concrete details. The [BREAKING_CHANGE_DEFINITIONS] placeholder is the most critical: it should contain a bulleted list of what your team considers a breaking change (e.g., "Removing a public method," "Changing a required field to optional," "Altering the type of a response field"). The [EXAMPLES] placeholder should include one or two few-shot examples showing a diff snippet and the expected JSON output, which dramatically improves classification accuracy. If you maintain an OpenAPI spec, GraphQL schema, or public interface document, paste it into [API_SPEC] to give the model a ground-truth contract to compare against. For CI/CD integration, wrap this prompt in a harness that validates the output JSON against the schema, logs any parse failures, and routes HIGH-risk findings for human review before merge.

The Breaking Change Detection Prompt is designed to be called automatically on every pull request that modifies public-facing API surfaces, schemas, or serialization contracts. In a typical CI/CD harness, a webhook triggers the workflow when a PR is opened or updated. The system extracts the diff, packages it with the current API specification (OpenAPI, GraphQL schema, or protobuf definitions) as the [CONTEXT], and sends the assembled prompt to the model. The output must be a structured JSON payload containing a breaking_changes array, each with a severity (MAJOR, MINOR, PATCH), location, description, and migration_notes field. Before the result is posted as a review comment, a validation layer must confirm the JSON schema, verify that every location references a line present in the diff, and check that MAJOR findings include non-empty migration_notes. If validation fails, the harness should retry once with the validation errors appended as [CONSTRAINTS] to help the model self-correct.

For model choice, a capable instruction-following model with strong code understanding is required—GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro are appropriate baselines. Smaller or local models will produce higher false-negative rates on subtle breaking changes like default value modifications or exception type narrowing. The harness should log every invocation: the PR ID, diff hash, model version, raw response, validation result, and final payload. This log becomes the audit trail for false-positive and false-negative analysis. To evaluate the prompt in production, maintain a golden dataset of known breaking and non-breaking changes from your project's changelog. Run the prompt against this dataset weekly and track precision and recall. A breaking change that the prompt misses (false negative) is more costly than a false positive, so tune the system prompt's [RISK_LEVEL] parameter to err toward over-detection when reviewing APIs with external consumers.

Human review remains mandatory for all MAJOR findings before a PR is merged. The harness should post MINOR and PATCH findings as non-blocking review comments, but MAJOR findings must block the merge and require a maintainer to either accept the breaking change (with a migration plan) or adjust the code to preserve compatibility. Do not use this prompt as a replacement for semantic versioning policy enforcement; it is a detection aid. The prompt cannot reason about undocumented behavioral contracts, performance SLAs, or client-side breakage from response timing changes. For those risks, pair this prompt with contract testing, integration test suites, and a human API design review. When the prompt flags a false positive, add the diff to the golden dataset as a negative example and consider including it as a few-shot counterexample in future prompt iterations.