API Response Contract Compatibility Check Prompt

Use this prompt when you have modified a system or user prompt that produces structured data (JSON, XML, YAML) and you need to verify that the new output schema is compatible with the existing contract expected by downstream services. This is a pre-deployment gate, not a post-deployment fix. It assumes you have access to the old prompt's output schema and a representative sample of the new prompt's output. The primary job-to-be-done is preventing silent integration failures where a prompt change introduces a breaking schema modification that downstream parsers, databases, or API consumers cannot handle.

The ideal user is an integration engineer, platform engineer, or prompt author preparing a release. You should have the previous prompt's expected schema (as a JSON Schema, TypeScript interface, or structured specification), a sample of at least 10-20 outputs from the new prompt version, and a clear understanding of which downstream systems consume the output. Do not use this prompt for evaluating free-text quality, for prompts that do not produce machine-readable structured output, or when you lack a reference schema to compare against. It is also not a substitute for full regression testing against golden datasets—it specifically targets the contract compatibility dimension.

Before running this prompt, ensure you have captured the old schema and a representative output sample from the new prompt. The prompt will produce a field-level compatibility matrix with breaking, additive, and deprecated change flags. After receiving the analysis, you should validate the findings against your own schema validation harness and decide whether the detected changes are intentional and safe. If breaking changes are flagged, either revert the prompt modification, update downstream consumers, or version the API contract before deployment.

This prompt template is designed to be dropped directly into your evaluation or CI/CD harness. It takes two JSON schemas—an old version and a new version—and produces a structured compatibility matrix. The core job is to prevent silent contract breakage: a downstream consumer that parses the old schema must not fail when receiving the new output. The prompt forces the model to reason about required fields, type changes, enum modifications, and structural renames, not just surface-level diffs.

codeCopy
You are a schema compatibility auditor. Your task is to compare two JSON output schemas and produce a field-level compatibility matrix.

## INPUT
Old Schema:
```json
[OLD_SCHEMA]

New Schema:

jsonCopy
[NEW_SCHEMA]

DEFINITIONS

• breaking: A change where a downstream consumer relying on the old schema would fail to parse, deserialize, or process the new output. Includes: removing a required field, changing a field type, narrowing an enum, renaming a field without an alias, or changing a field from required to absent.
• additive: A new optional field added to the schema that does not affect existing consumers. Includes: adding an optional field, adding a new enum value, or adding a new property to an existing object.
• deprecated: A field that exists in the new schema but is marked for removal or no longer populated. The field is still present but should not be relied upon.
• compatible: A change that does not break existing consumers. Includes: adding a description, changing a default value, or reordering fields.

OUTPUT SCHEMA

Return a JSON object with the following structure:

jsonCopy
{
  "summary": {
    "total_fields_analyzed": <integer>,
    "breaking_changes": <integer>,
    "additive_changes": <integer>,
    "deprecated_fields": <integer>,
    "compatible_changes": <integer>,
    "overall_risk": "high" | "medium" | "low",
    "deployment_blocker": <boolean>
  },
  "matrix": [
    {
      "field_path": "<string, e.g., 'user.address.city'>",
      "change_type": "breaking" | "additive" | "deprecated" | "compatible",
      "old_definition": "<string summarizing old schema for this field>",
      "new_definition": "<string summarizing new schema for this field>",
      "impact_description": "<string explaining what breaks or changes>",
      "migration_action": "<string: required action before deployment>"
    }
  ],
  "actions_required_before_deployment": [
    "<string: concrete step>"
  ]
}

CONSTRAINTS

• Analyze every field recursively, including nested objects and arrays.
• If a field is renamed, flag it as breaking and note the old and new names.
• If a required field becomes optional, flag it as compatible but note the behavioral change.
• If an enum loses values, flag it as breaking.
• If the root type changes (e.g., object to array), flag it as breaking with maximum severity.
• Do not invent fields not present in either schema.
• Set deployment_blocker to true if any breaking change exists.

To adapt this prompt, replace [OLD_SCHEMA] and [NEW_SCHEMA] with the actual JSON schemas you are comparing. If your schemas are large, consider extracting only the output-relevant portions—internal fields that consumers never see can be excluded to reduce noise. For OpenAPI or GraphQL schemas, preprocess them into the JSON Schema subset that your model outputs actually conform to. The prompt works best when both schemas use the same naming conventions; if your team renamed fields en masse, add a [FIELD_MAPPING] section to supply known aliases and reduce false-positive breaking flags.

After running this prompt, validate the output JSON against the declared output schema before trusting the matrix. A common failure mode is the model missing nested fields inside arrays of objects—add a post-processing check that counts the total number of leaf fields in both input schemas and compares it to total_fields_analyzed. If the counts diverge by more than 5%, re-run with explicit instructions to recurse into array items. For high-risk API contracts, always have a human review the breaking-change list before approving deployment, especially when the model flags a field as compatible but your integration tests suggest otherwise.

This prompt is designed to operate as a pre-release gate in a CI/CD pipeline, not as a one-off manual check. The harness should invoke the prompt whenever a structured-output prompt template is modified, comparing the new version's output contract against a committed golden schema or the previous version's contract. The prompt expects two structured output definitions as input: a baseline contract and a candidate contract. These can be JSON Schema documents, TypeScript interfaces, or structured markdown tables describing field names, types, required status, and enum values. The harness is responsible for extracting these contracts from the prompt version history or from actual model outputs deserialized against a known schema.

Pipeline integration pattern: On every pull request that modifies a prompt file in the repository, a CI job extracts the expected output schema from the new prompt version by running a representative set of inputs through the model and collecting the structured outputs. A second job deserializes these outputs against the baseline schema and the candidate schema, then feeds both schemas into the compatibility check prompt. The prompt returns a field-level compatibility matrix with breaking, additive, and deprecated change flags. The harness parses this JSON response and fails the CI check if any breaking change is detected unless an explicit [ALLOWED_BREAKING_CHANGES] list is provided in the PR description. For additive changes, the harness emits a warning but allows the pipeline to proceed. For deprecated fields, the harness checks whether a deprecation notice exists in the changelog and blocks if missing.

Validation and retry logic: The harness must validate the prompt's JSON output against a strict schema before acting on it. If the output fails to parse or is missing required fields like changes, breaking_changes, or compatibility_verdict, the harness retries once with a repair prompt that includes the raw output and the validation error. If the retry also fails, the harness fails the check with a human-review-required status. For high-risk API surfaces, the harness should also run a deserialization test: take the candidate prompt's actual outputs from a golden input set and attempt to deserialize them using the downstream service's client library. A compatibility matrix that claims no breaking changes but produces deserialization failures is a critical signal that the prompt's self-reported compatibility is unreliable.

Model choice and latency considerations: This prompt benefits from a model with strong structured reasoning and JSON output discipline. Use a model that supports strict JSON mode or structured outputs. Latency is not critical since this runs in CI, but cost can accumulate if run on every commit. Cache the baseline schema analysis and only re-run when the candidate schema changes. For teams running this across many prompt files, batch the compatibility checks and use a model router that falls back to a cheaper model for simple additive-change detections. Always log the full prompt input, raw output, parsed compatibility matrix, and pipeline decision to an audit trail for post-release analysis. If the prompt is used in a regulated context, require a human sign-off on any change that the prompt flags as potentially breaking, even if the harness allows it through on a technicality.