Tool Contract Version Compatibility Test Prompt

Platform engineers and tool providers use this prompt when a tool contract (API schema, function definition, or MCP server descriptor) changes and they need to understand the impact on agent behavior before deploying the new version. The prompt compares two versions of a tool contract and produces a migration impact report that classifies changes as additive, deprecating, or breaking, with specific guidance on which agent behaviors will fail. Use this before cutting a release, during CI/CD contract validation, or when an upstream dependency announces a version bump.

This prompt is designed for pre-deployment analysis, not runtime validation. It expects two complete tool contract representations—typically JSON Schema, OpenAPI fragments, or MCP tool descriptors—and produces a structured diff that goes beyond structural comparison to reason about semantic impact on agent decision-making. For example, a field rename that looks like a minor structural change will be correctly classified as breaking if agents rely on that field name for argument construction. The prompt requires both the old and new contract versions, and optionally accepts a list of known agent workflows that depend on the tool, which sharpens the impact analysis.

Do not use this prompt for runtime tool output validation or for testing agent behavior against mocked responses; those are separate playbooks. This prompt also does not generate test cases or execute actual tool calls—it performs static contract analysis. For workflows that require behavioral testing after the migration report, pair this with the Agent Tool Failure Injection Test Prompt or the Tool Contract Breaking Change Detection Prompt to build a complete pre-release validation pipeline. Always run this analysis before merging contract changes that affect production agents, and treat any breaking change finding as a release blocker until agent adaptations are confirmed.

This prompt template is designed to be pasted directly into your model. It instructs the model to act as a platform engineer analyzing two versions of a tool contract—an older version currently used by agents and a newer version being proposed or released. The goal is to produce a structured report that identifies breaking changes, deprecation gaps, and behavioral differences that would cause agent failures if not addressed. Replace every square-bracket placeholder with your actual tool contracts and constraints before running the prompt.

textCopy
You are a platform engineer responsible for tool contract compatibility. Your task is to compare two versions of a tool contract and produce a migration impact report for agent developers.

## INPUTS

### Tool Name
[TOOL_NAME]

### Version A (Current/Previous Contract)
[VERSION_A_CONTRACT]

### Version B (New/Proposed Contract)
[VERSION_B_CONTRACT]

### Additional Constraints
[CONSTRAINTS]

## OUTPUT SCHEMA

Return a JSON object with the following structure:
{
  "tool_name": "string",
  "version_a": "string",
  "version_b": "string",
  "summary": {
    "total_changes": "number",
    "breaking_changes": "number",
    "deprecation_gaps": "number",
    "additive_changes": "number",
    "behavioral_differences": "number",
    "migration_urgency": "low | medium | high | critical"
  },
  "changes": [
    {
      "id": "string",
      "category": "breaking | deprecation_gap | additive | behavioral",
      "severity": "low | medium | high | critical",
      "location": "string (field path, endpoint, or parameter)",
      "description": "string (what changed)",
      "agent_impact": "string (how this breaks or alters agent behavior)",
      "migration_action": "string (what agent developers must do)",
      "detection_test": "string (how to verify this change is handled correctly)"
    }
  ],
  "unaffected_dependencies": ["string (list of agent behaviors that remain safe)"],
  "recommended_migration_order": ["string (sequenced steps for safe migration)"],
  "rollback_risk": "string (what happens if agents revert to Version A after migration)"
}

## INSTRUCTIONS

1. Parse both contracts completely. Identify every field, parameter, endpoint, type, enum value, default value, required/optional flag, and description.
2. Classify each difference into exactly one category:
   - **breaking**: Agent code or prompts using Version A will fail or produce incorrect results with Version B.
   - **deprecation_gap**: A field or behavior is deprecated in Version B but agents still reference it, creating future risk.
   - **additive**: New fields, parameters, or endpoints added in Version B that agents may optionally adopt.
   - **behavioral**: The contract structure is unchanged but semantics, side effects, or output meaning differ.
3. For each breaking or deprecation_gap change, provide a concrete detection test that agent developers can run.
4. Assign migration_urgency based on:
   - **critical**: Breaking changes that will cause silent data corruption or security issues.
   - **high**: Breaking changes that will cause immediate agent failures.
   - **medium**: Deprecation gaps or behavioral differences with workarounds.
   - **low**: Additive changes only, no immediate action required.
5. If no changes are detected, return an empty changes array with migration_urgency "low" and a note that contracts appear identical.
6. Do not invent changes. Only report differences you can identify from the provided contracts.

After pasting this prompt, replace [TOOL_NAME] with the tool's identifier, [VERSION_A_CONTRACT] and [VERSION_B_CONTRACT] with the full contract definitions (including argument schemas, output schemas, descriptions, and endpoint signatures), and [CONSTRAINTS] with any additional rules such as "ignore description wording changes" or "flag any enum value removal as critical." The output is structured JSON suitable for automated migration gates, CI checks, or agent team notifications. For high-risk tools where breaking changes could cause data loss or security incidents, always route the output through human review before deploying updated agents.

This prompt is designed to run as a pre-commit or CI gate before a new tool contract version is published to an internal registry or MCP server. It takes two structured tool contracts (old and new) and produces a migration impact report. The primary integration point is a scripted pipeline that fetches the current and proposed contract schemas, passes them to the model, and then parses the structured output to decide whether to block the release, warn downstream agent developers, or auto-approve additive-only changes.

Wiring the prompt into a pipeline requires a thin harness that handles three stages. First, input assembly: serialize both tool contracts into a consistent JSON format that includes the tool name, description, parameter schema, output schema, and any declared error codes. The prompt's [OLD_CONTRACT] and [NEW_CONTRACT] placeholders expect this normalized representation. Second, model invocation: use a model with strong structured output support (GPT-4o, Claude 3.5 Sonnet, or equivalent) with response_format set to json_object and the output schema enforced. Set temperature=0 to maximize deterministic classification of breaking vs. non-breaking changes. Third, output validation: parse the model's JSON response and validate that every flagged change includes a change_type from the expected enum (breaking, deprecating, additive, behavioral), a field_path, and a non-empty impact_description. Reject any response that fails schema validation and retry once with the validation errors injected into the [CONSTRAINTS] block.

Failure modes to guard against include the model missing implicit behavioral changes (e.g., a timeout default changing from 30s to 5s without a schema type change) and over-flagging cosmetic differences as breaking. Mitigate the first by including runtime behavior defaults in the contract representation, not just type schemas. Mitigate the second by post-processing the output through a simple rule: if a flagged change has change_type: breaking but the field_path is a description or examples field, downgrade it to additive or deprecating unless the description change alters a documented constraint. Log every model invocation with the full prompt, response, and parsed report for auditability. For high-risk tool contracts (those affecting payment, auth, or data deletion), add a human approval gate that requires an engineer to confirm breaking changes before the report is published to agent developers. The next step is to feed the validated migration impact report into your agent test harness so that any agent depending on the old contract is re-tested against the new contract before deployment.