API Contract Test from OpenAPI Spec Prompt

This prompt is designed for API platform teams, backend engineers, and QA engineers who need to validate that a live API implementation conforms to its OpenAPI specification. The core job-to-be-done is generating executable contract tests that verify status codes, response schemas, required headers, and error formats for every endpoint and operation defined in the spec. You should use this when you have an OpenAPI 3.x document and a running API instance, and you need automated proof that the implementation matches the contract before shipping changes or onboarding new consumers.

The ideal user brings two concrete inputs: a complete OpenAPI specification file and the base URL of a running API instance. The prompt works best when the spec includes detailed schema definitions, example payloads, and documented error responses—not just happy-path success cases. For teams running in CI/CD pipelines, this prompt can be wired into a pre-release gate that blocks deployments when contract violations are detected. The generated tests should be treated as executable documentation that proves the API behaves as advertised, not as a replacement for integration tests that verify business logic, database state, or service orchestration.

Do not use this prompt when you need to validate business rules, database mutations, or multi-service workflows. It strictly validates the HTTP interface contract—status codes, headers, response body structure, and error formats. If your OpenAPI spec is incomplete, outdated, or lacks error response definitions, the generated tests will be incomplete and may produce false confidence. For high-risk APIs handling payments, health data, or authentication, always pair these contract tests with human review of the generated assertions and additional integration tests that verify the underlying business logic.

This prompt instructs an AI coding agent to read an OpenAPI specification and produce a suite of contract tests. The tests verify that an API implementation honors the defined status codes, response schemas, required headers, and error formats. The prompt is designed to be copied directly into your agent interface or LLM workbench. Before using it, ensure you have the OpenAPI spec accessible in your repository context and have decided on the target test framework and language.

codeCopy
You are an expert test engineer writing contract tests from an OpenAPI specification.

## INPUT
- OpenAPI specification: [OPENAPI_SPEC_PATH]
- API base URL: [BASE_URL]
- Target test framework: [TEST_FRAMEWORK]
- Target language: [LANGUAGE]

## TASK
Generate a contract test suite that validates the API implementation against the specification. For each endpoint defined in the spec, produce tests that verify:
1. **Status codes**: The implementation returns the documented success and error status codes for valid and invalid requests.
2. **Response schemas**: The response body matches the documented JSON schema, including required fields, types, and enum values.
3. **Headers**: Required response headers are present and match documented patterns.
4. **Error formats**: Error responses follow the documented error schema for 4xx and 5xx responses.
5. **Content negotiation**: If the spec defines multiple content types, test that the correct content type is returned.

## CONSTRAINTS
- Follow the existing test patterns and conventions found in [TEST_DIRECTORY].
- Use the test fixtures and helpers already present in the repository.
- Do not generate tests for deprecated endpoints or operations.
- Mark any test that depends on mutable server state with a `@stateful` annotation or comment.
- Include setup and teardown logic where required to isolate test state.
- Add a comment above each test linking to the specific path and operation in the OpenAPI spec.

## OUTPUT_SCHEMA
Return the test suite as a set of files organized by resource or tag from the OpenAPI spec. Each test file must:
- Import the project's existing test utilities.
- Define a test class or describe block named after the resource.
- Include at least one happy-path test and one error-path test per operation.
- Use parameterized tests for endpoints that accept multiple valid input combinations.

## EXAMPLES
Reference the existing test patterns in [TEST_DIRECTORY] for style, assertion library usage, and naming conventions.

## RISK_LEVEL
This is a test generation task. The generated tests must be reviewed by a human before merging. Flag any test that makes destructive state changes or depends on production data with a `@review` annotation.

How to adapt this template: Replace each square-bracket placeholder with concrete values before execution. [OPENAPI_SPEC_PATH] should point to the spec file in your repository. [BASE_URL] is the target environment for test execution. [TEST_FRAMEWORK] and [LANGUAGE] must match your project's stack (e.g., pytest and Python, or Jest and TypeScript). [TEST_DIRECTORY] should reference the directory containing existing tests so the agent can learn conventions. If your project lacks existing tests, remove the pattern-adherence constraint and provide a short example inline. For high-risk APIs (payments, health data, auth), add a [REVIEW_REQUIRED] flag and route all generated tests through human code review before execution. The output schema section can be tightened by appending a specific file-naming convention or directory structure requirement.

This prompt is designed to run as a step in a CI pipeline triggered by OpenAPI spec changes or scheduled contract validation runs. The harness should invoke the model with the spec content, existing test patterns, and target language constraints, then validate the generated test code before it enters the repository. Treat the model output as a draft that requires structural verification, not a final artifact.

Pipeline integration: Store the OpenAPI spec file path and target test directory as pipeline variables. Before calling the model, extract the spec version, endpoint inventory, and schema definitions to include as structured context. After generation, run a validation gate that checks: (1) the output parses as valid code in the target language, (2) every endpoint in the spec has at least one corresponding test case, (3) status code assertions cover the documented responses, and (4) response schema assertions match the spec schemas. Use a JSON schema validator on the model's structured output before extracting the test code block. If validation fails, retry once with the validation errors appended to the prompt as [PREVIOUS_ERRORS]. If the retry also fails, surface the failure to the PR author with the specific endpoints and schemas that are missing or mismatched.

Model choice and cost: Use a model with strong code generation and structured output capabilities. For OpenAPI specs under 10K tokens, a single prompt call is sufficient. For larger specs, split by endpoint group or tag and run parallel generation jobs, then merge results. Cache the spec-to-test mapping to avoid regenerating tests for unchanged endpoints. Human review gate: For APIs marked as production or externally consumed, require a human reviewer to approve the generated tests before merging. The review should focus on business logic assertions the model cannot infer from the spec alone, such as rate limiting behavior, authentication edge cases, and stateful workflow sequences. Log every generation with the spec version, model version, validation results, and reviewer decision for auditability.

Failure modes to monitor: The most common production failure is schema drift where the generated tests assert against an outdated spec version. Always pin the spec version in the test file header and compare against the live API during test execution. Another failure mode is the model generating tests that pass against the spec but fail against the real implementation because the spec is aspirational rather than descriptive. Flag endpoints where the spec and implementation diverge for separate remediation. Finally, watch for the model hallucinating endpoints or schemas not present in the spec—the endpoint coverage validator catches this, but log the frequency to track model reliability over time.