OpenAPI Spec-to-Documentation Consistency Prompt

This prompt is designed for API designers, technical writers, and platform engineers who need to verify that human-readable API documentation (Markdown, HTML, or a documentation portal) accurately reflects the machine-readable OpenAPI specification. The core job-to-be-done is preventing documentation drift by producing a structured diff report that flags mismatches in endpoint descriptions, parameter tables, schema definitions, and example payloads before they reach end users. The ideal user has access to both a valid OpenAPI spec and the rendered documentation, and needs a repeatable, automatable gate to ensure the two stay synchronized.

Use this prompt as a pre-release gate in your documentation workflow or as part of a CI/CD pipeline that triggers on spec changes or documentation deployments. The prompt expects two primary inputs: the raw OpenAPI specification (JSON or YAML) and the extracted text of the documentation page or section being audited. It works best when the documentation has clear endpoint boundaries—such as distinct sections for each operation—and when the spec is the authoritative source of truth. The output should be a structured report that can be parsed by downstream tooling, with severity levels for each finding so teams can block releases on critical mismatches or log warnings for minor inconsistencies.

Do not use this prompt when you lack a valid, complete OpenAPI spec, when the documentation format is purely unstructured prose without clear endpoint or parameter boundaries, or when you need runtime behavioral validation rather than spec-to-doc consistency. This prompt checks whether the docs say what the spec says—it does not verify whether the API actually behaves as specified. For runtime validation, pair this with a separate behavioral testing harness. Additionally, if your documentation is auto-generated from the spec, this prompt may produce false positives for intentional formatting differences; in those cases, a direct spec-to-render comparison tool is more appropriate than an LLM-based audit.

This prompt template is designed to be pasted directly into your AI harness. It instructs the model to act as an API documentation auditor, comparing a provided OpenAPI specification against a set of documentation pages. The model must identify discrepancies in endpoint descriptions, parameter tables, request/response schemas, and example payloads, then output a structured JSON diff report. All dynamic inputs are represented as square-bracket placeholders, which you must replace with real data before execution. The output format is strictly JSON to enable direct ingestion by downstream validation scripts, CI/CD pipelines, or review dashboards.

textCopy
You are an API documentation auditor. Your task is to compare the provided OpenAPI specification against the provided human-readable documentation and produce a structured consistency report.

## INPUTS
- OpenAPI Specification: [OPENAPI_SPEC]
- Documentation Pages: [DOCUMENTATION_PAGES]

## OUTPUT_SCHEMA
Return a single JSON object conforming to this structure:
{
  "summary": {
    "total_endpoints_in_spec": <integer>,
    "total_endpoints_documented": <integer>,
    "total_discrepancies": <integer>,
    "critical_discrepancies": <integer>,
    "warning_discrepancies": <integer>
  },
  "discrepancies": [
    {
      "id": "<string, unique discrepancy identifier>",
      "severity": "<critical | warning>",
      "category": "<missing_endpoint | missing_parameter | parameter_type_mismatch | missing_response_schema | schema_field_mismatch | example_payload_error | description_mismatch | missing_error_code | other>",
      "location": {
        "spec_path": "<string, JSON pointer to the relevant part of the OpenAPI spec>",
        "doc_reference": "<string, section heading or paragraph identifier in the documentation>"
      },
      "spec_value": "<string, the correct value from the OpenAPI spec>",
      "doc_value": "<string, the conflicting value found in the documentation>",
      "description": "<string, human-readable explanation of the discrepancy>",
      "suggested_fix": "<string, actionable suggestion to resolve the discrepancy>"
    }
  ],
  "undocumented_endpoints": [
    {
      "path": "<string>",
      "method": "<GET | POST | PUT | DELETE | PATCH>",
      "spec_summary": "<string, the endpoint's summary from the spec>"
    }
  ],
  "stale_documentation": [
    {
      "doc_reference": "<string>",
      "claimed_endpoint": "<string>",
      "issue": "<string, e.g., endpoint not found in spec>"
    }
  ]
}

## CONSTRAINTS
- [CONSTRAINTS]

## INSTRUCTIONS
1. Parse the OpenAPI specification to extract all paths, methods, parameters, request bodies, response schemas, and examples.
2. Parse the documentation pages to identify all described endpoints, parameters, schemas, and examples.
3. For every endpoint in the spec, verify its presence and accuracy in the documentation.
4. For every endpoint in the documentation, verify it exists in the spec.
5. Flag any mismatch as a discrepancy with the appropriate severity.
6. If no documentation is provided for an endpoint, list it under `undocumented_endpoints`.
7. If documentation describes an endpoint not in the spec, list it under `stale_documentation`.
8. Output ONLY the JSON object. No other text.

To adapt this template, replace the placeholders with your specific context. [OPENAPI_SPEC] should contain the full JSON or YAML text of your specification. [DOCUMENTATION_PAGES] should contain the full text of the documentation you are auditing, with clear section markers if possible. The [CONSTRAINTS] placeholder is where you inject specific rules, such as ignoring certain deprecated endpoints, focusing only on externally facing APIs, or setting a custom severity threshold for parameter description mismatches. After running the prompt, always validate the output against the provided JSON schema before passing it to any automated system. For high-stakes releases, a human reviewer should spot-check a sample of flagged discrepancies to ensure the model did not hallucinate a mismatch due to ambiguous documentation language.

This prompt is designed to operate as a diff engine between a source-of-truth OpenAPI specification and the rendered developer documentation. The implementation harness should treat the prompt as a comparison function: input the spec and the doc page, receive a structured inconsistency report, and then decide what to do with each finding. Because the output is a structured diff rather than freeform prose, the harness must validate the output schema before surfacing results to technical writers or API designers.

Wire the prompt into a CI/CD check that triggers on documentation pull requests or scheduled spec snapshots. The harness should: (1) fetch the latest OpenAPI spec from the canonical source (e.g., a published openapi.yaml or a build artifact); (2) extract the relevant documentation page from the PR diff or the live docs site; (3) populate the [OPENAPI_SPEC] and [DOCUMENTATION_PAGE] placeholders; (4) call the model with response_format set to the expected JSON schema; (5) validate the returned JSON against the schema, retrying once on parse failure; (6) filter findings by severity threshold (e.g., only error and warning for CI blocking, info for review queues); and (7) post the filtered report as a PR comment or ticket. For high-stakes APIs where documentation errors cause integration failures, require human approval on error-level findings before merging.

Common failure modes in the harness include: the model hallucinating parameter names not present in either the spec or the doc; the output exceeding token limits for large specs (mitigate by chunking the spec by endpoint or tag group); and false positives when the documentation intentionally simplifies spec details for readability. Build eval assertions that compare the model's flagged inconsistencies against a golden dataset of known spec-doc mismatches, measuring precision and recall. Log every comparison run with the spec version, doc version, model version, and the raw output for auditability. Start with a non-blocking advisory mode before promoting the check to a required CI gate.