API Reference Accuracy Audit Prompt Template

This prompt is built for technical writing teams and API platform engineers who need to validate that their published API reference documentation matches the actual implementation. The core job-to-be-done is catching parameter mismatches, incorrect types, missing endpoints, and stale descriptions before they reach developers. The ideal user has access to a source of truth—typically an OpenAPI specification, a runtime behavior log, or a set of live API responses—and needs a structured, machine-readable audit report that can be parsed by downstream CI/CD tooling or a documentation platform.

You should use this prompt before a release, during a documentation migration, or as a gating step in a deployment pipeline. It expects structured inputs: a documentation snapshot (the claims) and a source-of-truth artifact (the evidence). The prompt is most effective when the documentation is machine-readable (e.g., scraped HTML, Markdown, or a structured JSON representation) and the source of truth is a valid OpenAPI spec or a log of actual request/response pairs. Do not use this prompt for subjective prose reviews, style guide adherence, or code sample compilability checks—those are separate audit workflows. It also should not be used when the source of truth is incomplete or when the documentation is purely conceptual with no corresponding API surface.

Before running this prompt, ensure you have a clear output schema defined so the audit report can be consumed by your automation. If the audit flags a high-severity mismatch—such as a missing required parameter or a documented endpoint that no longer exists—the workflow should block the release or trigger a human review. For lower-severity findings like stale descriptions, you may route the report for automatic ticket creation. The next step after reading this section is to prepare your input artifacts and define the severity thresholds that matter for your release process.

The following prompt template is designed to be copied directly into your AI harness, notebook, or evaluation runner. It uses square-bracket placeholders for all dynamic inputs, allowing you to swap in different API documentation sources, specifications, and runtime behavior logs without rewriting the core instruction. The template is structured to produce a consistent, machine-readable audit report that can be fed into downstream validation tools or human review queues.

textCopy
You are an API documentation auditor. Your task is to compare the provided API reference documentation against the authoritative source of truth and produce a structured accuracy audit report.

## INPUTS
- **Documentation to Audit:**
[DOCUMENTATION_CONTENT]

- **Authoritative Source (choose one or more):**
[OPENAPI_SPEC]
[RUNTIME_BEHAVIOR_LOG]
[SOURCE_CODE_ANNOTATIONS]

- **Audit Scope:**
[AUDIT_SCOPE]

## OUTPUT SCHEMA
Return a JSON object with the following structure:
{
  "audit_metadata": {
    "audit_id": "string",
    "timestamp": "ISO8601",
    "scope": "string"
  },
  "findings": [
    {
      "severity": "critical | high | medium | low",
      "category": "parameter_mismatch | missing_endpoint | incorrect_type | stale_description | broken_example | missing_error_code | schema_violation | deprecation_gap | security_concern | other",
      "location": "string (section, paragraph, or line reference)",
      "documented_claim": "string (exact text from docs)",
      "actual_behavior": "string (what the source of truth shows)",
      "impact": "string (what breaks for the consumer)",
      "suggested_fix": "string (actionable correction)"
    }
  ],
  "summary": {
    "total_findings": "number",
    "by_severity": {"critical": 0, "high": 0, "medium": 0, "low": 0},
    "by_category": {},
    "coverage_gaps": ["string (endpoints or sections missing entirely)"]
  }
}

## CONSTRAINTS
- Only report findings where documented claims diverge from the authoritative source.
- Do not flag stylistic preferences, word choice, or formatting unless they cause functional confusion.
- For each finding, quote the exact documented text and the exact source-of-truth evidence.
- If a parameter, endpoint, or error code exists in the source but is absent from the docs, flag it as a coverage gap with severity "high" or "critical."
- If the documentation is correct, return an empty findings array and note coverage completeness in the summary.
- Do not invent findings. If evidence is ambiguous, note uncertainty and assign severity "low."

## EXAMPLES
[FEW_SHOT_EXAMPLES]

## RISK LEVEL
[RISK_LEVEL]

To adapt this template, replace each square-bracket placeholder with your actual inputs. For [DOCUMENTATION_CONTENT], paste the full text of the API reference page or section you are auditing. For the authoritative source, provide at least one of [OPENAPI_SPEC], [RUNTIME_BEHAVIOR_LOG], or [SOURCE_CODE_ANNOTATIONS]. The [AUDIT_SCOPE] placeholder should specify which endpoints, parameters, or documentation sections to examine—use "all" for a full audit or a comma-separated list for targeted checks. The [FEW_SHOT_EXAMPLES] placeholder accepts one or two example finding objects to calibrate severity and category assignment. Set [RISK_LEVEL] to "low," "medium," or "high" to adjust the auditor's sensitivity; high-risk mode flags even minor discrepancies, while low-risk mode only reports breaking changes.

Before running this prompt in production, validate that the output JSON conforms to the schema using a JSON Schema validator. For high-stakes audits—such as those covering authentication, billing, or security endpoints—always route findings through human review before publishing corrections. If you are integrating this prompt into a CI/CD pipeline, pair it with a retry mechanism that re-runs the audit when the model returns malformed JSON or an empty response. Store each audit result with the prompt version, model identifier, and input hashes for traceability.

This prompt is designed to be the reasoning core of an automated documentation audit system, not a one-off chat interaction. The implementation harness must treat the LLM as an intelligent comparator that produces structured findings, while the surrounding application code handles evidence collection, validation, and integration with existing doc platforms. The prompt expects a specific input shape—a documentation snippet paired with a live API reference (OpenAPI spec, runtime response, or SDK signature)—and produces a JSON audit report. The harness is responsible for assembling that input correctly, validating the output schema, and deciding what to do with each finding.

Input Assembly: Before calling the prompt, the harness must gather two data sources: the documentation claim and the ground truth. For endpoint audits, extract the relevant section from your docs (Markdown, HTML, or a CMS API) and pair it with the corresponding OpenAPI operation object or a captured response from the live API. For parameter checks, isolate the parameter table from the docs and the parameter definition from the spec. The harness should normalize both into the [DOCUMENTATION_SNIPPET] and [API_SPEC_OR_RESPONSE] placeholders as structured text. If you're auditing at scale, build a mapping file that links doc page URLs to spec operation IDs so the harness can iterate without manual pairing. Validation Layer: The prompt returns a JSON object with a findings array. The harness must validate this against a strict schema before accepting it. Each finding must have severity, category, location, expected, actual, and recommendation fields. If the model returns malformed JSON, use a repair prompt or a retry with the error message appended. For high-stakes audits (security auth flows, breaking changes), flag any critical severity finding for human review before publishing.

Integration Points: Wire the harness into your CI/CD pipeline so that every docs PR triggers an audit on changed pages. Use a model with strong structured output support (GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro) and set temperature=0 for consistency. Log every audit result—including the input pair, raw model response, and validated output—to an observability platform for traceability. Failure Modes to Watch: The most common production failure is a mismatch between the documentation snippet's scope and the spec fragment's scope. If you pass an entire endpoint page but only one parameter's spec definition, the model may flag false positives. The harness should chunk documentation into logical units (one endpoint, one parameter group, one auth flow) and pair each chunk with its precise spec counterpart. A second failure mode is spec staleness: if your OpenAPI file is outdated, the audit will flag correct docs as wrong. The harness should verify the spec's info.version or fetch a fresh runtime response before running the audit. Next Step: After implementing the harness, run it against a known-good documentation set to calibrate severity thresholds and identify categories where the model over-flags or under-flags issues. Use those results to tune the [CONSTRAINTS] placeholder with category-specific guidance before scaling to the full doc set.