API Error Response Body Standard Prompt

This prompt is designed for API platform teams and SREs who need to enforce a consistent error contract across multiple microservices. The primary job-to-be-done is automated conformance checking: you provide a target specification—which may include required fields like error_code, message, request_id, timestamp, and a field-level validation error structure—and an actual error response payload. The prompt acts as a strict auditor, comparing the provided payload against the specification and producing a structured report that flags missing fields, type mismatches, and structural violations. It also supports an optional RFC 7807 Problem Details compliance check, making it suitable for teams that have adopted or are migrating toward that standard.

This is not a prompt for generating error messages from scratch or for designing your error schema. It assumes you already have a defined internal standard and need to validate existing responses against it. Ideal users are platform engineers integrating this check into a CI/CD pipeline, QA engineers verifying new endpoints before release, or SREs auditing production error responses for drift. The prompt requires two concrete inputs: a [TARGET_SPECIFICATION] that defines the expected schema (as a JSON Schema, TypeScript interface, or plain-text field list), and an [ERROR_RESPONSE_PAYLOAD] that is the actual JSON response body to audit. Without both, the prompt cannot produce a useful conformance report. Do not use this prompt when you need to evaluate the quality of an error message's prose or its helpfulness to an end user—that requires a different evaluation rubric focused on clarity and actionability, not structural conformance.

Before integrating this prompt into an automated pipeline, establish a clear source of truth for your target specification and version it alongside your API contracts. The prompt's output is a structured conformance report that can be parsed by downstream tooling, but you should implement a validation layer in your application code to confirm the report's JSON schema before acting on its conclusions. For high-risk releases, pair the automated check with a human review step that examines any FAIL results to determine whether the violation is a genuine contract breach or an acceptable deviation. Start by running this prompt against a golden set of known-valid and known-invalid error responses to calibrate your pass/fail thresholds before deploying it as a gate in your release pipeline.

This prompt template is designed to be pasted directly into your AI harness, such as an internal QA tool, a CI/CD pipeline step, or a manual review interface. Its purpose is to evaluate a single REST API error response body against a defined standard, producing a structured conformance report. The prompt is self-contained, requiring only that you replace the square-bracket placeholders with your specific standard, the response payload, and the request context before execution.

textCopy
You are an API standards auditor. Your task is to evaluate the provided API error response body against the [ERROR_BODY_STANDARD]. The standard defines required fields, their types, and their descriptions. You must also check for optional compliance with RFC 7807 Problem Details if [RFC_7807_CHECK] is set to true.

## Input Data
- **Error Response Body:**
  ```json
  [ERROR_RESPONSE_BODY]

• Request Context:
  • HTTP Method: [REQUEST_METHOD]
  • Endpoint: [ENDPOINT_PATH]
  • Response Status Code: [STATUS_CODE]

Evaluation Criteria

1. Field Presence: Check if all fields required by [ERROR_BODY_STANDARD] are present in the response body.
2. Type Conformance: Verify that the value of each present field matches the expected type (e.g., string, integer, array) defined in [ERROR_BODY_STANDARD].
3. Content Quality: For key fields like message and details, assess if the content is actionable for a developer. A good message explains what went wrong and hints at the next step, rather than just restating the status code.
4. RFC 7807 Compliance (Optional): If [RFC_7807_CHECK] is true, check for the presence of type, title, status, detail, and instance fields. Note that status should match the HTTP response status code.

Output Schema

Return your findings as a single JSON object conforming to this schema: { "overall_conformance": "pass" | "fail", "rfc_7807_conformance": "pass" | "fail" | "not_evaluated", "field_checks": [ { "field_name": "string", "requirement": "required" | "optional", "status": "pass" | "fail", "expected_type": "string", "actual_type": "string", "diagnosis": "A brief explanation of the finding, or null if passed." } ], "content_quality_notes": [ "A list of strings describing issues with message clarity, missing context, or unhelpful information." ], "summary": "A final one-sentence assessment of the error response's usefulness for a developer." }

Constraints

• Base your evaluation strictly on the provided [ERROR_RESPONSE_BODY] and [ERROR_BODY_STANDARD]. Do not infer missing context.
• If a field is missing, set its actual_type to "missing".
• If the response body is not valid JSON, set overall_conformance to "fail" and explain the parsing error in the summary.

To adapt this template, start by defining your [ERROR_BODY_STANDARD] as a clear, inline list of field names, types, and descriptions within the prompt. For a more dynamic implementation, you could replace the placeholder with a reference to a tool that fetches your company's OpenAPI spec. After pasting the prompt, wire the output JSON into a validator in your harness. A critical next step is to build an eval set of known good and bad error responses to test the prompt's accuracy before it gates a CI pipeline, ensuring it doesn't produce false negatives for acceptable variations in error formatting.

This prompt is designed to be integrated into an API gateway's contract-testing suite or a pre-release QA gate. The primary implementation pattern is a CI/CD pipeline job that triggers on OpenAPI spec changes or new API version tags. The job should fetch the live or mock API responses for a curated list of error scenarios (e.g., 400, 401, 404, 422, 500), feed each response body into the prompt alongside your internal error standard document, and collect the structured conformance reports. For manual audits, the same prompt can be used in a review tool where an API designer pastes a response body and receives immediate feedback.

For automated CI integration, build a harness that: 1) Validates the model's output schema before accepting results. The prompt requests a JSON report with a conformance_score and a violations array. Your harness must parse this JSON and fail the pipeline if conformance_score falls below a configurable threshold (e.g., 0.9) or if any violation.severity == 'critical' is present. 2) Implements retry logic with exponential backoff for model API calls, as transient network errors or rate limits should not fail the build. 3) Logs the full prompt, response, and parsed report as a CI artifact for auditability. This is critical for proving that a release met the error body standard. 4) Uses a deterministic model version (e.g., gpt-4o-2024-08-06 or claude-3-5-sonnet-20241022) pinned in your pipeline configuration to prevent model behavior drift from causing spurious build failures. 5) For high-risk APIs (payments, healthcare, identity), adds a manual approval gate that requires a human to review and sign off on the generated conformance report before the release can proceed, even if the automated score passes.

When wiring this into a real application, avoid treating the model's output as the final source of truth for regulatory compliance. The prompt is a detective control, not a preventive one. It catches deviations from your standard but does not enforce the standard at the code level. The strongest implementation pairs this prompt with a code-level schema validation library (e.g., an OpenAPI response validator) that blocks non-conforming responses before they reach the model. Use the prompt as a secondary, semantic check for issues that schema validation misses, such as unhelpful human-readable messages, missing request IDs despite the field being present, or RFC 7807 type URIs that are not resolvable. For the RFC 7807 compliance option, ensure your harness explicitly sets the [STANDARD] placeholder to your organization's Problem Details profile, and consider adding a post-prompt step that uses curl or a link checker to verify any type URIs returned in the error body are actually reachable.