Tool Output Schema Compliance Check Prompt

This prompt is for platform engineers and AI quality engineers who need to validate that a tool's runtime output actually conforms to its declared schema. The job-to-be-done is automated contract verification: you have a JSON schema that defines what a tool should return, and you have the raw payload the tool did return. The prompt acts as a compliance auditor, comparing the two and producing a structured report that flags type mismatches, missing required fields, null violations, and unexpected fields. This is a gatekeeping prompt—it should sit in your CI/CD pipeline, pre-deployment test harness, or runtime monitoring loop before a malformed tool response corrupts an agent's context.

Use this prompt when you are integrating a new tool, updating a tool's contract, or monitoring production tool calls for schema drift. It is designed for JSON-serializable outputs where you have a formal schema (JSON Schema, OpenAPI parameter definitions, or a custom contract spec). The ideal user has access to both the schema artifact and the actual tool response, and needs a machine-readable compliance report to feed into automated gates. Do not use this prompt when you lack a formal schema to compare against, when the tool output is unstructured text without a type contract, or when you need to validate the semantic correctness of the output rather than its structural compliance. For semantic validation, pair this with a separate evaluation prompt.

The prompt requires three inputs: the declared output schema, the actual tool response payload, and a set of compliance rules (e.g., strict mode that fails on unexpected fields, or lenient mode that only flags missing required fields). The output is a structured compliance report with per-field findings, a pass/fail summary, and a severity classification suitable for automated decisioning. Before deploying this into a production gate, run it against a golden dataset of known-compliant and known-noncompliant responses to calibrate your thresholds. If the tool is in a regulated domain, always route failures to a human review queue rather than auto-blocking the pipeline.

This prompt template is designed to be dropped into an automated validation pipeline that runs after every tool invocation in your test harness. It takes the tool's declared output schema and the actual response payload, then produces a machine-readable compliance report. Use it as a gate before allowing tool outputs to re-enter the agent's context window—catching type mismatches, missing required fields, null violations, and unexpected fields before they cause silent failures downstream.

textCopy
You are a schema compliance validator for agent tool outputs. Your job is to compare an actual tool response against its declared output schema and produce a structured compliance report.

## INPUTS

**Declared Output Schema:**
[OUTPUT_SCHEMA]

**Actual Tool Response:**
[ACTUAL_RESPONSE]

**Tool Name:** [TOOL_NAME]
**Invocation ID:** [INVOCATION_ID]
**Strictness Level:** [STRICTNESS_LEVEL]

## TASK

Compare the actual tool response against the declared output schema and identify every deviation. Check for:

1. **Type mismatches** — field values that don't match the declared type (string, number, boolean, array, object, enum).
2. **Missing required fields** — fields declared as required that are absent from the response.
3. **Null violations** — null values in fields declared as non-nullable.
4. **Unexpected fields** — fields present in the response but absent from the schema.
5. **Enum violations** — values that fall outside the declared enum set.
6. **Format violations** — values that don't match declared format constraints (date-time, email, URI, pattern).
7. **Range violations** — numeric values outside declared minimum/maximum bounds.
8. **Length violations** — string or array lengths outside declared minLength/maxLength or minItems/maxItems.
9. **Nested object violations** — recursive application of all checks to nested objects and arrays of objects.

## STRICTNESS LEVELS

- **strict**: Report every deviation as an error. Unknown fields are errors.
- **lenient**: Report type and required-field violations as errors. Unknown fields are warnings. Null in non-nullable fields is a warning if a default exists.
- **permissive**: Only report type mismatches and missing required fields as errors. Everything else is informational.

## OUTPUT FORMAT

Return a JSON object with this exact structure:

```json
{
  "tool_name": "string",
  "invocation_id": "string",
  "compliance_status": "pass" | "fail" | "warning",
  "strictness_level": "string",
  "total_checks": <number>,
  "errors": [
    {
      "field_path": "string (dot-notation path to the violating field)",
      "check_type": "type_mismatch" | "missing_required" | "null_violation" | "unexpected_field" | "enum_violation" | "format_violation" | "range_violation" | "length_violation",
      "expected": "string (what the schema requires)",
      "actual": "string (what the response contained)",
      "severity": "error" | "warning" | "info"
    }
  ],
  "warnings": [
    {
      "field_path": "string",
      "check_type": "string",
      "expected": "string",
      "actual": "string",
      "severity": "warning"
    }
  ],
  "info": [
    {
      "field_path": "string",
      "check_type": "string",
      "message": "string",
      "severity": "info"
    }
  ],
  "summary": "string (one-sentence summary of findings)"
}

CONSTRAINTS

• Do not hallucinate fields that don't exist in either the schema or the response.
• If the schema is malformed or unparseable, return compliance_status "fail" with a single error describing the schema parse failure.
• If the response is not valid JSON, return compliance_status "fail" with a single error describing the parse failure.
• For nested objects, use dot-notation field paths (e.g., "user.address.city").
• For array items, use bracket notation (e.g., "items[0].name").
• If no violations are found, return compliance_status "pass" with empty errors, warnings, and info arrays.
• Do not include the original schema or response in your output—only the compliance report.

To adapt this template, replace the square-bracket placeholders with your pipeline's runtime values. The [OUTPUT_SCHEMA] should be the full JSON Schema or OpenAPI schema object for the tool's response. The [ACTUAL_RESPONSE] should be the raw response body from the tool invocation—don't pre-process or sanitize it, as that would mask the violations you're trying to catch. Set [STRICTNESS_LEVEL] based on your tolerance: use strict in pre-deployment gates, lenient in integration test suites, and permissive only when you're monitoring production and don't want to block on warnings. Wire the output compliance_status into your CI/CD or test harness so that fail blocks the pipeline and warning triggers a notification. For high-risk tools that mutate state or access sensitive data, always use strict and require human review of any fail status before the agent proceeds.

This prompt is designed to operate as a programmatic validation step, not a one-off manual check. The most effective harness wraps the prompt in a script or microservice that receives a tool contract (JSON Schema or OpenAPI fragment) and a captured tool response, then returns a structured compliance report. Because the prompt itself produces JSON, the harness should parse the output and act on the status field—treating non_compliant as a hard failure and compliant_with_warnings as a soft failure that may block deployment depending on your team's policy. The harness should never silently discard a non_compliant result; if the tool output violates its declared schema, downstream agents will eventually fail in ways that are harder to debug.

Validation and retry logic should be layered. First, validate that the prompt's own JSON output parses correctly and contains the required fields (status, violations, checked_fields, summary). If parsing fails, retry once with the same inputs—model output format errors are transient more often than content errors. If the retry also produces unparseable output, log the raw response and escalate for human review rather than silently passing the check. For the compliance verdict itself, do not retry: a non_compliant result means the tool output genuinely violates the schema, and re-prompting will not change the underlying data. Instead, route the violation report to the tool owner or the team that deployed the tool version. Model choice matters here: use a model with strong JSON-mode support and reliable schema-following behavior. GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro all perform well on this structured comparison task. Avoid smaller or older models that may conflate similar field names or miss subtle type mismatches like integer vs number.

Integration points depend on where tool outputs are captured. In a pre-deployment pipeline, run this check against a representative sample of tool responses collected from staging or synthetic test runs. In a production observability harness, sample a percentage of live tool calls, run the compliance check asynchronously, and emit the report to your monitoring stack (Datadog, Grafana, OpenTelemetry spans) with the tool name, version, and violation count as dimensions. For high-risk domains—healthcare, finance, safety-critical operations—add a human approval gate: any non_compliant result must be acknowledged before the tool version can remain in production. The harness should also track compliance trends over time; a rising count of compliant_with_warnings results often signals an impending breaking change in the tool's actual behavior before the contract is formally updated. Wire the prompt's output into your existing CI checks (GitHub Actions, GitLab CI, Jenkins) by exiting with a non-zero code on non_compliant and emitting the violation summary as an annotation on the relevant build or deployment.