Tool Output Contract Validation Prompt

This prompt is a pre-processing guard that sits between tool execution and agent context assembly. Its job is to catch field presence violations, type mismatches, unexpected nulls, and schema drift before they cause silent agent failures downstream. AI quality engineers and integration developers run this prompt inside a validation harness that inspects every tool response against the tool's declared output schema. The prompt produces a structured validation report—not a repaired output—so the calling system can decide whether to retry, fall back, escalate, or block the response from reaching the agent's context window.

The ideal user is an integration developer or AI quality engineer who already has a tool schema (OpenAPI, MCP, or internal contract) and a raw tool response. You wire this prompt into a pipeline step that executes after every tool call but before the agent assembles its next context. The prompt requires three inputs: the tool's declared output schema, the actual tool response, and a set of validation rules (field presence, type checking, null constraints, enum membership). Do not use this prompt for defining tool schemas from scratch, for repairing malformed outputs, or for runtime type coercion—those are separate workflows covered by schema definition and output repair prompts respectively.

The most common failure mode is schema ambiguity: if the declared schema uses loose types like object or string without constraints, the validator will pass responses that still break downstream logic. Always pair this prompt with a schema annotation step that enriches sparse schemas with min/max ranges, regex patterns, and required field flags before validation. For high-risk domains—finance, healthcare, security operations—add a human review gate when the validation report shows unexpected nulls in required fields or type mismatches in critical paths. Start by copying the prompt template below, then adapt the validation rules to match your tool contracts.

This prompt template is the core of the Tool Output Contract Validation workflow. It instructs the model to act as a schema validator, comparing a raw tool response against its declared output contract. The model must produce a structured validation report that downstream systems—such as agent orchestrators, middleware, or logging pipelines—can parse and act on. The template uses square-bracket placeholders that you replace with actual values at runtime: the tool's declared output schema, the raw response payload, and any domain-specific constraints or risk thresholds.

textCopy
You are a tool output contract validator. Your job is to compare a raw tool response against its declared output schema and produce a structured validation report.

## INPUTS
- Declared Output Schema: [OUTPUT_SCHEMA]
- Raw Tool Response: [RAW_RESPONSE]
- Additional Constraints: [CONSTRAINTS]
- Risk Level: [RISK_LEVEL]

## VALIDATION RULES
1. Check that every field declared as required in the schema is present in the response.
2. For each present field, verify that its type matches the schema declaration (string, number, boolean, array, object, null).
3. Flag any field present in the response that is not declared in the schema.
4. For array fields, check that each element conforms to the declared item schema.
5. For object fields, recursively validate nested properties against the nested schema.
6. Identify unexpected null values in fields declared as non-nullable.
7. Check enum fields against allowed values.
8. Validate string formats where format constraints are declared (date-time, email, uri, regex patterns).
9. Check numeric fields against min/max constraints if declared.
10. For partial success or batch responses, validate the envelope structure and per-item status fields.

## OUTPUT FORMAT
Return a JSON object with this exact structure:
{
  "validation_passed": boolean,
  "schema_version": string,
  "validation_timestamp": string,
  "summary": {
    "total_fields_checked": number,
    "passed": number,
    "warnings": number,
    "failures": number
  },
  "field_results": [
    {
      "field_path": string,
      "status": "pass" | "warning" | "fail",
      "expected_type": string,
      "actual_type": string,
      "expected_value_constraint": string | null,
      "actual_value_sample": string,
      "issue_description": string | null,
      "severity": "critical" | "high" | "medium" | "low"
    }
  ],
  "unexpected_fields": [string],
  "missing_required_fields": [string],
  "type_mismatches": [
    {
      "field_path": string,
      "expected": string,
      "actual": string
    }
  ],
  "null_violations": [string],
  "enum_violations": [
    {
      "field_path": string,
      "received_value": string,
      "allowed_values": [string]
    }
  ],
  "recommendations": [string],
  "agent_guidance": {
    "can_proceed": boolean,
    "fallback_action": string,
    "fields_to_ignore": [string],
    "retry_suggested": boolean,
    "escalation_required": boolean
  }
}

## INSTRUCTIONS
- If RISK_LEVEL is "critical", treat all warnings as failures.
- If RISK_LEVEL is "low", allow type-coercible mismatches as warnings rather than failures.
- For any field marked as containing PII or credentials in CONSTRAINTS, mask the actual_value_sample.
- If the RAW_RESPONSE is not valid JSON, set validation_passed to false and explain the parse error.
- If the response indicates a tool-level error, validate the error structure against the schema's error contract.
- Do not hallucinate fields. Only report on fields actually present or declared.

To adapt this template, replace each placeholder with live data at invocation time. [OUTPUT_SCHEMA] should contain the complete JSON Schema or equivalent type definition for the tool's expected response. [RAW_RESPONSE] is the actual payload returned by the tool, exactly as received. [CONSTRAINTS] can include field-level rules such as PII markers, required format patterns, or business logic invariants not captured in the schema itself. [RISK_LEVEL] accepts one of critical, high, medium, or low and controls the strictness of the validation pass/fail threshold. Before wiring this into production, run the output through a JSON schema validator to confirm the model returned the declared structure—if the validation report itself is malformed, your pipeline cannot trust it. For high-risk domains, always route validation_passed: false results to a human review queue rather than silently discarding or retrying.

The Tool Output Contract Validation Prompt is not a standalone utility; it is a pre-processing gate that sits between tool execution and agent consumption. In a production harness, every tool response passes through this validation prompt before the agent's reasoning context receives it. The harness is responsible for calling the prompt, parsing its structured validation report, and deciding whether to pass the response through, repair it, retry the tool call, or escalate. This section covers the integration points, failure modes, and operational controls needed to make validation reliable at scale.

Integration architecture. The harness should implement a validate_tool_output(tool_name, declared_schema, raw_output) function that wraps the prompt call. Inputs include the tool's declared output schema (from your tool registry or MCP server manifest), the raw tool response, and optional context like the tool call arguments. The prompt returns a structured validation report with pass/fail status, a list of violations (field presence, type mismatch, unexpected nulls, extra fields), and a confidence score. The harness must parse this report and route based on severity: pass → forward to agent context; minor violations (e.g., extra fields) → strip and forward with a warning log; major violations (e.g., missing required fields, type mismatches) → block and trigger retry or fallback; critical violations (e.g., completely malformed output) → escalate to human review or circuit-break the tool. Never forward a failed validation silently.

Retry and repair loops. When validation fails, the harness should not blindly retry the same tool call with identical arguments. Instead, implement a staged recovery: (1) If the failure is a known transient issue (e.g., partial JSON truncation), attempt a repair prompt that reconstructs the output from the partial response. (2) If repair fails or is inappropriate, retry the tool call with explicit error context injected into the tool's input—many tools can self-correct when told what was wrong. (3) If retries exhaust (cap at 2-3 attempts), fall back to a degraded response that tells the agent the tool is unavailable and provides whatever partial data is safe to use. Log every stage with the tool name, attempt number, violation summary, and disposition. This trace data is essential for debugging silent agent failures and for tuning tool schemas that produce consistently invalid outputs.

Model choice and latency budget. Validation prompts are high-frequency, low-latency-critical operations. Use a fast, cheap model for this gate—GPT-4o-mini, Claude Haiku, or a fine-tuned small model are appropriate. The prompt's output schema is rigid and the task is classification-heavy, so larger models add cost without proportional accuracy gains. Set a strict timeout (500-1000ms) on the validation call; if the validator times out, treat it as a fail-open or fail-closed decision based on your risk tolerance. For high-risk domains (finance, healthcare, security), fail closed and escalate. For lower-risk internal tools, fail open with a warning log to avoid blocking the agent. Cache validation results per unique (tool_name, schema_version, output_hash) tuple to avoid re-validating identical responses in retry loops or repeated tool calls.

Observability and schema drift detection. The harness must emit structured logs and metrics from every validation call: tool name, schema version, pass/fail rate, violation type distribution, and latency. Aggregate these metrics to detect schema drift—when a tool's actual outputs systematically diverge from its declared contract. A rising rate of unexpected_null violations on a field that was previously reliable signals a backend change. A spike in type_mismatch violations after a tool deployment indicates a breaking change. Wire these metrics into your alerting pipeline so that tool contract violations surface before they cause agent failures. Additionally, store a sample of failed validations (with PII redacted) for periodic review by the platform team to improve schema definitions and validation rules.

What to avoid. Do not use the validation prompt as a substitute for proper tool schema design—garbage schemas produce garbage validation. Do not run validation on every agent turn if the tool output hasn't changed; cache aggressively. Do not let the validation harness mutate the tool output without recording the original and the changes in an audit log. And do not treat validation as optional in production: an agent consuming invalid tool output will hallucinate, misreason, or silently produce incorrect results that are far harder to debug than a blocked validation gate.