Prompt Output Contract Compliance Check Prompt

This prompt is for API and platform teams who need to verify that structured AI outputs conform to a defined schema contract before those outputs reach downstream parsers, databases, or user-facing surfaces. The primary job-to-be-done is automated regression testing: after changing a prompt, model, or generation parameter, you run a batch of known test cases and confirm that every output still matches the expected JSON schema, field types, enum values, and required fields. The ideal user is an engineering lead or infrastructure engineer embedding this check into a CI/CD pipeline where malformed outputs are a release blocker, not a post-deployment surprise.

Use this prompt when you have a formal output contract—a JSON Schema, a typed object specification, or a strict set of field rules—and a golden dataset of inputs with known expected output shapes. It is designed for high-reliability workflows where partial compliance is still a failure: a missing required field, a wrong enum value, or a type mismatch must be caught and reported with exact field paths. The prompt produces a structured violation report, not a quality score. It does not judge whether the content is good, only whether it is valid against the contract. This makes it suitable for gating deployments of customer-facing APIs, internal microservices, and any system where downstream code assumes a specific output shape.

Do not use this prompt for evaluating semantic quality, tone, factual accuracy, or instruction adherence beyond structural compliance. It will not tell you if the answer is helpful, only if it is well-formed. If you need to check both structure and content, pair this prompt with a separate rubric-based evaluation step. Also avoid using this prompt for loosely specified outputs where the schema is advisory rather than contractual—it will over-flag acceptable variations. For high-risk domains such as healthcare, finance, or legal workflows, always route violation reports to human review before taking automated action, and ensure your golden dataset covers edge cases that could produce dangerous but technically valid outputs.

This prompt template is designed to be dropped into a regression testing pipeline where every model output must conform to a known schema contract before it reaches downstream consumers. It accepts the raw model output, the expected JSON schema, and optional constraints such as required fields, enum values, and type rules. The prompt instructs the model to act as a contract compliance checker—not a general-purpose validator—so it focuses exclusively on structural and type-level violations rather than content quality or semantic correctness. Use this when you need machine-readable violation reports that can block a release or trigger a repair loop.

textCopy
You are a contract compliance checker for structured outputs. Your only job is to validate whether a given output conforms to a specified schema contract. Do not evaluate content quality, factual accuracy, or style. Report only structural violations.

## INPUT
Model Output:
```json
[OUTPUT]

Expected Schema:

jsonCopy
[SCHEMA]

CONSTRAINTS

• Required fields: [REQUIRED_FIELDS]
• Enum constraints: [ENUM_CONSTRAINTS]
• Type constraints: [TYPE_CONSTRAINTS]
• Additional rules: [ADDITIONAL_RULES]

INSTRUCTIONS

1. Parse the model output and the expected schema.
2. Check every field in the output against the schema for:
   • Missing required fields
   • Incorrect types (string vs number, object vs array, etc.)
   • Values outside allowed enum sets
   • Extra fields not defined in the schema (if [ALLOW_EXTRA_FIELDS] is false)
   • Null values in non-nullable fields
   • Array length violations if [ARRAY_CONSTRAINTS] are specified
3. For each violation, record:
   • The exact JSON path (e.g., $.user.address.zip)
   • The violation type (missing_required, type_mismatch, invalid_enum, extra_field, null_not_allowed, array_length)
   • The expected value or constraint
   • The actual value found
4. If no violations are found, return a single result with compliance_status set to "compliant".

OUTPUT FORMAT

Return a JSON object with this exact structure:

jsonCopy
{
  "compliance_status": "compliant" | "non_compliant",
  "total_violations": <integer>,
  "violations": [
    {
      "path": "<JSON path string>",
      "violation_type": "<type string>",
      "expected": "<constraint description>",
      "actual": "<value found>",
      "severity": "error" | "warning"
    }
  ],
  "validation_notes": "<optional summary string>"
}

RISK LEVEL

[RISK_LEVEL]

If RISK_LEVEL is "high", flag any violation as severity "error" and recommend blocking the release. If RISK_LEVEL is "medium", distinguish between blocking errors and non-blocking warnings based on whether the violation would break a downstream parser.

Adapt this template by replacing the square-bracket placeholders with your actual values. The [SCHEMA] placeholder should contain a complete JSON Schema object, not a reference or URL. For [REQUIRED_FIELDS], provide a comma-separated list of field paths that must be present. The [ENUM_CONSTRAINTS] and [TYPE_CONSTRAINTS] placeholders can be left empty if your schema already encodes these rules, but explicit constraints help smaller models catch violations they might otherwise miss. Set [ALLOW_EXTRA_FIELDS] to true only if your downstream consumers tolerate unknown fields without breaking. Before wiring this into a CI/CD pipeline, run it against a known set of compliant and non-compliant outputs to verify that the violation paths match your application's error-handling code. If you are validating outputs in a regulated domain, always route non-compliant results to a human review queue rather than auto-repairing silently.

This prompt is designed to run as a post-generation validation step inside a regression test harness, not as a standalone chat interaction. After each test case produces an output, the harness sends the output, the expected JSON schema, and any additional constraints to the compliance check prompt. The prompt returns a structured violation report that your pipeline can parse and act on. The primary integration point is between your test runner and your LLM invocation layer—treat this prompt as a deterministic gate that either passes the output forward or flags it for review.

Wire the prompt into your test harness by constructing a request payload with three required inputs: the generated output under test, the expected JSON schema (as a JSON Schema object or string), and a list of constraint rules such as required fields, enum membership, or type constraints. The model should return a JSON object with a compliant boolean and a violations array. Your harness should parse this response and fail the test case if compliant is false. For high-reliability pipelines, implement a retry layer: if the compliance check itself returns malformed JSON, retry up to two times with a stricter instruction variant that emphasizes valid JSON output. Log every compliance check result—including the input output, schema, and violation report—to an append-only audit store so you can trace regressions across prompt versions.

Model choice matters here. Use a model with strong JSON mode or structured output support (e.g., GPT-4o with response_format set to json_object, or Claude with tool-use mode forcing a typed response). Avoid models that struggle with exact field-path reporting, as violation paths like $.items[3].price must be precise enough for automated remediation. If your pipeline processes high-volume or latency-sensitive outputs, consider batching multiple compliance checks into a single request with an array of output-schema pairs, but cap batch size at 10–20 to avoid truncation. For regulated domains, always route compliant: false results to a human review queue before blocking deployment—the prompt can flag violations, but a human should confirm severity and decide whether to halt a release.