JSON Schema Compliance Judge Prompt Template

Use this prompt when you need an automated, structured verdict on whether an LLM-generated payload conforms to a formal JSON Schema. The core job-to-be-done is catching malformed outputs that a deterministic parser might accept but that will break downstream logic—think near-miss enum values like 'shipped' instead of 'shipped', a number rendered as a string, or a missing required field in a deeply nested object. This judge is for API platform teams, ingestion pipeline owners, and agent developers who are building guard loops around LLM calls and need a machine-readable pass/fail signal with field-level diagnostic detail before data hits a database, client application, or another model.

The ideal deployment point is in a post-generation validation layer, such as a CI/CD pipeline for prompt testing, a streaming validation step before a database write, or a retry loop where a failed verdict triggers a self-correction prompt. The judge prompt expects two inputs: a [CANDIDATE_PAYLOAD] and a [JSON_SCHEMA]. It returns a structured verdict containing a top-level pass boolean, an array of errors with JSON Path locations, and a summary string. This is not a prompt for generating content; it is a gate. It is most valuable when the cost of a malformed output is high—corrupted analytics, a broken user-facing feature, or a cascading failure in an agent's tool-call chain.

Do not use this prompt when a standard, deterministic JSON Schema validator like ajv in Node.js or jsonschema in Python can do the job. If your schema is strict, your types are unambiguous, and your payloads are machine-generated without the subtle coercion errors typical of LLMs, a code-based validator will be faster, cheaper, and perfectly accurate. This judge prompt is for the gray-area failures that deterministic validators miss or handle poorly, such as a string that looks like an enum value but has a trailing space, or a boolean written as the string 'true'. Reserve it for the layer that catches what your standard parser lets through. Before wiring this into production, prepare a test set of known malformed payloads to measure the judge's false-positive and false-negative rates against your specific schema.

This template is designed to be used as a system instruction for an LLM judge. Its job is to take a JSON Schema and a target JSON payload, then produce a deterministic, structured report. The report must clearly state whether the payload is valid, and if not, it must pinpoint every violation with a JSON path, the constraint that was broken, and the offending value. This is not a prompt for generating JSON; it is a prompt for auditing it.

textCopy
You are a strict JSON Schema compliance judge. Your only job is to validate a provided JSON payload against a provided JSON Schema and return a structured report.

## Inputs
- [JSON_SCHEMA]: The JSON Schema to validate against.
- [TARGET_PAYLOAD]: The JSON payload to validate.

## Task
1. Parse and understand the [JSON_SCHEMA].
2. Validate the [TARGET_PAYLOAD] against every constraint in the schema, including but not limited to:
   - Required field presence.
   - Type correctness (string, number, integer, boolean, array, object, null).
   - Enum and const value adherence.
   - Pattern (regex) matching for strings.
   - Minimum/maximum, minLength/maxLength, minItems/maxItems constraints.
   - Nested object and array structure.
3. Produce a JSON output that conforms exactly to the [OUTPUT_SCHEMA] below.

## [OUTPUT_SCHEMA]
{
  "verdict": "PASS" | "FAIL",
  "summary": "A one-sentence summary of the validation result.",
  "errors": [
    {
      "path": "$.field_name or $.array[0].nested_field",
      "constraint": "The specific schema constraint that was violated (e.g., 'required field missing', 'type mismatch: expected integer, got string').",
      "found": "The actual value found in the payload.",
      "expected": "The expected value or type as defined by the schema."
    }
  ]
}

## [CONSTRAINTS]
- If the payload is valid, the 'errors' array must be empty.
- Report every violation you find. Do not stop at the first error.
- Use standard JSONPath notation for the 'path' field.
- If the payload is not valid JSON, return a single error with path "$" and constraint "invalid JSON syntax".
- Do not add any commentary outside the JSON output.

To adapt this template, replace the placeholders with your specific inputs. For a CI/CD pipeline, [JSON_SCHEMA] would be your canonical schema file, and [TARGET_PAYLOAD] would be the LLM's latest output. The [OUTPUT_SCHEMA] is itself a contract; your application code should parse the judge's response and check the verdict field. If the judge itself produces malformed JSON, your harness must catch that as a system-level error, not a payload failure. For high-stakes applications, always log the full judge response for debugging false positives or negatives.

The JSON Schema Compliance Judge is designed to sit inside an automated validation layer, not as a one-off manual check. In a typical API product, the LLM generates a structured output, and this judge prompt acts as a pre-ingestion gate before the payload reaches downstream parsers, databases, or customer-facing responses. The judge receives the raw LLM output and the target JSON Schema, then returns a structured verdict with field-level error locations. This verdict should be treated as a machine-readable signal—your application code reads the pass boolean and routes the payload accordingly, not the human operator.

To wire this into an application, wrap the prompt in a validation function that accepts llm_output and schema as arguments. After calling the model, parse the judge's JSON response and check the top-level pass field. If pass is true, forward the original llm_output to the next stage. If pass is false, log the full errors array with field paths, expected types, and actual values for debugging. Implement a configurable retry loop: on failure, feed the judge's error report back to the generating LLM as correction context and re-validate. Set a maximum of 2-3 retries before escalating to a dead-letter queue or human review. For high-throughput systems, consider running the judge on a smaller, faster model (e.g., a fine-tuned 8B parameter model) for initial screening, reserving a larger model for ambiguous cases where the judge's confidence is below a threshold you define.

Evaluation and monitoring are critical because the judge itself can produce false positives (flagging valid JSON as invalid) and false negatives (missing real schema violations). Maintain a golden test set of known-valid and known-invalid payloads with ground-truth labels. Run this test set against the judge on every prompt or model change, and track precision and recall over time. Log every judge verdict alongside the original LLM output, the schema version, the model used, and the retry count. This trace data lets you diagnose whether failures originate in the generating model, the judge prompt, or the schema definition itself. When the judge flags a failure, always preserve the raw LLM output before any repair attempt—never discard evidence that could explain a downstream incident.