Data Contract Validation Prompt for Schema Registry

This prompt is designed for data platform teams who need to validate AI-generated extraction output against a registered schema before publishing to downstream systems. It acts as a gate between extraction and ingestion, producing a pass/fail verdict with field-level violation details. Use it when your pipeline must enforce strict schema contracts, detect compatibility breaks, or prove data quality before records enter a data warehouse, event stream, or serving layer. This prompt assumes you already have an extracted payload and a target schema definition. It does not perform extraction or repair; it validates and reports.

The ideal user is a data engineer, platform operator, or integration developer responsible for a production ingestion pipeline. You should have a registered schema in a schema registry (such as Confluent Schema Registry, AWS Glue, or an internal contract store), a sample of AI-extracted records, and a clear understanding of your downstream system's tolerance for malformed data. The prompt works best when you can provide the full schema definition, including field types, required fields, enum constraints, and compatibility mode. It is not a substitute for programmatic schema validation libraries; it is a pre-flight check that catches semantic mismatches, type coercion failures, and structural drift that static validators might miss.

Do not use this prompt when you need to repair malformed records, perform extraction from raw text, or handle schema evolution decisions. It validates and reports violations but does not fix them. If your pipeline requires automatic type coercion, default value population, or field renaming, pair this prompt with a repair or normalization prompt. Also avoid using this prompt for real-time, low-latency validation on high-throughput streams where a native schema validator (like an Avro deserializer) is more appropriate. This prompt is best suited for batch validation gates, CI/CD checks on extraction quality, or human-in-the-loop review queues where a detailed, human-readable violation report adds value beyond a binary pass/fail.

This prompt is the core instruction set for a data contract validation step. It is designed to sit between your extraction pipeline and your ingestion target (Kafka topic, database table, API endpoint). The model receives an extracted payload and the target schema definition, then returns a structured verdict. Copy this template directly into your orchestration layer, replacing the square-bracket placeholders with values from your runtime context. The prompt enforces strict schema adherence, compatibility checks, and explicit null handling—critical for preventing bad data from poisoning downstream systems.

textCopy
You are a data contract validator. Your task is to validate a single extracted data payload against a registered target schema. You must produce a strict pass/fail verdict with detailed field-level violation information.

## INPUT
[EXTRACTED_PAYLOAD]

## TARGET SCHEMA
[SCHEMA_DEFINITION]

## SCHEMA COMPATIBILITY MODE
[COMPATIBILITY_MODE]

## VALIDATION RULES
1. Check that every field in the target schema is present in the extracted payload, unless the schema marks it as optional.
2. Verify that the data type of each field in the payload exactly matches the schema definition. Pay special attention to string vs. integer, boolean vs. string "true"/"false", and timestamp formats.
3. If [COMPATIBILITY_MODE] is "BACKWARD", allow extra fields in the payload not present in the schema. If "FORWARD", reject payloads with fields missing from the schema. If "FULL", reject both extra and missing fields.
4. For nullable fields, explicitly distinguish between a missing key, a null value, and an empty string. Flag any ambiguity.
5. For enum fields, verify the value is one of the allowed members.
6. For nested objects and arrays, recursively validate structure and types.

## OUTPUT FORMAT
Return a single JSON object with the following structure. Do not include any text outside the JSON.
{
  "verdict": "PASS" | "FAIL",
  "schema_id": "string",
  "compatibility_mode": "string",
  "violations": [
    {
      "field_path": "string (e.g., root.field or root.nested[0].key)",
      "violation_type": "MISSING_REQUIRED | TYPE_MISMATCH | ENUM_VIOLATION | EXTRA_FIELD | NULL_AMBIGUITY | NESTED_VIOLATION",
      "expected": "string (description of what was required)",
      "actual": "string (description of what was found)",
      "severity": "ERROR | WARNING"
    }
  ],
  "warnings": [
    {
      "field_path": "string",
      "message": "string (non-blocking issue, e.g., deprecated field used)"
    }
  ]
}

## CONSTRAINTS
- Do not hallucinate violations. Only report actual discrepancies.
- If the payload is completely valid, return an empty violations array and verdict "PASS".
- If the schema definition itself is malformed or unparseable, return a single violation with field_path "schema" and violation_type "INVALID_SCHEMA".

To adapt this template, start by wiring [EXTRACTED_PAYLOAD] to the output of your extraction step—this is the JSON your model or pipeline produced. [SCHEMA_DEFINITION] should be the canonical schema from your registry (Avro, JSON Schema, or Protobuf descriptor rendered as JSON). Set [COMPATIBILITY_MODE] based on your registry's configured compatibility type. For production use, always run this validation before publishing to a topic or inserting into a database. If the verdict is FAIL, route the payload to a dead-letter queue and trigger an alert. For high-risk domains like finance or healthcare, add a human review step for any TYPE_MISMATCH or NULL_AMBIGUITY violation before automated remediation is attempted.

This prompt is designed to sit between your extraction pipeline and your schema registry's compatibility check. It should be called after a record is assembled but before it is published to the registry or written to a downstream topic. The harness must supply the prompt with the extracted record, the target schema, and the compatibility mode. The model returns a structured verdict that your application code can act on: pass the record through, block it, or route it for human review.

Wrap the prompt in a lightweight service or serverless function that performs pre-validation before calling the model. Check that the record is valid JSON and that the schema is parseable. If either is malformed, fail fast and log a PRE_VALIDATION_FAILURE event—do not waste tokens on a prompt that cannot succeed. After receiving the model's response, parse the JSON verdict and validate it against a strict output schema: { "verdict": "PASS"|"FAIL", "violations": [{"field": "...", "issue": "...", "severity": "ERROR"|"WARN"}] }. If the model's output does not parse or does not match this shape, retry once with a simplified prompt that asks only for the verdict and a single-line reason. If the retry also fails, escalate to a human review queue with the original record, schema, and raw model output attached.

For high-throughput pipelines, consider batching records that share the same schema version. Send one prompt per batch, asking the model to validate each record and return an array of verdicts. This reduces per-record token overhead and lets you amortize the schema context across multiple validations. However, if a single record in the batch causes the model to return malformed JSON, the entire batch fails. Implement a batch-splitting fallback: if a batch validation fails, split it into individual records and retry each one separately. Log batch-level and record-level metrics separately so you can distinguish schema-wide issues from record-specific problems.

Model choice matters here. Use a model with strong JSON mode and instruction-following for the primary validation path. For the retry path, use a faster, cheaper model that can still produce a simple pass/fail verdict. If your schema registry enforces strict compatibility (e.g., Avro FULL or Protobuf backward compatibility), add a pre-check in your harness that compares the record's field set against the schema's required fields before calling the model. This catches obvious structural mismatches without spending tokens. Finally, log every verdict—pass or fail—with the schema version, record ID, model version, and timestamp. This audit trail is essential for debugging schema evolution issues and proving data quality to downstream consumers.