Ingestion Readiness Gate Check Prompt

This prompt is designed for data engineers and platform operators who need an automated, deterministic gate before extracted records land in a downstream system. The job-to-be-done is validating that a batch of extracted records—typically JSON objects from an LLM extraction pipeline—conforms to a target schema, satisfies required-field constraints, and meets type-correctness rules before ingestion. The ideal user is someone managing a data pipeline where schema contract violations, missing fields, or type mismatches can silently corrupt a data warehouse, break an API, or poison an analytics model. They need a pass/fail decision with a structured list of blocking issues, not a probabilistic summary.

Use this prompt when you have a defined ingestion contract—a JSON Schema, a database table definition, or an API specification—and you need to programmatically reject records that violate it. The prompt expects a [TARGET_SCHEMA] describing required fields, types, and constraints, and a [RECORDS_BATCH] containing the extracted payloads. It is not a fuzzy deduplication tool, a data quality profiler, or a semantic anomaly detector. Do not use it for exploratory data analysis, for scoring extraction confidence, or for making subjective judgments about data fitness. It is a strict gate check: if a record violates the schema, it fails. The output is a machine-readable gate decision with a list of blocking issues per record, suitable for logging, alerting, or routing to a dead-letter queue.

Before deploying this prompt, ensure your ingestion contract is unambiguous. The prompt cannot infer missing constraints or guess your intent. If your schema allows nulls but your downstream system does not, the gate will pass records that later fail. Always pair this prompt with a validation harness that enforces the same schema in application code as a second layer of defense. For high-stakes pipelines—financial ledgers, clinical data, compliance records—require human review of any batch that fails the gate before retrying or discarding records. The next section provides the prompt template you can adapt and embed directly into your ingestion workflow.

The following prompt template implements an ingestion readiness gate check. It takes extracted records and a target schema contract, then evaluates each record for schema compliance, required field presence, type correctness, and constraint satisfaction. The prompt produces a structured pass/fail decision with blocking issues enumerated, making it suitable for insertion into data pipeline automation before any downstream system receives the payload.

textCopy
You are an ingestion gate validator. Your job is to check whether extracted data records meet the requirements of a downstream ingestion contract before loading.

## INPUT

**Target Schema Contract:**
[SCHEMA_CONTRACT]

**Extracted Records:**
[EXTRACTED_RECORDS]

**Ingestion Constraints:**
[CONSTRAINTS]

## TASK

For each record in the extracted records, evaluate it against the target schema contract and constraints. Produce a gate decision for each record.

### Checks to Perform

1. **Schema Compliance:** Does the record contain only fields defined in the target schema? Flag any unexpected fields.
2. **Required Field Presence:** Are all required fields present? Required fields are those marked `required: true` in the schema contract.
3. **Type Correctness:** Does each field value match its declared type (string, integer, float, boolean, date, enum, array, object)?
4. **Constraint Satisfaction:** Does each field value satisfy any declared constraints (min/max length, min/max value, pattern regex, enum membership, nullability rules)?
5. **Null Handling:** For nullable fields, is null a valid value? For non-nullable fields, is the value non-null and non-empty?

### Output Format

Return a JSON object with this structure:

```json
{
  "gate_result": "PASS" | "FAIL" | "PARTIAL",
  "records_evaluated": <integer>,
  "records_passed": <integer>,
  "records_failed": <integer>,
  "blocking_issues": [
    {
      "record_index": <integer>,
      "field": "<field_name>",
      "issue_type": "MISSING_REQUIRED" | "TYPE_MISMATCH" | "CONSTRAINT_VIOLATION" | "UNEXPECTED_FIELD" | "NULL_VIOLATION",
      "expected": "<expected_value_or_type>",
      "actual": "<actual_value_or_type>",
      "severity": "BLOCKING" | "WARNING"
    }
  ],
  "warnings": [
    {
      "record_index": <integer>,
      "field": "<field_name>",
      "message": "<description>",
      "severity": "WARNING"
    }
  ],
  "summary": "<one-sentence summary of gate decision>"
}

Decision Rules

• PASS: All records pass all checks with zero blocking issues.
• FAIL: One or more records have blocking issues that prevent ingestion.
• PARTIAL: Some records pass and some fail. List which records passed and which failed.

Important

• Do not modify or repair any record values. Only evaluate.
• If a field is missing from the source but the schema marks it as optional, do not flag it.
• If the schema contract is ambiguous, note it as a warning but do not fail the record.
• Return only the JSON output. No additional commentary.

To adapt this template, replace [SCHEMA_CONTRACT] with your target schema definition in JSON Schema format or a simplified field specification listing field names, types, required flags, and constraints. Replace [EXTRACTED_RECORDS] with the array of records output by your extraction pipeline. Replace [CONSTRAINTS] with any additional business rules not captured in the schema, such as cross-field validation rules, allowed value ranges, or temporal ordering requirements. For high-stakes ingestion pipelines, always pair this gate check with human review for records that produce warnings or partial passes, and log every gate decision with the record index and timestamp for audit traceability.

The Ingestion Readiness Gate Check prompt is designed to sit directly upstream of a data loading operation. It should be called after extraction and normalization are complete, but before any INSERT, MERGE, or PUBLISH command is executed. The prompt receives a batch of extracted records and a downstream schema contract, then returns a structured gate decision. In a production harness, this prompt is not a one-off manual check; it is an automated control point that can block a pipeline stage, route records to a dead-letter queue, or trigger an alert to the data platform team.

The application layer must enforce the gate decision, not just log it. If the prompt returns "gate": "FAIL", the harness should halt the ingestion step and prevent any records from reaching the target system. For a "gate": "PASS_WITH_WARNINGS" decision, the harness can proceed but must persist the warning details alongside the ingested batch for later audit. The harness should validate the prompt's JSON output against a strict schema before acting on it—if the model returns malformed JSON or missing required fields like gate or blocking_issues, treat that as a FAIL with an "error": "INVALID_GATE_OUTPUT" reason. Implement a retry loop with a maximum of 2 attempts for malformed outputs, using a repair prompt that includes the raw response and the expected schema. If retries are exhausted, escalate to a human review queue with the full payload attached.

Model choice matters here. Use a model with strong JSON mode and instruction-following capabilities, such as gpt-4o or claude-3.5-sonnet, and set response_format to json_object or use the provider's structured output feature with the gate schema defined as a tool. Temperature should be set to 0 to minimize variance in pass/fail decisions. Log every gate check result—including the prompt version, model version, input record count, gate decision, and all blocking issues—to an observability system. This creates an audit trail for compliance and enables trend analysis on ingestion failure patterns. Avoid running this gate check on batches larger than 50 records; if the extraction pipeline produces larger batches, split them before invoking the prompt to keep the context window manageable and the gate reasoning precise.