Field Presence and Required Attribute Audit Prompt

This prompt is for ingestion pipeline owners and platform engineers who need to verify that every required field exists in structured LLM outputs before data reaches downstream systems. It acts as a schema-aware gate that distinguishes between missing keys, null values, and empty strings, producing a per-field presence report against a registry of required and optional attributes. Use this when malformed JSON payloads break ETL jobs, API contracts, or database inserts, and you need an automated, model-graded audit that runs at inference time.

The ideal user is an engineer who already has a defined schema registry—a source of truth that declares which fields are required, which are optional, and what constitutes a valid value. This prompt is not a schema designer; it assumes you bring the contract. It is also not a type checker (see the sibling Type Correctness and Coercion Detection Prompt for that). Instead, it answers one question with high precision: 'Is every required field present, and are optional fields correctly absent or populated?' The output is a structured report you can feed into a validation gateway, an alerting system, or a repair loop. For example, if your customer_order object requires order_id, timestamp, and line_items, but the model returns order_id as null and omits timestamp entirely, this prompt will flag both failures distinctly—one as a null violation, the other as a missing key.

Do not use this prompt when you need to validate the semantic content of fields (e.g., 'Is this email address deliverable?'), when you need cross-field consistency checks (e.g., 'Does status: shipped require a non-null tracking_number?'), or when you are validating against a formal JSON Schema with type and format constraints. Those jobs belong to other prompts in the Format Compliance and Schema Validation group. This prompt is the first gate: presence before correctness. Wire it directly after LLM generation and before any downstream transformer, database writer, or API responder. If the presence report shows failures, route the payload to a repair prompt or a human review queue—never let it silently hit production.

The following prompt template is designed to be dropped directly into your evaluation harness. It instructs the model to act as a field-presence auditor, comparing a structured payload against a provided schema registry that defines required, optional, and nullable fields. The prompt is built to produce a deterministic, machine-readable report rather than conversational prose, making it suitable for automated CI/CD gates or pre-ingestion validation steps.

textCopy
You are a field-presence auditor for structured LLM outputs. Your task is to compare the provided [PAYLOAD] against the [SCHEMA_REGISTRY] and produce a per-field presence report.

## INPUT
[PAYLOAD]

## SCHEMA REGISTRY
[SCHEMA_REGISTRY]

## NULL-HANDLING POLICY
[NULL_HANDLING_POLICY]

## INSTRUCTIONS
1. Parse the [PAYLOAD] as [INPUT_FORMAT].
2. For every field defined in the [SCHEMA_REGISTRY], determine its presence status in the [PAYLOAD].
3. Classify each field into exactly one of the following categories:
   - **PRESENT_AND_VALID**: The field exists and its value is non-null and non-empty (unless the schema explicitly allows empty values).
   - **MISSING_REQUIRED**: The field is defined as required in the schema but is absent from the payload.
   - **NULL_WHEN_REQUIRED**: The field exists but its value is null, and the schema marks it as required and non-nullable.
   - **NULL_WHEN_OPTIONAL**: The field exists with a null value, and the schema permits null for this field.
   - **EMPTY_WHEN_REQUIRED**: The field exists with an empty value (empty string, empty array, empty object), and the schema requires a non-empty value.
   - **PRESENT_BUT_UNDEFINED**: The field exists in the payload but is not defined in the [SCHEMA_REGISTRY].
4. Apply the [NULL_HANDLING_POLICY] to resolve ambiguous cases where the schema and payload disagree on nullability intent.
5. Do not validate types, formats, or value ranges. Only audit field presence and null/empty status.

## OUTPUT FORMAT
Return a JSON object with the following structure:
{
  "audit_summary": {
    "total_schema_fields": <number>,
    "present_and_valid_count": <number>,
    "missing_required_count": <number>,
    "null_when_required_count": <number>,
    "null_when_optional_count": <number>,
    "empty_when_required_count": <number>,
    "present_but_undefined_count": <number>,
    "overall_pass": <boolean>
  },
  "field_results": [
    {
      "field_path": "<string using dot notation for nested fields>",
      "status": "<PRESENT_AND_VALID | MISSING_REQUIRED | NULL_WHEN_REQUIRED | NULL_WHEN_OPTIONAL | EMPTY_WHEN_REQUIRED | PRESENT_BUT_UNDEFINED>",
      "schema_definition": {
        "required": <boolean>,
        "nullable": <boolean>,
        "allow_empty": <boolean>
      },
      "observed_value_preview": "<truncated string representation of the value, max 100 chars>"
    }
  ]
}

## CONSTRAINTS
- Do not add commentary outside the JSON output.
- Use dot notation for nested field paths (e.g., "address.city" not "address['city']").
- If the [PAYLOAD] is not parseable as [INPUT_FORMAT], return a single error object: {"error": "UNPARSEABLE_PAYLOAD", "detail": "<reason>"}.
- Treat missing fields and null fields as distinct conditions per the [NULL_HANDLING_POLICY].

To adapt this template for your pipeline, replace the square-bracket placeholders with your actual inputs. The [SCHEMA_REGISTRY] should be a JSON object mapping field paths to their definitions (required, nullable, allow_empty). The [NULL_HANDLING_POLICY] should be a short text block resolving edge cases—for example, whether a field present with a null value counts as 'missing' when the schema says non-nullable. Wire the output JSON into a downstream validator that can block malformed payloads from reaching your ingestion pipeline or trigger a repair workflow when overall_pass is false.

This prompt is designed to sit directly after structured output generation and before any downstream database write or API dispatch. In a typical ingestion pipeline, the LLM generates a JSON payload, and this audit prompt receives both the raw output and the expected schema registry (a mapping of field names to their required/optional/nullable status). The harness should call this prompt synchronously as a validation gate: if the audit report shows any missing required fields, the pipeline must not proceed. Treat the audit prompt's output as a structured decision, not a suggestion.

To implement, wrap the prompt in a function that accepts generated_output (the raw JSON string from the primary LLM) and schema_registry (a JSON object defining each field's constraints). The function should inject these into the [OUTPUT_TO_AUDIT] and [SCHEMA_REGISTRY] placeholders. On return, parse the audit report and check for a has_violations boolean or scan the field_results array for any entry where status is missing and required is true. If violations exist, log the full audit report, increment a field_presence_failure metric, and route the record to a dead-letter queue for human review. For high-throughput systems, consider batching multiple outputs into a single audit call with a clear delimiter to reduce API overhead, but ensure each record's report is independently parseable.

Model choice matters here. This is a classification and comparison task, not a generative one, so prioritize models with strong instruction-following and low hallucination rates on structured analysis. A smaller, faster model (e.g., Claude Haiku or GPT-4o-mini) is often sufficient and keeps latency low for this synchronous gate. Do not use a creative or general-purpose model that might invent field statuses. Implement a simple retry with exponential backoff (max 3 attempts) if the audit prompt itself returns malformed JSON, but if the audit report is valid and shows violations, do not retry the audit—retry the primary generation step instead, or escalate. Always log the full prompt input and output for every audit failure to enable later analysis of schema drift or model behavior changes.

Avoid wiring this prompt as an asynchronous, fire-and-forget check. The value is in blocking bad data before it lands in your database or triggers downstream workflows. If your pipeline uses a streaming architecture, buffer the generated output until the audit verdict returns. For high-risk domains like finance or healthcare, add a human-in-the-loop step: if a required field is missing, surface the record in a review queue with the audit report pre-filled, rather than auto-discarding or auto-repairing. Finally, maintain your schema registry as a versioned artifact in the same repository as your prompts. When the schema changes, re-run your eval suite (see the Regression Testing pillar) to ensure the audit prompt's false-positive and false-negative rates haven't drifted.