Default Value Population Array Prompt for Missing Fields

This prompt is for data pipeline builders and platform engineers who must ingest incomplete source data into a downstream system with a strict schema. The core job-to-be-done is transforming an array of sparse or inconsistent records into a clean, homogeneous array where every missing or null field is explicitly populated with a predefined default value. The ideal user is an engineer integrating an LLM into an ETL, data migration, or API ingestion workflow where source data quality is unreliable, but the target database, API contract, or analytics table cannot tolerate missing keys or null type ambiguity.

You should use this prompt when you have a default map—a known specification of what value each field should take when the source omits it—and you need the model to apply that map consistently across every record in a batch. This is not a prompt for inferring defaults from context or for imputing statistical best guesses. The model's job is to follow the provided default map exactly, distinguishing between a field that is genuinely missing (populate the default) and a field that is intentionally null (preserve the null if the schema allows it). The prompt works best when paired with a post-generation validation harness that checks for missing keys, type mismatches against the default map, and records where the model hallucinated a value instead of using the specified default.

Do not use this prompt when you need statistical imputation, when the default value depends on complex business logic better handled in application code, or when the source data volume is so large that per-record LLM processing is cost-prohibitive. For high-throughput pipelines, consider pre-processing records with deterministic rules first and reserving the LLM only for records that fail rule-based population. For regulated data, always log the before and after state of every populated field and require human review of the default map before it is applied to production data.

The following prompt template is designed to be copied directly into your AI harness. It instructs the model to process an array of source records and apply a provided default value map to any missing, null, or undefined fields. The template uses square-bracket placeholders for all dynamic inputs, allowing you to swap in your own schemas, data, and constraints without rewriting the core instruction logic. This prompt is the starting point for building a reliable data normalization step in your ingestion pipeline.

textCopy
You are a precise data normalization engine. Your task is to process an input array of records and ensure every record conforms to a target schema by populating missing or null fields with explicit default values.

## INPUT DATA
[INPUT_ARRAY]

## DEFAULT VALUE MAP
[DEFAULT_MAP]

## TARGET SCHEMA
[OUTPUT_SCHEMA]

## CONSTRAINTS
[CONSTRAINTS]

## INSTRUCTIONS
1. Iterate over every record in [INPUT_ARRAY].
2. For each record, check every field defined in [OUTPUT_SCHEMA].
3. If a field is missing (key not present) or its value is null, populate it with the corresponding default value from [DEFAULT_MAP].
4. If a field is present and has a non-null value, keep the original value. Do not overwrite it.
5. If a field is explicitly marked as null in the input and [CONSTRAINTS] specifies that intentional nulls should be preserved, do not populate a default for that field.
6. Ensure the type of the populated default value matches the type specified in [OUTPUT_SCHEMA]. Do not coerce types.
7. Return the complete array of records as a valid JSON array. Do not include any additional commentary, markdown fences, or text outside the JSON structure.
8. If a required field in [OUTPUT_SCHEMA] has no default in [DEFAULT_MAP] and is missing from the input, include an error object for that record in a separate `_errors` array.

## OUTPUT FORMAT
{
  "records": [ ... ],
  "_errors": [
    {
      "record_index": 0,
      "field": "field_name",
      "reason": "No default available for required field"
    }
  ]
}

To adapt this template for your own pipeline, replace the placeholders with concrete values. [INPUT_ARRAY] should be your source JSON array. [DEFAULT_MAP] is a JSON object mapping field names to their default values (e.g., {"status": "pending", "count": 0}). [OUTPUT_SCHEMA] should describe the expected fields and their types, either as a JSON Schema snippet or a plain-text field list. Use [CONSTRAINTS] to define special rules, such as preserving intentional nulls, skipping fields not in the schema, or enforcing strict type matching. If your workflow is high-risk—such as financial or healthcare data—add a human review step before the output is ingested into a system of record. Always validate the output array against your schema programmatically after generation, and log any records that appear in the _errors array for manual inspection.

This prompt is designed to sit inside a data ingestion or cleaning pipeline where incomplete source records arrive and must be normalized before downstream storage. The implementation harness should treat the prompt as a stateless transformation step: incomplete JSON array in, fully populated JSON array out. Because the prompt relies on a provided default-value map, the application layer must construct that map dynamically from a configuration store, a schema registry, or a metadata service before each call. Do not hardcode defaults inside the prompt template itself—keep them as a separate [DEFAULT_MAP] input that can be updated without changing the prompt text.

Wire the prompt into your application with a validate-then-retry loop. After the model returns the populated array, run a structural validator that checks: (1) every record has all fields listed in the default map, (2) no field contains null unless the default map explicitly permits null for that field, (3) field types match the expected types from the default map (string defaults produce strings, integer defaults produce integers), and (4) the output array length matches the input array length. If validation fails, construct a retry prompt that includes the original input, the default map, the failed output, and a specific error message describing which records and fields failed. Limit retries to 2 attempts before routing the batch to a human review queue or a dead-letter store for manual inspection. Log every attempt with the input hash, output hash, validator errors, and retry count for observability.

For model choice, prefer structured-output-capable models (GPT-4o with response_format, Claude 3.5 Sonnet with tool use set to a typed array schema, or Gemini 1.5 Pro with controlled JSON mode). If using a model without native structured output support, add a post-generation JSON repair step using a library like json_repair before validation. In high-throughput pipelines, batch multiple small arrays into fewer large calls to reduce API overhead, but keep individual batch sizes under 50 records to avoid field-drift across long outputs. Always include a circuit breaker that stops processing if the failure rate exceeds 10% across a rolling window of 100 calls—this catches silent prompt degradation or upstream schema changes before they corrupt your data store.