Confidence-Weighted Record Assembly Prompt

This prompt is designed for data product teams who need to merge multiple extraction passes into a single, auditable record. The core job-to-be-done is conflict resolution: when two extraction runs produce different values for the same field, the prompt must weigh each candidate by its confidence score, select the most reliable value, and flag any conflicts that cannot be automatically resolved. The ideal user is a data engineer or pipeline architect integrating this prompt into an ETL workflow where downstream systems require a single source of truth, not a list of possibilities. Required context includes the raw text source, a list of field definitions, and at least two extraction passes with per-field confidence scores.

Use this prompt when you have multiple extraction passes from the same source document and need a defensible, weighted assembly. It is particularly valuable when extraction passes come from different models, different prompt strategies, or different points in time, leading to genuine disagreement. The prompt expects structured input: a schema definition, a list of extraction passes with field_name, value, and confidence (0.0–1.0) for each field, and a conflict resolution strategy such as 'highest confidence wins' or 'majority vote with confidence weighting'. Do not use this prompt for single-pass extraction with confidence annotation—that is a separate workflow covered by the Confidence-Annotated Field Extraction Prompt Template. Do not use it when the extraction passes are not aligned to the same schema, as the assembly logic depends on field-level comparison.

Before wiring this into production, define your conflict resolution rules explicitly. The prompt should produce a final record with a composite_confidence score, a resolution_log detailing which pass supplied each field value, and an unresolved_conflicts array for fields where confidence scores are too close to call or fall below a configured threshold. These unresolved conflicts should be routed to a human review queue. A common failure mode is the model inventing a compromise value instead of flagging a conflict—mitigate this by including strict instructions and a negative example in the prompt. The next step after reading this section is to review the prompt template and adapt the conflict resolution strategy to your specific tolerance for automation versus human review.

The following prompt template is designed to be dropped into a data assembly pipeline. It expects multiple extraction passes over the same source material, each producing a partial record with per-field confidence scores. The model's job is to resolve conflicts, produce a single composite record, and flag any fields where the evidence remains too weak for automation. Use square-bracket placeholders to inject your specific extraction payloads, field schemas, and conflict resolution rules before sending the prompt to the model.

codeCopy
You are a data assembly engine. Your task is to merge multiple extraction passes into one final record.

## INPUTS
You will receive:
1. **Source Text**: The original document from which all extractions were made.
   [SOURCE_TEXT]

2. **Extraction Passes**: A list of extraction results. Each pass contains a partial record with per-field confidence scores (0.0 to 1.0) and, where available, source spans.
   [EXTRACTION_PASSES]

3. **Target Schema**: The desired output schema with field names, types, and nullability.
   [TARGET_SCHEMA]

## RULES
- For each field in the target schema, select the value from the extraction pass with the highest confidence score for that field.
- If multiple passes have the same confidence score for a field, prefer the value that appears most frequently. If still tied, prefer the value with the longest source span.
- If no extraction pass provides a value for a field, set the field to `null` and set `null_reason` to `"MISSING"`.
- If a field's highest confidence score is below the threshold specified in [CONFIDENCE_THRESHOLD], do not auto-accept the value. Instead, set the field to `null`, set `null_reason` to `"LOW_CONFIDENCE"`, and add the field to the `conflicts` array with all candidate values and their scores.
- For every field in the output, include a `composite_confidence` score calculated as the maximum confidence across all passes for that field.
- For every field, include a `source` indicating which extraction pass provided the winning value (e.g., `"pass_1"`, `"pass_2"`). If the field is null, set `source` to `null`.
- If two or more passes disagree on a field and the highest confidence is above the threshold, still record the conflict in the `conflicts` array for audit purposes, but populate the field with the winning value.

## OUTPUT SCHEMA
Return a single JSON object with this structure:
{
  "record": {
    "field_name": <winning value or null>,
    ...
  },
  "field_metadata": {
    "field_name": {
      "composite_confidence": <float 0.0-1.0>,
      "source": <string or null>,
      "null_reason": <"MISSING" | "LOW_CONFIDENCE" | null>
    },
    ...
  },
  "conflicts": [
    {
      "field": <string>,
      "candidates": [
        {"value": <any>, "confidence": <float>, "source": <string>},
        ...
      ],
      "resolution": <"AUTO_RESOLVED" | "UNRESOLVED">
    }
  ],
  "assembly_summary": {
    "total_fields": <int>,
    "auto_resolved": <int>,
    "low_confidence_escalated": <int>,
    "missing": <int>,
    "conflicts_detected": <int>
  }
}

## CONSTRAINTS
- Do not invent values. Every value must come from at least one extraction pass.
- Do not alter the source text. Use it only to verify span accuracy when resolving ties.
- If the source text contradicts all extraction passes, flag the field in `conflicts` with `resolution: "UNRESOLVED"` and include the source text span as an additional candidate.
- Output only the JSON object. No surrounding text, no markdown fences.

To adapt this template for your pipeline, replace [TARGET_SCHEMA] with your actual field definitions, including types and nullability rules. The [CONFIDENCE_THRESHOLD] placeholder should be set to a float between 0.0 and 1.0—start at 0.7 for most production pipelines and tune based on your tolerance for false positives versus missed extractions. The [EXTRACTION_PASSES] placeholder expects an array of objects, each containing a partial record and per-field confidence scores. If your extraction passes use different field names, add a mapping step before assembly rather than asking the model to guess alignments.

Before deploying this prompt to production, run it against a golden dataset of source texts with known conflicts, missing fields, and low-confidence extractions. Validate that composite_confidence scores are correctly computed as the max across passes, that null_reason codes are consistent, and that the conflicts array captures every disagreement. For high-stakes data pipelines—such as those feeding into financial, legal, or clinical systems—route all records with low_confidence_escalated > 0 or conflicts_detected > 0 to a human review queue before ingestion.

The Confidence-Weighted Record Assembly Prompt is designed to sit downstream of multiple extraction passes. In a typical pipeline, you run two or more extraction calls against the same source document—perhaps using different models, different prompt variants, or different temperature settings—and collect their outputs as a list of candidate records. This prompt consumes those candidates and produces a single assembled record with weighted field values, a composite confidence score, and a conflict log. The implementation harness must handle candidate collection, prompt invocation, output validation, and downstream routing based on the composite confidence score.

Wire the prompt into your application as a post-extraction merge step. Collect candidate records into a JSON array, each with a source identifier and a confidence score for the overall extraction pass. Pass this array as the [CANDIDATE_RECORDS] placeholder. The prompt expects each candidate to contain the same schema fields; if your extraction passes produce heterogeneous schemas, normalize them to a common schema before assembly. After invoking the LLM, validate the output against your expected record schema plus the assembly-specific fields: composite_confidence, conflict_log, and per-field weighted_value / weighted_confidence pairs. Reject outputs that omit required fields, contain malformed confidence scores outside 0.0–1.0, or fail to document conflicts when field values diverge across candidates. On validation failure, retry once with the validation errors injected into the [CONSTRAINTS] block; if the second attempt fails, route the record to a human review queue with the raw candidates attached.

Conflict resolution is the core risk in this workflow. The prompt uses confidence-weighted voting: when two candidates disagree on a field value, the higher-confidence candidate wins, but the conflict is logged with both values and their confidences. If confidence scores are tied or within a configurable margin (e.g., within 0.1), the prompt flags the field as unresolved. Your harness should define a conflict_margin parameter and pass it in [CONSTRAINTS]. After assembly, scan the conflict_log for unresolved conflicts and route those records to human review. For high-stakes domains like clinical data or financial figures, consider routing any record with a composite_confidence below 0.85 or any unresolved conflict to a review queue regardless of the margin. Log every assembly decision—candidate confidences, winning values, conflict resolutions, and final composite score—for auditability and drift detection.

Model choice matters for this task. The prompt requires careful arithmetic reasoning (weighted averaging across candidates) and strict schema adherence. Use a model with strong instruction-following and JSON output capabilities, such as GPT-4o, Claude 3.5 Sonnet, or a fine-tuned variant if you have consistent schema shapes. Avoid smaller or older models that may hallucinate confidence calculations or drop conflict log entries. If latency is critical, consider batching multiple records into a single prompt call, but keep batch sizes small (3–5 records) to avoid attention dilution. For cost optimization, cache the assembled output keyed by a hash of the candidate records and the conflict margin; identical inputs should produce identical assemblies, making this a strong cache candidate.

Before shipping, build a regression test suite with known conflict scenarios: tied confidences, one candidate missing a field, three-way disagreements, and edge cases like null vs. empty string vs. zero. Run these through the harness and verify that the conflict log captures every divergence, the composite confidence drops appropriately for unresolved conflicts, and the weighted values match expected outcomes. Monitor production assemblies for confidence score drift, conflict rate spikes, and validation failure rates. If the conflict rate rises, investigate whether your upstream extraction passes are diverging more than expected—this often signals prompt drift or model behavior changes in the extraction layer.