Chart-Claim-to-Underlying-Data Reconciliation Prompt

This prompt is designed for analysts, data engineers, and verification pipeline builders who need to programmatically reconcile claims made in charts or visualizations against the underlying raw data. Use it when a chart makes a specific assertion about a trend, a value, a comparison, or an uncertainty range, and you have access to the dataset that was used to generate it. The prompt is built for a structured verification workflow, not for open-ended chart interpretation. It expects a specific claim extracted from a chart and a structured representation of the raw data. The output is a machine-readable reconciliation verdict, not a narrative summary. This is a critical component in automated fact-checking, report auditing, and data journalism pipelines where visualizations can mislead through axis manipulation, cherry-picked ranges, or misrepresented error bars.

The ideal user is someone integrating this prompt into a larger verification harness. You should already have a pipeline that extracts atomic claims from chart captions, annotations, or accompanying text, and you should have the raw data in a structured format such as CSV, JSON, or a database table. The prompt does not perform OCR, chart image analysis, or claim extraction—those are separate upstream tasks. It assumes the claim and the data are already isolated and formatted. The prompt is most effective when the claim is specific and falsifiable: 'Revenue grew 15% YoY in Q3' rather than 'Revenue performed well.' It also requires that the raw data columns map clearly to the claim's dimensions, such as time periods, categories, or measurement units. If the mapping is ambiguous, the prompt will struggle and may produce false confidence.

Do not use this prompt when the chart is unavailable for inspection, when the raw data is incomplete or aggregated differently than the chart implies, or when the claim is subjective or qualitative. It is not suitable for verifying claims about visual design choices, color encoding, or chart aesthetics. It also should not be used as a standalone truth arbiter without human review in high-stakes contexts such as financial filings, clinical trial reports, or regulatory submissions. In those cases, the prompt's output should feed into a human review queue with the original chart, the raw data, and the reconciliation verdict all presented together. The prompt is a verification accelerator, not a replacement for auditor judgment.

Before using this prompt, ensure your upstream claim extraction produces structured objects with fields for the claim text, the chart element referenced (e.g., 'bar chart, Q3 2024 bar'), the asserted value or trend, and any qualifiers such as confidence intervals or percentage changes. Your raw data should be normalized to consistent units, time zones, and categories. If the raw data requires transformation to match the chart's aggregation level, perform that transformation before calling this prompt, and document the transformation steps. The prompt's reliability depends on the quality of these inputs. Garbage in, garbage out applies with full force here.

Common failure modes include axis misrepresentation where the chart uses a non-zero baseline, truncated ranges, or logarithmic scales that distort the visual impression of the claim. The prompt includes specific checks for these patterns, but it can only detect them if the claim explicitly references the visual encoding. If the claim says 'Revenue grew sharply' and the chart uses a truncated y-axis, the prompt may reconcile the numerical values correctly while missing the visual-manipulation angle unless the claim includes the visual description. Always pair this prompt with a chart-inspection step that captures axis parameters, scale types, and visual encodings before claim extraction.

After running this prompt, validate the output against a schema that enforces the verdict structure, evidence references, and confidence scores. Log every reconciliation attempt with the claim, the raw data snapshot, the prompt version, and the model used. This creates an audit trail that lets you debug false positives or negatives later. In production, route low-confidence verdicts and contradiction flags to human review. Over time, use these human-reviewed cases to build a golden dataset for evaluating prompt changes and model upgrades.

This prompt template is designed to be pasted directly into your prompt layer, whether you're using a model API, a playground, or an orchestration framework. It accepts a chart-derived claim, the raw data that supposedly generated the chart, and optional metadata about the chart's visual encoding. Replace each square-bracket placeholder with your actual values before execution. The template is structured to force the model to perform explicit numerical comparison rather than relying on visual pattern matching, which reduces optical-illusion misinterpretation risks common in chart-to-data reconciliation tasks.

textCopy
You are a chart-audit specialist. Your task is to reconcile a claim made about a chart against the underlying raw data that supposedly generated that chart. You must perform explicit numerical comparisons. Do not rely on visual reasoning or pattern matching.

## INPUT

**Chart Claim:** [CHART_CLAIM]

**Raw Data:** [RAW_DATA]

**Chart Metadata (optional):**
- Chart Type: [CHART_TYPE]
- Axis Labels: [AXIS_LABELS]
- Visual Encoding Notes: [VISUAL_ENCODING_NOTES]

## CONSTRAINTS

[CONSTRAINTS]

## OUTPUT SCHEMA

Return a JSON object with the following structure:

[OUTPUT_SCHEMA]

## INSTRUCTIONS

1. Parse the raw data into a structured internal representation.
2. Extract every numerical assertion from the chart claim.
3. For each assertion, locate the corresponding value in the raw data.
4. Compare the claimed value to the raw data value using the tolerance specified in constraints.
5. Flag any of the following:
   - Axis misrepresentation (e.g., truncated axes, non-zero baselines not disclosed)
   - Trend-direction mismatches (e.g., claim says 'increasing' but data shows decrease)
   - Error-bar-to-confidence-interval misalignment (e.g., error bars shown but confidence interval width doesn't match)
   - Aggregation-scope mismatches (e.g., claim uses 'total' but data supports only 'average')
   - Rounding or significant-figure distortions
6. For each flag, provide the exact data point reference and a brief explanation.
7. If the raw data is insufficient to verify a claim, mark it as 'Unverifiable' with a reason.

## EXAMPLES

[EXAMPLES]

## RISK LEVEL

[RISK_LEVEL]

Adaptation guidance: The [CONSTRAINTS] placeholder should include your numerical tolerance windows (e.g., '±0.5% for percentages, ±2 units for absolute values'), unit-normalization rules, and any domain-specific reconciliation rules. The [OUTPUT_SCHEMA] should define the exact JSON structure you expect, including fields for claim_id, claimed_value, raw_data_value, match_result, flags, and evidence_reference. The [EXAMPLES] placeholder should contain at least one correct reconciliation example and one example showing a flagged misrepresentation. Set [RISK_LEVEL] to 'HIGH' if the chart claim relates to financial, medical, or safety-critical data, which should trigger additional validation and human review in your harness. For production use, always validate the output JSON against your schema before accepting the reconciliation results, and log every reconciliation attempt with the raw data hash for auditability.

This prompt is designed to be embedded in a multi-step verification pipeline, not used as a standalone chat interaction. The typical harness receives a user-submitted chart image and a structured dataset (CSV, JSON, or database extract) that supposedly generated the chart. The harness must pre-process both inputs before calling the model: the chart image is passed directly to a vision-capable model, while the underlying data is serialized into a compact text representation (e.g., a markdown table or JSON array) and injected into the [UNDERLYING_DATA] placeholder. The [CLAIM_STATEMENT] placeholder should be populated with the specific assertion being verified—this may come from a human analyst, an upstream claim extraction prompt, or the chart's own title and axis labels.

Post-generation validation is critical because this prompt operates on numerical data where small errors carry high consequences. Implement a schema validator that checks the output JSON for required fields (claim_id, claim_summary, verdict, evidence, discrepancies, confidence_score) and enforces enum values for verdict (supported, contradicted, unverifiable, misleading). Add a numerical sanity checker that verifies any extracted values in the evidence array fall within the range of the provided underlying data—a common failure mode is the model hallucinating data points not present in the source. For high-stakes financial, clinical, or regulatory use cases, route all contradicted and misleading verdicts to a human review queue with the original chart, underlying data, and model output packaged together.

Model choice significantly impacts accuracy. Use a vision-capable model (GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro) for chart interpretation, and consider a two-pass architecture: first extract the visual data series into a structured format using a dedicated chart-reading prompt, then pass that extracted data alongside the underlying data to a text-only model for reconciliation. This separation reduces optical illusion errors and makes each step independently testable. Implement retry logic for malformed JSON outputs, but cap retries at 3 attempts with exponential backoff. Log every reconciliation attempt with the model version, input hashes, output, and validation results for audit trails. For batch processing, set a concurrency limit of 5-10 parallel requests to avoid rate limiting while maintaining throughput.

Testing this harness requires a golden dataset of chart-data pairs with known discrepancies. Include cases with axis truncation, inverted y-axes, dual-axis confusion, missing error bars, and correct charts to measure false-positive rates. Run regression tests before every prompt or model version change. Monitor production performance with eval metrics: precision and recall on discrepancy detection, schema compliance rate, and human-review overturn rate. If the overturn rate exceeds 20%, revisit your prompt instructions or confidence thresholds. Avoid deploying this prompt on charts with hand-drawn annotations, complex subplot grids, or 3D perspective distortions without first testing extensively—these visual patterns are known failure modes for current vision models.