Mean vs Median and Distribution Claim Verification Prompt

This prompt is for verification engineers and data analysts who need to check whether a claim about central tendency (mean, median, mode) or distribution shape (skew, spread, modality) is supported by the underlying data. Use it when a report, article, or dashboard asserts something like 'the average customer spends $120' or 'the distribution is normal' and you need to validate that the correct statistic was chosen and the claim matches the evidence. The prompt is designed for post-hoc verification of existing claims, not for generating new analysis. It assumes you already have the claim text and the source data or summary statistics available.

The prompt forces the model to compute both mean and median, assess skew, flag misleading 'average' usage, and check whether dispersion context was omitted. It is most effective when the claim is a single, atomic assertion about a dataset's central tendency or shape. Do not use this prompt for multi-claim paragraphs, causal claims, trend direction assertions, or statistical significance statements—those require separate specialized verification prompts. The ideal user has access to the raw data or a reliable summary table and needs a structured, repeatable check before publishing or acting on the claim.

Before using this prompt, ensure you have the exact claim text and either the full dataset or sufficient summary statistics (mean, median, standard deviation, quartiles, and sample size). If you only have the claim without underlying data, this prompt cannot perform verification and you should instead use an evidence-matching or source-attribution prompt. After running the verification, always review the skew assessment and dispersion check manually—these are the most common failure points when the model misinterprets distribution shape from summary statistics alone.

This prompt is designed to be pasted directly into your verification workflow. It instructs the model to act as a statistical auditor, comparing a provided claim about an average or distribution against a supplied dataset. The core task is to determine whether the correct measure of central tendency was used given the data's shape, and to flag any misleading omissions about dispersion or skew. Replace every square-bracket placeholder with your specific inputs before execution.

textCopy
You are a statistical verification auditor. Your task is to evaluate a numerical claim about a dataset's central tendency or distribution.

[INPUT]
CLAIM: [The exact text of the claim to verify, e.g., "Our typical customer spends $120 per month."]
DATA_SUMMARY: [A structured summary of the dataset. Include N, mean, median, standard deviation, skewness, and a five-number summary (min, Q1, median, Q3, max). If raw data is provided via a tool, describe it here.]
[/INPUT]

[CONSTRAINTS]
1. Compare the claim's statistic (mean vs. median) against the data's skewness. Flag if the wrong measure was used to represent a "typical" value.
2. If the claim uses an ambiguous term like "average" or "typical," state which specific statistic it most likely refers to and whether that is misleading.
3. Check if the claim omits critical context about dispersion (e.g., standard deviation, range).
4. If the claim implies a distribution shape (e.g., "most users are around..."), verify it against the provided skewness and kurtosis.
5. Do not invent data. If information is missing to perform a check, state it explicitly.
[/CONSTRAINTS]

[OUTPUT_SCHEMA]
{
  "claim_summary": "string (restate the claim being verified)",
  "statistic_used_in_claim": "mean | median | mode | ambiguous | other",
  "data_shape": {
    "skewness": "string (e.g., 'highly positive', 'symmetric')",
    "outlier_presence": "yes | no | unknown"
  },
  "verdict": "accurate | misleading | unverifiable",
  "verification_details": {
    "correct_measure": "string (the measure that should have been used)",
    "misleading_aspects": ["string (list specific problems, e.g., 'Mean is inflated by outliers')"],
    "missing_context": ["string (list omitted dispersion info, e.g., 'No standard deviation provided')"]
  },
  "evidence": "string (explain reasoning using the provided DATA_SUMMARY)"
}
[/OUTPUT_SCHEMA]

[RISK_LEVEL]
HIGH: Numerical claims in regulated or public-facing reports require human review of this output before publication.
[/RISK_LEVEL]

To adapt this template, start by replacing the [INPUT] block with your specific claim and a structured data summary. If you have access to a code execution tool, you can replace DATA_SUMMARY with a [TOOLS] block that allows the model to calculate the required statistics directly from a raw dataset, which is more reliable. The [OUTPUT_SCHEMA] is designed for programmatic consumption; keep the JSON structure intact so your application harness can parse the verdict field for automated routing. For high-stakes claims, always route misleading and unverifiable verdicts to a human review queue as specified in the [RISK_LEVEL] block.

Before deploying, test this prompt with a known dataset where the mean and median diverge significantly, such as income data with a few very high earners. A claim that "the average income is $150k" when the median is $75k should produce a misleading verdict. Also test a symmetric dataset to ensure the prompt does not over-flag accurate mean usage. The most common production failure is providing an insufficient DATA_SUMMARY; the model needs skewness and a five-number summary at minimum to perform a reliable audit.

This prompt is designed to be called as a single step within a larger numerical claim verification pipeline. It expects a structured input containing the original claim text, the reported statistic (mean or median), the provided data sample or distribution summary, and any context about the claim's source. The output is a structured JSON object that downstream systems can parse to flag misleading central tendency claims, route for human review, or attach to an audit trail. Do not use this prompt in isolation without first extracting the numerical claim and its associated data from source material—pair it with a claim extraction prompt from the Claim Extraction and Decomposition group before invocation.

Validation and retry logic: Parse the model's JSON output and validate it against a strict schema before accepting it. At minimum, confirm that statistic_used is one of mean, median, or mode; that distribution_skew_detected is a boolean; that alternative_statistic_recommended is populated when a mismatch is found; and that confidence_score is a float between 0.0 and 1.0. If the output fails schema validation, retry once with the same input and an explicit instruction appended: Your previous output failed JSON schema validation. Return ONLY valid JSON matching the required schema. If the second attempt also fails, log the failure, attach the raw output, and escalate to a human review queue rather than silently accepting a malformed verification result.

Model choice and tool integration: This prompt works best with models that have strong numerical reasoning capabilities (GPT-4o, Claude 3.5 Sonnet, or equivalent). For high-throughput pipelines, consider a two-tier architecture: run the prompt on a faster, cheaper model first, and only escalate to a more capable model when the confidence score falls below 0.85 or when distribution_skew_detected is true and the claim is high-stakes. If your pipeline includes a calculation engine or statistical library, wire the prompt's output to a tool that independently computes the mean, median, skewness, and outlier count from the provided data sample. Compare the tool's computed values against the prompt's computed_mean and computed_median fields. A discrepancy greater than a configurable tolerance (e.g., 0.1% for floating-point values) should trigger a verification failure and human review, regardless of the model's confidence score.

Logging and audit trail: Every invocation should log the full prompt input, the raw model response, the parsed and validated output, the tool-computed comparison values (if applicable), and the final disposition (passed, flagged, escalated). Store these records with a unique verification_id that links back to the original claim and source document. This audit trail is essential for downstream governance, regression testing, and debugging false positives. When the prompt flags a claim as misleading (e.g., mean used on skewed data without median disclosure), the output should be routed to a structured verification report generator that includes the original claim, the evidence of skew, the recommended alternative statistic, and a human-readable explanation suitable for non-technical reviewers.

What to avoid: Do not treat this prompt's output as a final truth verdict without independent calculation verification. The model can reason about distribution shape and statistic appropriateness, but it can also hallucinate computed values or misclassify mild skew as severe. Never expose the raw model output directly to end users without the validation and tool-comparison harness described above. For regulated or high-stakes domains (financial reporting, clinical data, public policy statistics), always require human review when the prompt detects a mean-median mismatch, regardless of confidence score. The cost of a false negative—letting a misleading average claim through—is higher than the cost of a human reviewer checking a flagged case.