Prompt

Evidence Grounding Prompt for Numerical Claims

A practical prompt playbook for grounding quantitative answers in retrieved evidence, with source value extraction, unit normalization, and calculation transparency.

Get in touch Learn more

Stylish WeWork-like workspace with hot desks and document wall, professional searching through enterprise knowledge base on a mounted ultrawide display, warm industrial pendants overhead.

PROMPT PLAYBOOK

When to Use This Prompt

Define the job, reader, and constraints for numerical claim grounding in RAG systems.

This prompt is designed for RAG system builders who need to answer quantitative questions with precision, not just fluency. The primary job-to-be-done is extracting, normalizing, and calculating with numbers found in retrieved documents—converting unstructured text like 'revenue grew by twelve percent to $3.2M' into a verifiable, traceable numerical answer. The ideal user is an engineering lead or AI developer integrating a Q&A feature into a financial, scientific, or operational product where a wrong number carries real business cost. Required context includes a set of retrieved document chunks likely to contain the relevant figures, the user's original question, and any known unit or currency preferences.

This prompt is not a general-purpose Q&A tool. Do not use it when the question is purely qualitative, when no retrieved context is available, or when the model is expected to rely on its own parametric knowledge. It is also inappropriate for questions requiring subjective judgment, legal interpretation, or medical dosing calculations without a mandatory human review step. The prompt assumes that the retrieval step has already occurred and that the provided context contains the answer; it will not perform a web search or query a database. If the evidence is insufficient, the prompt is designed to abstain rather than guess.

Before integrating this prompt, ensure your upstream retrieval pipeline is tuned for recall over precision—it is better to provide noisy context that contains the number than to miss it entirely. You should also implement a post-generation validation layer that parses the structured output, checks that extracted source values match the original text, and flags any calculation for manual review if the confidence score falls below a defined threshold. For high-stakes domains like financial reporting or clinical data, always route outputs through a human approval queue before they reach an end user. The next step is to copy the prompt template, adapt the placeholders to your document schema, and run it against a golden dataset of known numerical Q&A pairs to establish a baseline accuracy metric.

PRACTICAL GUARDRAILS

Use Case Fit

Where the Evidence Grounding Prompt for Numerical Claims works, where it fails, and what inputs it assumes.

Good Fit: Quantitative RAG Pipelines

Use when: Your RAG system must answer questions with specific numbers, statistics, or financial figures extracted from a trusted knowledge base. Why: The prompt's structured extraction, unit normalization, and calculation transparency are designed for high-precision quantitative claims, not general summarization.

Bad Fit: Subjective or Qualitative Analysis

Avoid when: The user is asking for sentiment, opinion, or open-ended strategic advice. Risk: The prompt will force a numerical structure onto non-numerical content, producing low-confidence or hallucinated figures. Guardrail: Route to a standard RAG synthesis prompt first and only invoke this playbook when a numerical claim is explicitly requested.

Required Inputs: Source Text and Unit Context

What you must provide: A set of retrieved documents containing the raw numerical values, the user's specific quantitative question, and explicit unit-of-measure context (e.g., 'USD', 'kilograms', 'percent'). Guardrail: If the retrieval set lacks the specific number or its unit, the prompt must abstain rather than guess. Validate retrieval recall before assembly.

Operational Risk: Numerical Hallucination

What to watch: The model confidently states a specific number that is not present in any source document, or performs a calculation with a fabricated operand. Guardrail: Implement a post-generation validation step that uses a deterministic script to verify every output number against the raw source text. Flag any unverifiable claim for human review.

Operational Risk: Unit Conversion Errors

What to watch: The model incorrectly converts between units (e.g., miles to kilometers) or mixes incompatible units in a calculation, leading to a precise but wrong answer. Guardrail: Constrain the prompt to perform calculations only in a single, pre-declared base unit. Use a tool or deterministic function for all unit conversions instead of relying on the model's internal knowledge.

When to Escalate: Multi-Hop Calculations

What to watch: The user's question requires chaining multiple numerical facts from different documents to arrive at a final answer. Risk: The model may correctly extract each fact but fail in the logical assembly or arithmetic. Guardrail: Decompose the task. Use this prompt to extract individual claims, then use a separate, testable code function (not an LLM) to perform the final calculation.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A reusable prompt template with square-bracket placeholders for grounding numerical claims in retrieved evidence.

This template is designed to be copied directly into your prompt assembly pipeline. It forces the model to extract numerical values from provided context, normalize units, show calculation steps, and flag any claims that cannot be verified. Use it as the core instruction block in a RAG system where quantitative precision matters more than fluent prose.

text
You are an evidence-grounded numerical analyst. Your task is to answer a question using ONLY the provided context documents. You must extract, normalize, and calculate numerical values with full transparency.

## CONTEXT
[CONTEXT]

## QUESTION
[QUESTION]

## INSTRUCTIONS
1. Identify every numerical claim in the context that is relevant to the question.
2. For each relevant number, extract the exact value, its unit, and the source document identifier.
3. Normalize all values to a common unit before performing any calculations. Show your unit conversion steps explicitly.
4. If a calculation is required, show the formula, the substituted values, and the final result.
5. If the context contains conflicting numbers, report all values with their sources and explain the discrepancy.
6. If the context is insufficient to answer the question, state exactly what information is missing rather than guessing.

## OUTPUT FORMAT
Return a JSON object with this exact schema:
{
  "answer": "The final numerical answer with unit, or null if insufficient evidence",
  "extracted_values": [
    {
      "value": number,
      "original_unit": "string",
      "normalized_value": number,
      "normalized_unit": "string",
      "source_document_id": "string",
      "source_span": "exact text from context"
    }
  ],
  "calculation_steps": ["step 1", "step 2"],
  "conflicts": [{"value_a": number, "value_b": number, "source_a": "id", "source_b": "id", "explanation": "string"}],
  "confidence": "high|medium|low|insufficient_evidence",
  "missing_information": ["what is needed but absent"]
}

## CONSTRAINTS
- Never invent numbers not present in the context.
- If a unit is missing from the context, flag it in missing_information rather than assuming.
- Round final answers to [SIGNIFICANT_FIGURES] significant figures.
- If the confidence is "low" or "insufficient_evidence", set answer to null.

To adapt this template, replace [CONTEXT] with your retrieved passages, each prefixed by a document ID and chunk reference. Replace [QUESTION] with the user's query. Set [SIGNIFICANT_FIGURES] based on your domain's precision requirements—financial data might need 2 decimal places, while scientific measurements might need 4 significant figures. Before deploying, run this prompt against a golden dataset of known numerical answers and measure both the answer accuracy and the correctness of the extracted_values array. If your use case involves regulated data, add a human review step whenever confidence is "low" or conflicts are present.

IMPLEMENTATION TABLE

Prompt Variables

Required inputs for the Evidence Grounding Prompt for Numerical Claims. Each variable must be populated before inference to ensure reliable numerical extraction, unit normalization, and calculation transparency.

Placeholder	Purpose	Example	Validation Notes
[USER_QUERY]	The user's question or instruction containing a numerical claim to verify or a quantitative question to answer	What was the revenue growth rate for Acme Corp in Q3 2024?	Must be non-empty string. Check for explicit numerical targets or implicit quantitative intent before invoking this prompt
[RETRIEVED_CONTEXT]	The set of retrieved documents, passages, or data sources that may contain evidence for the numerical claim	Passage 1: Acme Corp reported Q3 revenue of $4.2B, up from $3.8B in Q2... Passage 2: Industry analysts noted...	Must contain at least one passage. Validate that context includes source identifiers for citation anchoring. Null or empty context should trigger abstention path
[SOURCE_METADATA]	Metadata for each retrieved source including document ID, publication date, authority score, and retrieval timestamp	{"doc_id": "acme-10q-2024", "pub_date": "2024-10-15", "authority": 0.92, "retrieved_at": "2024-11-01T14:22:00Z"}	Must be valid JSON object per source. Validate pub_date is ISO 8601, authority is float 0-1. Missing metadata should downgrade source confidence
[NUMERICAL_PRECISION_REQUIREMENT]	The required precision level for the numerical answer including significant figures, rounding rules, and unit expectations	Round to 1 decimal place, express as percentage, include absolute values alongside percentage change	Must be explicit string or null. If null, default to 2 significant figures. Check for conflicting precision instructions across system and user directives
[UNIT_NORMALIZATION_RULES]	Rules for converting extracted values to a common unit system before comparison or calculation	Convert all currency to USD using exchange rate from source date. Normalize percentages to same base period.	Must be valid string or null. If null, flag in output that no normalization was applied. Validate that rules reference specific conversion factors or sources
[CONFIDENCE_THRESHOLD]	Minimum confidence score required to include a numerical claim in the final answer. Claims below threshold trigger abstention or qualification	0.75	Must be float between 0.0 and 1.0. Default 0.7 if not specified. Validate that threshold is not set below 0.5 in production without explicit review approval
[OUTPUT_SCHEMA]	The expected structure for the grounded numerical answer including fields for extracted values, calculations, source citations, and confidence scores	{"answer": "number", "unit": "string", "source_values": [{"value": "number", "source_id": "string", "span": "string"}], "calculation_steps": ["string"], "confidence": "float"}	Must be valid JSON Schema or example structure. Validate that schema includes fields for source attribution, calculation transparency, and confidence. Missing schema should trigger format error before inference
[ABSTENTION_CONDITIONS]	Specific conditions under which the model should refuse to provide a numerical answer and instead return a structured abstention response	Abstain if: no source contains the target metric, extracted values differ by more than 20% across sources, or confidence falls below threshold	Must be explicit string or null. Validate that conditions are testable and not contradictory. Null abstention conditions should trigger a warning in preflight checks

PRACTICAL GUARDRAILS

Common Failure Modes

Numerical grounding fails in predictable ways. These are the most common failure modes when prompting models to extract, calculate, and cite quantitative claims from evidence, along with concrete guardrails to catch them before they reach users.

Unit Mismatch and Conversion Drift

What to watch: The model extracts a number but ignores unit context, comparing dollars to euros or miles to kilometers without conversion. This is especially common when sources use mixed units. Guardrail: Require explicit unit extraction and normalization in the output schema. Add a validator that checks all extracted values have matching units before comparison or aggregation.

Precision Inflation from Source Numbers

What to watch: A source states 'approximately 3.4 million' and the model treats it as exactly 3,400,000, then performs multi-step arithmetic that implies false precision. The final answer looks authoritative but is misleading. Guardrail: Enforce significant figure rules in the prompt. Require the model to report source precision explicitly and propagate uncertainty through calculations. Flag outputs where derived precision exceeds source precision.

Hallucinated Numerical Evidence

What to watch: When retrieved context lacks a specific number the user requested, the model invents a plausible value rather than abstaining. This is the most dangerous failure mode for quantitative RAG. Guardrail: Require every numerical claim to include a direct source quote or explicit abstention marker. Add a post-generation check that verifies each number appears verbatim in the provided context. If not found, trigger a repair or escalation step.

Aggregation Without Source Alignment

What to watch: The model sums or averages numbers from different time periods, populations, or measurement methodologies as if they were directly comparable. The arithmetic is correct but the comparison is invalid. Guardrail: Require the model to check temporal alignment, population scope, and methodology consistency before aggregating. Add a 'comparability note' field to the output schema that flags when sources measure different things.

Missing Denominator for Rate Claims

What to watch: A source reports '500 incidents' and the model reports this as a rate or percentage without the denominator. The number is factually correct but contextually meaningless. Guardrail: Require explicit denominator extraction for any rate, percentage, or per-capita claim. If the denominator is absent from the retrieved context, the model must either request it or clearly state that the rate cannot be calculated.

Source Conflict Silently Resolved

What to watch: Two retrieved sources report different numbers for the same metric. The model picks one without acknowledging the conflict, or averages them without noting the discrepancy. The user receives a single confident number with no visibility into the underlying disagreement. Guardrail: Require the model to surface all conflicting values explicitly in the output. Add a 'source agreement' field that reports when values diverge beyond a configurable threshold. Escalate to human review when conflicts exceed tolerance.

IMPLEMENTATION TABLE

Evaluation Rubric

Criteria for testing output quality before shipping numerical grounding prompts. Use these checks to catch hallucinated numbers, unit mismatches, and missing source anchors before they reach users.

Criterion	Pass Standard	Failure Signal	Test Method
Numerical Value Extraction	Every number in the output matches a source value exactly or is flagged as derived	Output contains a number not present in any source passage and not labeled as calculated	Diff output numbers against source set; flag unmatched values
Unit Normalization	All quantities use consistent units with conversion steps shown when sources differ	Mixed units appear without conversion notes or a conversion factor is applied incorrectly	Parse units from output; verify conversion math against known rates
Source Citation Completeness	Every numerical claim has an inline citation pointing to a specific source passage	A number appears without a citation anchor or the citation points to a passage without that number	Regex scan for citation markers; cross-reference each cited passage for the claimed value
Calculation Transparency	Derived values include the formula, input values, and intermediate steps	A calculated result appears with no formula or the formula uses values not cited in sources	Check that derived fields contain formula and operand references; verify arithmetic
Range and Precision Fidelity	Output preserves source precision and uses ranges when sources express uncertainty	A precise number is reported when the source gives a range, or decimal precision exceeds source data	Compare output precision to source precision; flag fabricated significant digits
Abstention Trigger	Prompt returns structured abstention when no source contains the requested number	Output fabricates a number or returns a generic fallback when evidence is absent	Test with queries where the answer is not in the retrieval set; expect abstention response
Unit Conversion Accuracy	All unit conversions are correct to within 0.1% of the standard conversion factor	A conversion error exceeds tolerance or a non-standard conversion factor is used without justification	Validate conversions against known conversion tables; flag deviations beyond tolerance
Confidence Score Calibration	Confidence scores correlate with source agreement and evidence quality	High confidence assigned to a claim with conflicting sources or low-quality evidence	Run multiple test cases; check that confidence drops when sources disagree or are thin

Enabling Efficiency, Speed & Accuracy

Intelligent Analysis, Decision & Execution

We build AI systems for teams that need search across company data, workflow automation across tools, or AI features inside products and internal software.

Talk to Us

Search across company data

Give teams answers from docs, tickets, runbooks, and product data with sources and permissions.

Useful when people spend too long searching or get different answers from different systems.

Enterprise searchRAGPermissions

Automate internal workflows

Use AI to route work, draft outputs, trigger actions, and keep approvals and logs in place.

Useful when repetitive work moves across multiple tools and teams.

AI agentsWorkflow automationGovernance

Add AI to products and internal tools

Build assistants, guided actions, or decision support into the software your team or customers already use.

Useful when AI needs to be part of the product, not a separate tool.

AI integrationDecision supportModel routing

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Use the base prompt with a single model call and minimal post-processing. Replace [OUTPUT_SCHEMA] with a simple JSON structure containing only claim, extracted_value, source_value, and normalized_value. Skip unit normalization logic and rely on the model to handle simple conversions.

code
[OUTPUT_SCHEMA]: {
  "claim": "string",
  "extracted_value": "number",
  "source_value": "string",
  "normalized_value": "number",
  "unit": "string"
}

Watch for

Model inventing values when source text is ambiguous
Unit mismatches between source and normalized fields
Missing source citations when multiple documents are provided
Silent failures when the model cannot extract a number and returns null without explanation

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.

Limited slotsGet a Free AI Consultation

How We Work

Custom AI workflows for your Business

One-fit-all AI don't work for modern businesses. At Inferensys, we aim to understand your business & custom requirements; which we use to define most efficient agentic workflows, the data, and the tools for your business.