Inferensys

Prompt

Factual Grounding Retry with Evidence Constraints Prompt

A practical prompt playbook for using Factual Grounding Retry with Evidence Constraints Prompt in production AI workflows.
Wide-angle shot of a modern WeWork open floor plan with creative walls covered in AI system architecture diagrams, product team collaborating in standing desk area with industrial lighting.
PROMPT PLAYBOOK

When to Use This Prompt

Defines the specific failure conditions and production contexts that warrant this strict evidence-constrained retry harness.

This prompt is a recovery step for high-stakes QA and RAG systems. Use it when a previous model response contained unsupported claims, hallucinated details, or assertions that could not be traced to the provided evidence. The prompt forces the model to regenerate the answer under a strict contract: every factual assertion must cite a specific source passage, and any territory without direct evidence support must trigger explicit abstention. This is not a first-pass generation prompt. It is a retry harness designed for compliance-sensitive domains such as legal research, clinical summarization, financial audit, and regulated customer support where an unsupported claim is a product defect.

The ideal user is an AI engineer or platform developer who has already observed a grounding failure in production or during evaluation. The required context includes the original user query, the set of retrieved evidence passages, and the previously generated response that failed validation. You should wire this prompt into a post-generation validation loop: a citation coverage check or a claim-to-source alignment evaluator flags the output, and this prompt is invoked as the first recovery step before escalating to human review. Do not use this prompt for creative writing, open-ended brainstorming, or tasks where factual precision is not the primary requirement—it will over-constrain the model and produce unnecessarily cautious outputs.

Before invoking this retry, confirm that the evidence passages provided are complete and relevant. If the retrieval step itself returned poor results, fix retrieval before applying grounding constraints—forcing a model to cite irrelevant passages produces brittle, misleading outputs. Set a retry budget: if this prompt fails to produce a fully grounded answer after two attempts, escalate to human review rather than looping indefinitely. In regulated workflows, log every retry attempt, the validator scores that triggered it, and the final grounded output for audit trails. The next section provides the copy-ready prompt template you can adapt to your evidence schema and output format requirements.

PRACTICAL GUARDRAILS

Use Case Fit

Where the Factual Grounding Retry with Evidence Constraints Prompt works, where it fails, what it needs, and what risks it introduces.

01

Good Fit: High-Stakes QA with Provided Evidence

Use when: you have a retrieved context or document set and need every assertion traceable to a source. Guardrail: always pass the evidence payload alongside the original answer so the retry prompt can perform cross-referencing.

02

Bad Fit: Open-Domain Creative Generation

Avoid when: the task is brainstorming, story generation, or subjective analysis without a fixed evidence corpus. Guardrail: route these requests to a standard generation prompt instead; applying evidence constraints here will produce excessive abstentions.

03

Required Input: Source-Anchored Evidence Payload

What to watch: the prompt cannot function without explicit source passages, document chunks, or retrieved records. Guardrail: validate that the evidence payload is non-empty and contains identifiable source identifiers before invoking the retry.

04

Required Input: The Original Ungrounded Answer

What to watch: the retry prompt needs the full original response to identify unsupported claims. Guardrail: pass the complete answer text, not a summary, to ensure the constraint checker can map every assertion back to evidence.

05

Operational Risk: Excessive Abstention

What to watch: strict evidence constraints can cause the model to abstain on reasonable inferences that users expect. Guardrail: implement an abstention threshold and log abstention rates; if they exceed 30%, review whether the evidence retrieval scope is too narrow.

06

Operational Risk: Retry Loop Amplification

What to watch: if the retry prompt itself produces ungrounded claims, you can enter a costly loop. Guardrail: enforce a maximum of 2 retries, then escalate to human review or a fallback model with different grounding instructions.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

Paste this template into your retry harness. Replace square-bracket placeholders with live data before sending to the model.

This template is designed to be the core instruction set for a retry harness that regenerates an answer after a grounding failure. It enforces a strict contract: every assertion must be directly traceable to the provided evidence, and the model must explicitly abstain from making claims that lack support. Use this when a previous model response contained hallucinations, missing citations, or unsupported inferences, and you need a corrected, fully grounded replacement.

text
You are a strict factual grounding assistant. Your task is to regenerate an answer to the user's question under absolute evidence constraints.

## INPUT
- USER_QUESTION: [USER_QUESTION]
- PREVIOUS_FAILED_ANSWER: [PREVIOUS_FAILED_ANSWER]
- FAILURE_REASON: [FAILURE_REASON]
- EVIDENCE_SOURCES: [EVIDENCE_SOURCES]

## CONSTRAINTS
1. Every declarative statement you make MUST be directly and unambiguously supported by a passage in EVIDENCE_SOURCES.
2. For each claim, you MUST provide an inline citation in the format [Source: <source_id>].
3. If the EVIDENCE_SOURCES do not contain sufficient information to answer any part of the USER_QUESTION, you MUST explicitly state that you cannot answer that part and explain why the evidence is insufficient.
4. Do not infer, extrapolate, or use external knowledge. Your answer must be a strict synthesis of the provided text.
5. If the EVIDENCE_SOURCES contain conflicting information, you must present both sides, cite them, and state that a conflict exists. Do not choose a winner.

## OUTPUT FORMAT
Return a JSON object with the following schema:
{
  "answer": "The fully grounded answer with inline citations.",
  "abstentions": [
    {
      "question_part": "The specific part of the question you could not answer.",
      "reason": "Why the evidence was insufficient."
    }
  ],
  "conflicts": [
    {
      "claim_a": "First conflicting claim with citation.",
      "claim_b": "Second conflicting claim with citation.",
      "summary": "Brief description of the conflict."
    }
  ]
}

## RISK LEVEL
[RISK_LEVEL]

## ADDITIONAL CONTEXT
[ADDITIONAL_CONTEXT]

To adapt this template, replace the square-bracket placeholders with data from your application's retry context. [USER_QUESTION] is the original prompt. [PREVIOUS_FAILED_ANSWER] and [FAILURE_REASON] provide the model with the specific error it must correct, such as 'hallucinated source' or 'claim without citation.' [EVIDENCE_SOURCES] should be a pre-formatted string of your retrieved documents with clear source IDs. The [RISK_LEVEL] field is critical for high-stakes domains like healthcare or finance; set it to a value like 'HIGH' to reinforce the abstention policy. [ADDITIONAL_CONTEXT] can be used to inject domain-specific terminology or output length limits. After the model responds, your harness must parse the JSON and validate that every citation in the answer field maps to a valid ID in [EVIDENCE_SOURCES] before surfacing the result to the user or proceeding to the next step in the pipeline.

IMPLEMENTATION TABLE

Prompt Variables

Required inputs for the Factual Grounding Retry with Evidence Constraints prompt. Each placeholder must be populated before the retry loop executes. Missing or malformed inputs will cause the retry to fail silently or produce ungrounded output.

PlaceholderPurposeExampleValidation Notes

[ORIGINAL_QUERY]

The user's original question or instruction that produced the ungrounded answer

What were the key findings from the Q3 clinical trial?

Must be non-empty string. Preserve exact user wording to avoid intent drift during retry.

[PREVIOUS_ANSWER]

The full model output that failed grounding checks and triggered the retry

The trial showed a 23% improvement in patient outcomes across all cohorts.

Must be non-empty string. Include the complete answer, not a summary, so the retry prompt can audit every claim.

[EVIDENCE_PASSAGES]

The retrieved source documents or passages that should have grounded the answer

[{"source_id": "doc_42", "text": "In Q3, the treatment arm demonstrated a 12% improvement (p=0.03) in the primary endpoint."}]

Must be valid JSON array of objects with source_id and text fields. Empty array triggers abstention path. Validate schema before prompt assembly.

[GROUNDING_FAILURE_REPORT]

Structured description of which claims failed grounding and why

Claim at position 3 ('23% improvement') has no supporting passage. Claim at position 1 ('all cohorts') contradicts source doc_42 which specifies primary endpoint only.

Must be non-empty string or structured JSON. Generated by upstream grounding checker. Null or empty means retry cannot target specific failures and should default to full regeneration.

[CITATION_STYLE]

The required citation format for regenerated output

inline-parenthetical

Must be one of: inline-parenthetical, footnote, endnote, source-id-bracket. Invalid values should default to inline-parenthetical with a warning.

[ABSTENTION_THRESHOLD]

Minimum fraction of claims that must be groundable before the model should abstain entirely

0.5

Must be float between 0.0 and 1.0. Values below 0.3 risk generating answers from thin evidence. Validate as numeric; reject non-numeric strings.

[MAX_RETRY_ATTEMPTS]

How many times this retry prompt has been invoked in the current recovery loop

2

Must be integer >= 1. Used to decide whether to escalate to human review instead of retrying again. Compare against retry budget before invoking this prompt.

[OUTPUT_SCHEMA]

The expected JSON structure for the regenerated answer with evidence mapping

{"answer": "string", "claims": [{"text": "string", "source_id": "string", "confidence": 0.0-1.0}], "abstained": false}

Must be valid JSON Schema or example object. Every field in the schema must be populated in the output. Validate output against this schema after generation.

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the Factual Grounding Retry with Evidence Constraints prompt into a production application with validation, retry budgets, and observability.

This prompt is not a standalone fix; it is a recovery step inside a larger harness. The harness should catch a primary generation failure—typically a validation error indicating unsupported claims or missing citations—and then invoke this retry prompt with the original question, the retrieved evidence, and the failed answer. The harness must supply the [EVIDENCE] placeholder with the exact retrieved passages, not summaries, because the prompt's constraint compliance depends on verbatim source access. The [FAILED_ANSWER] placeholder should contain the full previous model output so the retry prompt can contrast unsupported claims against the evidence.

Before calling the model, the harness should validate that [EVIDENCE] is non-empty and that [FAILED_ANSWER] contains at least one claim flagged as unsupported. If evidence is empty, skip the retry and escalate to a retrieval expansion workflow or human review. After receiving the retry output, run a constraint compliance evaluator: a separate LLM call or rule-based check that verifies every sentence in the regenerated answer is either (a) directly traceable to a provided evidence passage, or (b) an explicit abstention statement. Use a structured output schema for the evaluator—fields like claim, source_passage_id, match_type (direct|abstention|unsupported). Any unsupported match should either trigger one more retry with the specific violation fed back into [FAILED_ANSWER], or escalate to human review if the retry budget is exhausted.

Set a retry budget of 2-3 attempts for this prompt. Beyond that, the model is unlikely to converge on a fully grounded answer, and further retries waste latency and compute. Log every attempt with the retry index, the evaluator output, and the final disposition (grounded, partial_with_abstention, escalated). For high-stakes domains like healthcare or legal, always route escalated outputs to a human review queue with the evidence, the failed answers, and the evaluator trace attached. Model choice matters: use a model with strong instruction-following and long-context fidelity, such as claude-sonnet-4-20250514 or gpt-4o, because smaller models often drift from the strict traceability constraint after the first few sentences. Avoid using this prompt with models that have known hallucination tendencies under constraint pressure.

IMPLEMENTATION TABLE

Expected Output Contract

Fields, format, and validation rules for the model response when using the Factual Grounding Retry with Evidence Constraints Prompt. Every assertion must be traceable to provided evidence, with explicit abstention for unsupported territory.

Field or ElementType or FormatRequiredValidation Rule

answer_text

string

Must contain only claims directly supported by [EVIDENCE_PASSAGES]. Parse check: no standalone assertions without a corresponding citation marker.

citations

array of objects

Each object must include source_id matching a provided [EVIDENCE_PASSAGES] identifier, excerpt string no longer than 200 characters, and a relevance_score between 0.0 and 1.0. Schema validation required.

abstention_flags

array of objects

Each flag must contain claim_text, reason enum from [INSUFFICIENT_EVIDENCE, CONTRADICTORY_SOURCES, OUT_OF_SCOPE, AMBIGUOUS_QUERY], and attempted_retrieval boolean. Null allowed if no abstentions.

constraint_compliance

object

Must include total_claims integer, grounded_claims integer, ungrounded_claims integer, and compliance_passed boolean. Ungrounded claims greater than 0 must set compliance_passed to false.

evidence_mapping

array of objects

Each object maps a claim_id to one or more source_id values. Every claim_id in answer_text must appear here. Missing mapping triggers retry.

confidence_scores

object

Must include overall_confidence between 0.0 and 1.0, and per_claim_confidence array with claim_id and score. Scores below [CONFIDENCE_THRESHOLD] must trigger abstention or flagging.

regeneration_metadata

object

Must include retry_count integer starting at 1, previous_failure_reason string from prior validation error, and constraints_applied array listing all active constraint IDs from [CONSTRAINTS].

PRACTICAL GUARDRAILS

Common Failure Modes

Factual grounding retry prompts fail in predictable ways. These are the most common failure modes, why they happen, and how to guard against them in production cited-answer systems.

01

Silent Hallucination Under Constraint

What to watch: The model fabricates plausible-sounding citations or rephrases unsupported claims to appear grounded when the evidence is thin. The output looks compliant but contains invented sources. Guardrail: Run a source-to-claim alignment verification step after every regeneration. Flag any citation that cannot be traced to a passage in the provided evidence set. Require explicit abstention markers for unsupported territory.

02

Over-Abstention Collapse

What to watch: Strict evidence constraints cause the model to abstain on answerable questions because it cannot find a verbatim match. The system refuses to answer when a reasonable inference from provided sources would suffice. Guardrail: Define a sufficiency threshold that distinguishes between direct citation, supported inference, and genuine evidence gaps. Include examples of acceptable inference in the prompt. Monitor abstention rate against a baseline.

03

Citation Drift in Long Answers

What to watch: Citations start correctly anchored to source passages but drift across multi-paragraph answers. Later claims reference earlier sources that no longer support them, or citations shift to adjacent passages with different meaning. Guardrail: Implement paragraph-level re-verification. After regeneration, map each citation back to its claimed source passage and check semantic alignment. Use a drift distance metric to flag answers that need re-anchoring.

04

Retrieval-Answer Mismatch Loop

What to watch: The retry prompt triggers re-retrieval, but the new evidence is only marginally different. The model regenerates a nearly identical answer, fails validation again, and the cycle repeats until the retry budget is exhausted. Guardrail: Compare new retrieval results against previous retrieval before regenerating. If the evidence delta is below a similarity threshold, escalate to human review instead of retrying. Track retrieval novelty per attempt.

05

Constraint Compliance Theater

What to watch: The model satisfies the letter of the evidence constraint by inserting citation markers that point to irrelevant passages, or by citing the same generic source for every claim regardless of relevance. The output passes automated checks but fails substantive review. Guardrail: Add a citation relevance check that evaluates whether each cited passage actually supports the specific claim it anchors. Use an LLM judge with a pairwise comparison rubric. Flag answers where citation relevance scores fall below threshold.

06

Retry Budget Exhaustion Without Escalation

What to watch: The system retries until the budget is consumed, then returns the last failed output or a generic error. The user receives a degraded answer with no indication that grounding failed. Guardrail: Design an explicit escalation path that triggers when the retry budget is exhausted. Produce a structured fallback response that explains what evidence was sought, what was found, and why the answer could not be grounded. Route to human review when confidence is below operational threshold.

IMPLEMENTATION TABLE

Evaluation Rubric

Use this rubric to test whether the Factual Grounding Retry with Evidence Constraints Prompt produces outputs that are safe to ship. Each criterion includes a concrete pass standard, a failure signal, and a test method that can be automated or run manually.

CriterionPass StandardFailure SignalTest Method

Assertion-to-Evidence Traceability

Every factual assertion in the output is directly traceable to a specific passage in [EVIDENCE_BLOCK] via an inline citation or explicit reference.

Output contains a claim not present in any evidence passage, or a citation points to a passage that does not support the claim.

Parse output into discrete claims. For each claim, run a substring match against [EVIDENCE_BLOCK]. Flag claims with zero overlap. For cited claims, verify the cited passage contains the claim's core fact.

Abstention Discipline

The output explicitly abstains from answering any sub-question where [EVIDENCE_BLOCK] lacks sufficient information, using the required abstention phrase from [ABSTENTION_PHRASE].

Output provides an answer for a sub-question with no supporting evidence, or uses a non-standard abstention phrase.

Identify all sub-questions in [INPUT]. For each, check if [EVIDENCE_BLOCK] contains relevant information. If not, assert the output contains [ABSTENTION_PHRASE] for that sub-question.

Constraint Compliance Score

Output achieves a constraint compliance score of 1.0 when evaluated by the internal [CONSTRAINT_EVALUATOR] prompt, meaning zero constraint violations.

The [CONSTRAINT_EVALUATOR] returns a score below 1.0 or lists any violation.

Run the output through the [CONSTRAINT_EVALUATOR] prompt. Assert the returned score equals 1.0 and the violations list is empty.

No Hallucinated Sources

All citations reference sources that exist within [EVIDENCE_BLOCK]. No fabricated document titles, URLs, or author names are present.

Output contains a citation with a source identifier not found in [EVIDENCE_BLOCK] metadata.

Extract all source identifiers from output citations. Assert each is a substring match or exact key match within the provided [EVIDENCE_BLOCK] metadata.

Uncertainty Language for Weak Evidence

Claims supported by low-confidence or ambiguous evidence are accompanied by uncertainty qualifiers such as 'suggests,' 'may indicate,' or 'is not conclusive.'

A claim based on weak evidence is stated as a definitive fact without any hedging language.

For each claim mapped to evidence, classify the evidence passage as strong or weak using a predefined heuristic. For weak-evidence claims, assert the presence of at least one uncertainty qualifier from a predefined list.

Output Schema Validity

Output is valid JSON conforming to the [OUTPUT_SCHEMA] with all required fields present and correctly typed.

JSON parsing fails, a required field is missing, or a field contains an incorrect data type.

Parse the output with a JSON validator. Validate against the [OUTPUT_SCHEMA] using a schema validator. Assert no errors.

Retry Termination

The retry loop terminates within [MAX_RETRIES] attempts and does not produce an output identical to a previously failed attempt.

The loop exceeds [MAX_RETRIES] or regenerates an output that failed the same constraint check in a prior attempt.

Log each retry attempt. Assert the total attempts is less than or equal to [MAX_RETRIES]. Assert the final output is not a duplicate of any previously failed output.

No Spurious Additions

Output contains no introductory or concluding commentary, apologies, or meta-commentary about the task unless explicitly requested in [CONSTRAINTS].

Output includes phrases like 'Here is the answer,' 'I hope this helps,' or 'I cannot do that' outside of the abstention format.

Check the output string for a blocklist of common spurious phrases. Assert zero matches.

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Start with the base prompt and a small set of 5-10 QA pairs with known evidence. Remove the constraint compliance scoring section initially. Use a single retry attempt with the regenerated answer printed alongside the original for manual comparison. Keep the abstention rule but skip the structured evaluation output.

code
You are a fact-checking assistant. Regenerate the following answer so that every factual claim is directly traceable to the provided evidence. If a claim cannot be supported, remove it or replace it with an explicit abstention.

[EVIDENCE]
[ORIGINAL_ANSWER]

Watch for

  • The model may still produce unsupported claims that sound plausible
  • Without structured scoring, you won't catch subtle grounding failures
  • Abstention may be too aggressive or too permissive without threshold tuning
  • Evidence passages may be misinterpreted when they contain ambiguous language
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.