Dynamic Grounding Instruction Prompt for RAG

This prompt is for RAG system builders who need to dynamically adjust grounding instructions based on the quality of retrieved context at runtime. The core job-to-be-done is preventing hallucination and overconfidence when retrieval returns poor, conflicting, or sparse results. Instead of using a static 'answer from context' instruction that fails silently when retrieval quality drops, this prompt adapts its own behavioral rules—demanding higher citation density, explicit abstention, or conflict disclosure—based on live signals like source count, relevance scores, and semantic similarity between passages. The ideal user is an engineering lead or AI/ML engineer integrating a RAG pipeline into a product where unsupported claims carry business risk, such as customer-facing Q&A, internal knowledge assistants, or compliance documentation tools.

Use this prompt when your RAG pipeline can surface retrieval quality metrics before the generation step. Required inputs include the user query, the retrieved context passages, and a set of runtime quality signals: the number of sources returned, a minimum relevance threshold flag, and a conflict detection boolean. The prompt template expects these signals to be resolved in the application layer and passed as explicit variables. Do not use this prompt when retrieval quality signals are unavailable or unreliable—guessing at context quality inside the prompt without data will produce worse behavior than a well-tuned static grounding instruction. This prompt is also not a replacement for upstream retrieval improvements; if your retriever consistently fails to find relevant documents, fix retrieval before layering on dynamic instruction logic.

Before wiring this into production, define clear thresholds for what counts as 'low relevance' or 'conflict' in your specific domain. A legal document RAG system will have different tolerance for conflicting passages than a product FAQ bot. Pair this prompt with eval checks that measure groundedness, citation accuracy, and abstention rates across your quality signal bands. The next section provides the copy-ready template with placeholders you can adapt to your retrieval pipeline's specific signal vocabulary.

This template is designed to be assembled at runtime by your application layer. Before the model ever sees this prompt, your RAG pipeline should compute the variables that describe the quality of the retrieved context: how many sources were found, their relevance scores, whether they conflict, and the overall confidence level. The prompt then branches its own internal instructions based on those variables, telling the model when to answer, when to hedge, when to disclose conflict, and when to abstain entirely. This keeps the grounding logic inside the prompt where the model can follow it, while the decision about which branch to activate stays in your code where it is auditable and testable.

textCopy
You are an assistant that answers questions using only the provided retrieved context.

[CONTEXT]

[INPUT]

[OUTPUT_SCHEMA]

[CONSTRAINTS]

GROUNDING RULES (select the ONE rule block that matches the current retrieval state):

--- BEGIN RULE BLOCK: high-confidence ---
Applicable when: [SOURCE_COUNT] >= 3 AND [MIN_RELEVANCE_SCORE] >= 0.8 AND [HAS_CONFLICT] is false.
Instructions:
- Answer the question directly using the context.
- Cite every factual claim with the source ID in brackets, e.g., [S1].
- If the context implies an answer but does not state it explicitly, mark that inference clearly: "(inferred from S2, S4)".
- Do not bring in outside knowledge.
--- END RULE BLOCK ---

--- BEGIN RULE BLOCK: low-confidence ---
Applicable when: [SOURCE_COUNT] < 3 OR [MIN_RELEVANCE_SCORE] < 0.8 OR [HAS_CONFLICT] is true.
Instructions:
- Begin your answer with: "Based on the limited or conflicting information available..."
- Answer what you can, but explicitly flag any part of the answer that relies on a single low-relevance source.
- If sources disagree, summarize each conflicting position with its source ID and state that the information is inconsistent.
- Do not resolve the conflict yourself. Present the disagreement and stop.
--- END RULE BLOCK ---

--- BEGIN RULE BLOCK: no-evidence ---
Applicable when: [SOURCE_COUNT] == 0.
Instructions:
- Respond with: "I could not find information about this in the available sources."
- Do not guess, infer, or use outside knowledge.
- If the question appears to be about a different topic than the sources cover, suggest the user rephrase.
--- END RULE BLOCK ---

[EXAMPLES]

[RISK_LEVEL]

To adapt this template, replace each square-bracket placeholder with values your application computes before inference. [CONTEXT] should contain the retrieved passages with their source IDs and relevance scores inline. [SOURCE_COUNT], [MIN_RELEVANCE_SCORE], and [HAS_CONFLICT] are boolean or numeric flags your RAG pipeline must calculate. Your assembly code should read those flags and include only the single applicable RULE BLOCK in the final prompt, removing the other two blocks entirely. This prevents the model from having to choose which rule block applies, which is a common source of instruction-following errors. [OUTPUT_SCHEMA] should contain your expected JSON structure or field descriptions. [CONSTRAINTS] can include word limits, tone rules, or domain-specific prohibitions. [EXAMPLES] should contain one or two few-shot demonstrations that match the active rule block. [RISK_LEVEL] is a string like "low", "medium", or "high" that can trigger additional refusal or human-review language if your application requires it.

Before deploying, validate that your assembly code correctly selects the rule block for each retrieval state. Write eval cases that simulate high-confidence, low-confidence, and no-evidence scenarios and confirm the model follows the active block's instructions. Common failure modes include the model applying the wrong rule block when your assembly code accidentally includes two blocks, the model ignoring the abstention instruction when [SOURCE_COUNT] is zero, and the model resolving source conflicts on its own despite instructions to present the disagreement. For high-risk domains, add a human-review step when the low-confidence or no-evidence block is active. Do not rely on the model alone to decide when an answer is too uncertain to surface.

This prompt is not a standalone artifact; it is a runtime instruction generator that sits between your retrieval system and your LLM. The implementation harness must first compute the grounding signals—source count, relevance score distribution, and conflict flags—from your retriever's output before assembling the prompt. These signals should be extracted by a lightweight pre-processing function that inspects the retrieved chunks and their metadata, not by the LLM itself. The function should produce a structured grounding_profile object containing source_count (integer), relevance_range (min/max scores), conflict_detected (boolean), and low_confidence_count (integer for sources below your relevance threshold). This profile becomes the runtime input that selects which instruction block the prompt template injects: abstention language for zero sources, hedging for low-confidence results, conflict disclosure for contradictory evidence, or standard grounding for clean retrieval.

Wire the prompt into your application as a function assemble_grounding_prompt(query, retrieved_chunks, grounding_profile) that returns the final prompt string. Before calling the LLM, run a preflight validator that checks: (1) the grounding profile values are consistent with the actual retrieved chunks—if source_count is 3 but only 2 chunks were passed, abort and log a mismatch error; (2) the assembled prompt does not exceed your model's context limit—use a token counter on the final string and truncate low-relevance chunks if needed, logging which chunks were dropped; (3) all square-bracket placeholders in the template have been resolved—scan for any remaining [ tokens and fail fast if found. After the LLM responds, run a post-generation eval that checks for unsupported claims by extracting each factual assertion and verifying it appears in or is directly inferable from the retrieved context. Use an LLM-as-judge check with a strict rubric: flag any claim that cannot be traced to a specific source passage. If the eval fails, route to a retry path that re-invokes the prompt with an added [RETRY_CONTEXT] block containing the specific flagged claims and a demand for source-grounded revision. Cap retries at 2 before escalating to human review with the full trace attached.

For observability, log the grounding_profile, the final assembled prompt, the model response, and the eval result as a single trace record. Include a prompt_version identifier so you can correlate behavior changes to template updates. In high-stakes domains—legal, medical, financial—always require human review when conflict_detected is true or when low_confidence_count exceeds 50% of retrieved sources. Do not treat this prompt as a set-and-forget instruction block; monitor the distribution of grounding profiles in production to detect retrieval degradation early. If you see a rising trend in zero-source or low-confidence profiles, investigate your retrieval pipeline before adjusting the prompt.