Low-Evidence Answer Abstention System Prompt

This prompt is for conversational AI engineers who need the model to refuse answering when retrieved evidence is too weak, too sparse, or too irrelevant to support a grounded response. The primary job-to-be-done is preventing hallucination at the policy layer—instructing the model to self-assess evidence sufficiency before generating an answer, rather than relying solely on post-hoc validators. Use this when the cost of a confident wrong answer (eroded user trust, compliance risk, safety impact) is higher than the cost of a polite refusal. It belongs in the system layer of a RAG pipeline, a customer-facing assistant, or any safety-critical application where hallucination prevention is a product requirement.

This prompt is not a post-hoc validator or a separate guard model. It is a behavioral policy that shifts the model's default posture from 'answer if possible' to 'answer only if grounded.' The ideal user is an AI engineer or product developer who controls the system prompt and can inject it before the retrieved context and user query. Required context includes a defined evidence sufficiency threshold, a retrieval pipeline that provides passages, and a clear understanding of the application's risk tolerance. Do not use this prompt when the application requires speculative creativity, open-ended brainstorming, or scenarios where the model is expected to reason beyond provided evidence. It is also unsuitable for low-latency, stateless systems where the overhead of evidence assessment violates the latency budget.

Before deploying, define your abstention triggers explicitly. Common triggers include: zero retrieval results, all passages below a relevance score, conflicting evidence with no clear resolution, or evidence that addresses a different topic than the user's question. You must also calibrate the refusal rate—an over-refusal pattern where the model abstains on answerable questions degrades user experience as much as hallucination. Implement eval checks that measure both hallucination reduction and false-abstention rate. Wire this prompt into a system that logs refusal reasons, evidence snapshots, and user feedback so you can tune the threshold over time. If the application domain is regulated or safety-critical, always include a human review escalation path for refusal cases that require expert judgment.

This system prompt defines a behavioral policy for conversational AI systems that must refuse to answer when retrieved or provided evidence falls below a defined quality threshold. It is designed to be placed in the system message of a chat completion call, ahead of any user, assistant, or tool messages. The prompt forces the model to evaluate evidence sufficiency before generating an answer, producing a structured refusal when conditions aren't met rather than a speculative or hallucinated response.

textCopy
You are an evidence-grounded assistant. Your primary directive is to answer questions ONLY when the provided evidence is sufficient to support a reliable answer. You must never speculate, infer beyond the evidence, or fabricate information.

## EVIDENCE SUFFICIENCY RULES
Before answering, evaluate the provided evidence against these thresholds. If ANY condition is not met, you MUST refuse to answer.

1. **Relevance**: At least one evidence passage must directly address the user's question. Tangential or loosely related passages do not count.
2. **Specificity**: The evidence must contain specific facts, data, or statements that answer the question. Vague or general background information is insufficient.
3. **Coverage**: All substantive sub-questions within the user's query must be addressable by the evidence. If any sub-question lacks support, the entire answer is blocked unless [PARTIAL_ANSWER_MODE] is enabled.
4. **Recency**: For time-sensitive questions, the evidence must be dated within [MAX_EVIDENCE_AGE_DAYS] days of the current date. If evidence dates are unavailable, treat recency as unknown and flag it.
5. **Authority**: Evidence from sources flagged as [LOW_AUTHORITY_SOURCES] must be corroborated by at least one source not on that list before it can be used.
6. **Conflict Resolution**: If evidence passages directly contradict each other on a material point, you must refuse to answer rather than choose sides, unless [CONFLICT_RESOLUTION_MODE] is set to 'report'.

## REFUSAL OUTPUT FORMAT
When you refuse to answer, respond with a JSON object in this exact schema:

{
  "status": "refused",
  "refusal_reason": "insufficient_evidence" | "no_evidence" | "conflicting_evidence" | "stale_evidence" | "low_authority_evidence" | "partial_coverage",
  "missing_evidence_description": "A specific, plain-language description of what evidence would be needed to answer this question.",
  "evidence_assessment": {
    "passages_reviewed": <count>,
    "relevant_passages": <count>,
    "sufficiency_score": <0.0 to 1.0>,
    "failed_conditions": ["list of condition names that failed"]
  },
  "alternative_suggestions": ["Optional. Up to 3 questions the evidence CAN answer, or null if none."],
  "escalation_recommended": <true | false>
}

## ANSWER OUTPUT FORMAT
When evidence IS sufficient, respond with a JSON object in this exact schema:

{
  "status": "answered",
  "answer": "The grounded answer text. Include inline citations as [1], [2] referencing the passage index.",
  "confidence": <0.0 to 1.0>,
  "cited_passages": [<indices of passages used>],
  "caveats": ["List any limitations, assumptions, or uncertainty notes. Empty array if none."]
}

## CONSTRAINTS
- Never output anything outside the JSON response format.
- Never include markdown fences around the JSON.
- If [RISK_LEVEL] is 'high', always set escalation_recommended to true on refusal.
- If [PARTIAL_ANSWER_MODE] is enabled, you may answer supported sub-questions while refusing unsupported ones. Mark unsupported sub-questions explicitly in caveats.
- Current date: [CURRENT_DATE]

To adapt this template, replace the square-bracket placeholders with your application's specific values. [MAX_EVIDENCE_AGE_DAYS] should reflect your domain's freshness requirements—set to 30 for news, 365 for technical documentation, or null to disable recency checks. [LOW_AUTHORITY_SOURCES] should be a list of domains or source identifiers your organization considers unreliable. [RISK_LEVEL] controls whether refusals automatically recommend human escalation; use 'high' for regulated domains like healthcare or finance. [PARTIAL_ANSWER_MODE] is a boolean that determines whether the model can answer supported sub-questions while refusing unsupported ones—enable this for customer-facing applications where partial value is better than total refusal. [CONFLICT_RESOLUTION_MODE] set to 'report' allows the model to surface contradictory evidence rather than refusing outright, which is useful for research and intelligence applications. Before deploying, run this prompt through your eval suite with test cases covering: empty evidence sets, partially relevant evidence, conflicting sources, stale documents, and boundary cases where evidence is barely sufficient. Monitor production refusal rates and false-abstention patterns to tune thresholds.

The Low-Evidence Answer Abstention System Prompt is not a standalone prompt—it is a behavioral policy layer that sits in the system instructions of a conversational AI application. It must be combined with a retrieval pipeline, a user query, and the retrieved evidence set before being sent to the model. The system prompt defines the abstention rules; the application is responsible for enforcing them, logging decisions, and handling edge cases where the model fails to comply.

Integration pattern: Place this system prompt as the top-level system message in your model request. The user message should contain the query and the retrieved evidence, formatted clearly so the model can assess sufficiency. A typical request structure looks like: [SYSTEM: Abstention policy prompt] → [USER: Query: [QUERY]\n\nRetrieved Evidence:\n[PASSAGE_1]\n[PASSAGE_2]...]. After receiving the model response, the application must run a post-processing validation step. Parse the output to detect whether the model abstained or answered. If the model answered, run a grounding check (e.g., an NLI model or a second LLM call with the Unsupported Claim Flagging Prompt) to verify that claims are supported by the provided evidence. If the model abstained, verify that the abstention reason references specific evidence gaps rather than a generic refusal—this prevents the model from taking the easy path of always refusing.

Retry and escalation logic: If the model answers when it should have abstained (a false negative on abstention), do not retry with the same prompt—the model has already committed to an answer. Instead, flag the response for human review and log the incident for prompt tuning. If the model abstains when evidence was actually sufficient (a false positive, or over-refusal), you may retry once with a modified user message that emphasizes the evidence quality, but cap retries at one to avoid frustrating users. For high-stakes domains, route all abstention decisions above a configured risk threshold to a human review queue. Model choice: This prompt works best with models that have strong instruction-following capabilities (e.g., GPT-4, Claude 3.5 Sonnet, Gemini 1.5 Pro). Smaller or less capable models may struggle with the nuanced boundary between partial evidence and insufficient evidence, leading to higher over-refusal or under-refusal rates. Test your chosen model against the boundary condition test cases before deploying.

Observability requirements: Log every decision with the following fields: query, retrieved_evidence_ids, model_response, abstention_decision (boolean), grounding_check_result (pass/fail), human_review_required (boolean), and timestamp. This audit trail is essential for tuning the abstention threshold, detecting drift, and demonstrating compliance in regulated environments. If you are using a RAG pipeline, also log the retrieval scores and the number of passages provided—low retrieval scores combined with abstention confirm the system is working correctly; low scores with confident answers indicate a dangerous failure mode that requires immediate investigation.