Evidence Threshold Decision Prompt for Safety-Critical Answers

This prompt is for engineering teams building AI systems in regulated or high-stakes domains where a wrong answer carries significant risk. Its job is to act as a policy-enforcement gate: it produces a binary decision (ANSWER or REFUSE) based on whether retrieved evidence meets a configurable sufficiency threshold. The ideal user is a production AI architect or backend engineer integrating this prompt into a RAG pipeline, a customer-support copilot, or a clinical decision-support tool. Required context includes the user's query, the full set of retrieved evidence passages, and a domain-specific sufficiency policy that defines what 'enough evidence' means for your use case. You should not use this prompt for open-ended creative tasks, general chatbot interactions, or low-stakes Q&A where a speculative answer is acceptable. It is also not a replacement for a full output-validation layer; it decides whether to proceed, not whether the final answer is faithful.

In practice, you wire this prompt as a synchronous gate immediately after retrieval and before answer generation. The model receives the query, the evidence set, and a structured policy definition—such as 'minimum 2 corroborating sources from authoritative journals published within 5 years' for a medical application, or 'at least one directly on-point regulatory filing' for a legal use case. The output is a strict JSON object with an action field set to ANSWER or REFUSE, a rationale string that cites specific evidence gaps or sufficiency, and an escalation_path field that recommends human review when the threshold is not met. This structured output is then parsed by your application to either proceed to answer generation or return a refusal response to the user. For high-risk domains, you must log the full prompt, evidence, and decision payload for audit trails. A human-review queue should be triggered automatically when the action is REFUSE and the escalation_path indicates a critical gap.

The primary failure mode to avoid is deploying this prompt without a well-defined, domain-validated sufficiency policy. A vague instruction like 'refuse if evidence is insufficient' will produce inconsistent and brittle results. Instead, invest time with domain experts to define concrete, measurable criteria for sufficiency. Also, do not use this prompt as a standalone safety guard; it should be one component in a defense-in-depth strategy that includes output faithfulness verification, hallucination detection, and human-in-the-loop review for high-severity use cases. If your application can tolerate a partially correct answer with caveats, consider using a 'Partial Answer with Caveats' prompt instead of a binary gate. Next, proceed to the prompt template section to copy and adapt the exact instructions for your domain.

This template is the core instruction set for an Evidence Threshold Decision Prompt. It forces the model to act as a binary gate—either the evidence is sufficient to answer, or it is not—and to produce a structured rationale that can be logged, audited, and escalated. The prompt is designed to be dropped into a production pipeline where the model's output is parsed by application code before any answer reaches a user. Every placeholder must be populated with domain-specific values before deployment; leaving a placeholder unresolved will cause the model to hallucinate policy.

textCopy
You are an evidence sufficiency gate for a [DOMAIN] application. Your only job is to decide whether the provided evidence is sufficient to answer the user's question at the required threshold for this domain.

## INPUT
User Question: [USER_QUESTION]
Retrieved Evidence Passages:
[EVIDENCE_PASSAGES]

## THRESHOLD DEFINITION
A sufficient answer must meet ALL of the following criteria:
- [CRITERION_1]
- [CRITERION_2]
- [CRITERION_3]

## CONSTRAINTS
- Do not answer the user's question. Only decide sufficiency.
- If any criterion is not met, the evidence is INSUFFICIENT.
- Do not infer, extrapolate, or fill gaps. Only use the provided passages.
- If passages contradict each other, treat the evidence as INSUFFICIENT unless [CONTRADICTION_POLICY].

## OUTPUT SCHEMA
Respond with a single JSON object matching this schema:
{
  "decision": "SUFFICIENT" | "INSUFFICIENT",
  "confidence": 0.0-1.0,
  "criteria_assessment": {
    "[CRITERION_1_LABEL]": {"met": true | false, "rationale": "string"},
    "[CRITERION_2_LABEL]": {"met": true | false, "rationale": "string"},
    "[CRITERION_3_LABEL]": {"met": true | false, "rationale": "string"}
  },
  "missing_evidence_summary": "string, empty if SUFFICIENT",
  "escalation_recommended": true | false,
  "escalation_reason": "string, empty if no escalation"
}

## ESCALATION RULES
Recommend escalation to human review when:
- [ESCALATION_CONDITION_1]
- [ESCALATION_CONDITION_2]

## EXAMPLES
[FEW_SHOT_EXAMPLES]

Adaptation guidance: Replace [DOMAIN] with a specific label like "clinical decision support" or "financial compliance." Populate [CRITERION_1] through [CRITERION_3] with measurable, domain-specific conditions—for example, "Evidence is from a source published within the last 12 months" or "At least two independent passages corroborate the finding." The [CONTRADICTION_POLICY] placeholder should resolve to a clear rule such as "one passage is from a higher-authority source and explicitly refutes the other." [FEW_SHOT_EXAMPLES] must include at least one SUFFICIENT and one INSUFFICIENT example with realistic evidence passages. [ESCALATION_CONDITION_1] and [ESCALATION_CONDITION_2] should capture high-risk scenarios like "decision would affect patient safety" or "regulatory reporting obligation exists."

Before wiring this prompt into a production pipeline, validate that the output JSON parses correctly across at least 50 test cases covering boundary conditions: borderline sufficiency, empty evidence sets, contradictory passages, and queries that require domain knowledge the evidence does not contain. If the model fails to produce valid JSON or misclassifies sufficiency in more than 2% of test cases, tighten the criteria definitions or add more few-shot examples before proceeding. Never deploy this gate without a human-review queue for all escalated decisions and a logging path that captures the full prompt, model response, and final routing action for audit.

This prompt is not a conversational nicety; it is a binary gate in a safety-critical pipeline. The implementation harness must treat the model's output as a structured decision that controls whether an answer is released to a user, held for review, or refused entirely. The harness is responsible for enforcing the schema, validating the rationale against the provided evidence, and ensuring that every REFUSE or ESCALATE decision is auditable. In regulated domains, the harness should never silently override a refusal.

Wire the prompt as a synchronous step after evidence retrieval and before answer generation. The input to this step is the user's query, the retrieved evidence set, and the domain-specific threshold configuration. The model must return a strict JSON object with decision (ANSWER, REFUSE, ESCALATE), confidence_score, evidence_sufficiency_rationale, and missing_evidence fields. Implement a post-processing validator that checks: (1) the JSON schema is valid, (2) the decision field matches one of the allowed enum values, (3) the confidence_score is a float between 0 and 1, and (4) if the decision is ANSWER, the evidence_sufficiency_rationale explicitly references at least one piece of provided evidence. If validation fails, retry once with the validation error injected into the prompt as a correction instruction. If the retry also fails, force an ESCALATE decision and log the failure for immediate review.

Log every decision with the full prompt, the raw model output, the validation result, and the final routed action. For ESCALATE decisions, push a structured payload to a human review queue containing the original query, the evidence set, the model's rationale, and a direct link to the audit log. Do not cache or reuse threshold decisions across queries, as evidence sufficiency is query-specific. Set a strict latency budget for this gate—typically under 2 seconds—and monitor the rate of forced escalations due to validation failures as a key reliability metric. If the forced escalation rate exceeds 1%, investigate prompt drift or model behavior changes immediately.