Refusal with Domain Expert Referral Prompt

This prompt is designed for a single, high-stakes job: refusing to answer a user's query when the available evidence is insufficient, while simultaneously preserving the full context of the interaction and generating a structured handoff payload for a human domain expert. The ideal user is an AI application builder integrating a model into a product where an incorrect or speculative answer carries significant cost—such as in clinical review, legal analysis, or financial audit workflows. The core assumption is that your application already has a defined expert routing path (e.g., a ticket queue, an escalation API, or a review dashboard). If that path does not exist, this prompt will generate a referral that leads nowhere, damaging user trust.

You should use this prompt when the cost of a wrong answer is unacceptably high and your system can actually deliver on the promise of a human follow-up. For example, in a healthcare setting, if a model cannot ground a treatment-related answer in provided clinical guidelines, it should not guess. Instead, it must refuse, explain that the evidence is insufficient, and format a handoff that includes the patient's original question, relevant context, and the specific evidence gap. This prompt is not for low-stakes chitchat, general knowledge Q&A where a simple 'I don't know' suffices, or systems that lack an operational expert escalation capability. Using it in those scenarios will create a frustrating user experience by promising a human review that will never happen.

Before implementing this prompt, you must confirm three prerequisites. First, you need a defined evidence sufficiency threshold; the prompt should only trigger when your retrieval or grounding system scores confidence below this bar. Second, you must have a downstream integration point, such as a webhook or API, that can receive the structured handoff payload and create a ticket. Third, you must establish a monitoring loop to track the rate of refusals and expert referrals, as a sudden spike can indicate a retrieval pipeline failure or a knowledge base gap. The next step is to adapt the prompt template in the following section, replacing placeholders with your specific domain taxonomy, routing instructions, and output schema.

This prompt template is designed to be wired into your application immediately after an evidence sufficiency gate returns a low-confidence result. Its job is not to answer the user's question, but to produce a transparent, actionable refusal that preserves context for a human expert. The template uses square-bracket placeholders for all dynamic inputs, making it straightforward to integrate into a prompt assembly pipeline. Before using this template, ensure you have already run your retrieval step and have a concrete set of evidence passages, even if they are insufficient.

textCopy
You are an AI assistant operating in a high-stakes domain where accuracy is critical. Your primary directive is to refuse to answer when evidence is insufficient and to facilitate a smooth handoff to a qualified human expert.

## User Query
[USER_QUERY]

## Retrieved Evidence
[RETRIEVED_EVIDENCE]

## Evidence Sufficiency Assessment
[SUFFICIENCY_ASSESSMENT]

## Constraints
- Do not speculate, infer, or synthesize an answer beyond what is directly stated in the provided evidence.
- Do not fabricate citations or sources.
- Your response must be a single JSON object conforming to the output schema.

## Output Schema
{
  "decision": "refuse_and_refer",
  "refusal_summary": "A concise, plain-language explanation of why the question cannot be answered from the available evidence. Mention specific gaps.",
  "preserved_context": {
    "original_query": "[USER_QUERY]",
    "key_entities": ["List of key entities, terms, or constraints extracted from the query"],
    "evidence_searched": "Brief description of the sources or knowledge base searched."
  },
  "expert_handoff": {
    "recommended_expertise": "The specific domain or role required (e.g., 'Senior Tax Accountant', 'Clinical Oncologist').",
    "priority": "routine | urgent | critical",
    "briefing_for_expert": "A concise summary of the user's situation, what was searched, and the specific gap preventing an AI answer. This should allow the expert to pick up the case without re-asking basic questions.",
    "suggested_next_steps": ["Actionable steps the expert or a triage system might take, e.g., 'Review patient history for prior authorization', 'Check updated regulatory filing from Q3'."]
  },
  "user_facing_message": "A polite, empathetic message to the user explaining the referral and setting expectations for follow-up."
}

To adapt this template, replace each bracketed placeholder with data from your application context. The [SUFFICIENCY_ASSESSMENT] should be the structured output from your upstream evidence gate, including its confidence score and identified gaps. The [RETRIEVED_EVIDENCE] should be the raw text of the top-k passages, as this allows the model to reference specific gaps in its refusal summary. For high-risk domains like healthcare or legal, always route the final user_facing_message through a human review step before delivery to ensure tone and accuracy. Test this prompt with cases where evidence is completely absent, partially relevant, and just below the confidence threshold to calibrate the priority field and the specificity of the briefing_for_expert.

This prompt is not a standalone chatbot response; it is a decision gate inside a production workflow. The application must first execute retrieval, pass the retrieved context and the user query to the model, and then parse the structured refusal output. The model's decision to refuse and refer should trigger a downstream action: creating a ticket in a human review queue, sending a notification to a domain expert, or logging the handoff for audit purposes. The prompt template expects [QUERY], [RETRIEVED_CONTEXT], [DOMAIN], [EXPERT_TEAM_NAME], [HANDOFF_INSTRUCTIONS], and [OUTPUT_SCHEMA] as inputs. The application layer is responsible for populating these placeholders with live data before inference.

Validation and routing logic: The prompt instructs the model to output a JSON object with decision (must be REFUSE_AND_REFER), evidence_gap_summary, handoff_package, and user_facing_message fields. After generation, the application must validate that decision matches the expected refusal value. If the model incorrectly returns ANSWER or malformed JSON, the response should be discarded, and the system should either retry with a stricter temperature setting or escalate directly to a human operator with a VALIDATION_FAILURE flag. On successful validation, extract the handoff_package and attach it to a ticket in your internal queue (e.g., Jira, ServiceNow, Zendesk). The handoff_package must include the original query, the insufficient evidence summary, and any context the expert needs to avoid rework. Do not display the raw JSON to the end user; render only the user_facing_message field in the chat interface.

Model choice and guardrails: Use a model with strong instruction-following and JSON mode capabilities (e.g., GPT-4o, Claude 3.5 Sonnet) and set temperature to 0 or a very low value (0.1) to prevent creative refusals that drift from the schema. Implement a retry budget of 2 attempts: if the first call fails schema validation, retry with the validation error injected as an additional [CONSTRAINTS] note. If the second call also fails, log the full prompt and raw output for debugging and route the interaction to a human triage queue. For high-stakes domains like healthcare or legal, add a human-in-the-loop confirmation step before the handoff ticket is created; a human reviewer should verify that the evidence gap is genuine and that referral is appropriate, preventing both over-refusal and under-refusal.

Observability and audit trail: Log every refusal event with a unique interaction_id, the full prompt payload, the validated model output, the routing decision, and the final handoff ticket ID. This log serves as an audit trail for compliance reviews and helps you monitor the refusal rate over time. Set up a dashboard alert if the refusal rate spikes suddenly, as this may indicate a retrieval pipeline failure or a knowledge base gap rather than a genuine increase in unanswerable questions. Finally, periodically review a sample of handoff packages with domain experts to ensure the context preservation is sufficient and that experts are not receiving incomplete or misleading summaries.