Uncertainty Disclosure Prompt for AI Outputs

This prompt is for product teams and AI engineers who need their application to communicate uncertainty transparently to end users. The core job-to-be-done is replacing silent model guessing or generic disclaimers with a structured, user-facing response that separates certain findings from uncertain ones, explains the source of doubt, and recommends next steps. The ideal user is an engineering lead or product manager integrating an LLM into a feature where the cost of a wrong answer is high—such as clinical summarization, financial data extraction, or legal document review—and where user trust depends on visible honesty about what the system does and does not know.

Use this prompt when your application operates in a domain where overconfidence erodes trust, creates liability, or forces users to manually verify every output. It is appropriate when the model has access to source context (via RAG, a document, or structured data) and can reason about gaps in that evidence. It is also appropriate when you can define a clear output schema that separates certain_findings from uncertain_areas, and when your UI can render that distinction meaningfully. Do not use this prompt for low-stakes queries where a simple disclaimer suffices, or when the model has no mechanism to assess its own confidence—such as open-ended creative generation with no grounding source. It is also the wrong tool when latency constraints prohibit the additional reasoning tokens required for structured uncertainty decomposition.

Before adopting this prompt, confirm that your evaluation pipeline can measure both honesty (does the model admit when it lacks evidence?) and clarity (can a user act on the structured output?). If you cannot test for false certainty—cases where the model asserts a finding confidently but is wrong—this prompt will give you a false sense of safety. Pair it with a confidence calibration check and a human review queue for any uncertain_areas that cross a risk threshold. If your application cannot support a human-in-the-loop fallback, constrain the prompt's scope to only the evidence types where the model's certainty is empirically calibrated.

This template is designed to be dropped directly into your application's prompt layer. It forces the model to structure its response around certainty boundaries rather than producing a single, overconfident block of text. The prompt uses square-bracket placeholders that your application must populate at runtime: [USER_QUESTION] for the original query, [RETRIEVED_CONTEXT] for any evidence provided by your RAG pipeline or knowledge base, and [CONFIDENCE_THRESHOLD] for the numeric score below which claims must be flagged as uncertain. The output schema is explicitly requested as a JSON object with separate certain_findings, uncertain_findings, and unknowns arrays, making it straightforward to parse and render differently in your UI—for example, displaying certain findings in a standard card and uncertain findings in a yellow warning callout.

textCopy
SYSTEM: You are an uncertainty-aware assistant. Your primary obligation is to separate what you can state with confidence from what you cannot. Never fabricate certainty. If evidence is missing, conflicting, or ambiguous, state that plainly.

USER: Answer the following question using only the provided context. Structure your response as a JSON object with three arrays:
- `certain_findings`: Claims supported by clear, consistent evidence in the context. Each claim must include a `statement` and the `source` passage that supports it.
- `uncertain_findings`: Claims where evidence is partial, conflicting, or below the confidence threshold of [CONFIDENCE_THRESHOLD]. Each claim must include a `statement`, the `evidence_available`, and a `reason_for_uncertainty` (e.g., 'single source', 'conflicting sources', 'vague language', 'inference required').
- `unknowns`: Specific questions the context cannot answer, with a `reason` explaining why (e.g., 'not mentioned in context', 'requires external knowledge', 'requires subjective judgment').

If you cannot answer the question at all, set `certain_findings` and `uncertain_findings` to empty arrays and explain in `unknowns`.

Question: [USER_QUESTION]

Context:
[RETRIEVED_CONTEXT]

Confidence Threshold: [CONFIDENCE_THRESHOLD]

Output ONLY the JSON object. Do not include any text outside the JSON.

To adapt this template, start by identifying the confidence threshold that makes sense for your domain. For high-risk workflows such as clinical or financial applications, set [CONFIDENCE_THRESHOLD] to 0.9 or higher and route all uncertain_findings to a human review queue. For lower-risk informational use cases, you might set the threshold to 0.7 and display uncertain findings with a visual caveat rather than blocking the response. If your model does not natively return logprobs, pair this prompt with a confidence extraction step—either a separate self-assessment call or a calibration check against known-outcome test cases—before populating the threshold comparison. The JSON output schema is intentionally strict; validate it in your application layer with a schema check that rejects responses missing any of the three required arrays. On validation failure, retry once with a stronger constraint instruction, then escalate to a human operator with the raw model output and the validation error attached. Never ship an unvalidated uncertainty disclosure to end users.

The Uncertainty Disclosure Prompt is not a standalone widget; it is a component inside a larger reliability workflow. In production, this prompt sits between the primary model's output and the user-facing response. The application calls the primary model, captures its answer, and then invokes this prompt with the original query, the model's answer, and any retrieved context. The output is a structured uncertainty disclosure that the application can parse and decide whether to show directly to the user, append as a disclaimer, or route to a human review queue. The key integration point is the [CONFIDENCE_THRESHOLD] parameter: if the disclosed certainty score falls below this threshold, the application should not display the answer without additional safeguards.

Wire the prompt into a post-processing step with a strict JSON schema validator. The expected output schema includes certain_findings (array of claims with high confidence), uncertain_findings (array of claims with low confidence and reasons), overall_certainty_score (0-1 float), and recommended_action (enum: display, display_with_caveats, escalate, suppress). If the model returns malformed JSON, retry once with a repair prompt that includes the raw output and the schema. If the second attempt fails, escalate the entire interaction to a human reviewer with the raw model output attached. Log every disclosure attempt, including the input, output, validation result, and final action taken, for audit and calibration analysis.

For high-risk domains—healthcare, finance, legal—never display uncertain_findings directly to end users without human review. Instead, configure the harness to route any response with overall_certainty_score below 0.85 or recommended_action equal to escalate into a review queue. The queue item should include the original query, the primary model's full response, the parsed uncertainty disclosure, and a pre-formatted decision prompt for the reviewer. Use a dead-letter queue pattern: if a review item sits unaddressed beyond a configurable timeout, surface a safe fallback message to the user rather than defaulting to the unverified AI output. Avoid the temptation to lower the threshold to reduce review volume; that trades safety for throughput in precisely the scenarios where uncertainty disclosure matters most.