Uncertainty Quantification Prompt for Edge Cases

This prompt is designed for AI reliability engineers who need to decompose model uncertainty on boundary inputs before the system takes action or generates a final answer. It forces the model to separate aleatoric uncertainty (inherent randomness or ambiguity in the input itself) from epistemic uncertainty (gaps in the model's knowledge or training distribution). The primary job is to produce a structured uncertainty breakdown and escalate cases where epistemic uncertainty exceeds a defined threshold, routing them to a human review queue with diagnostic context. Use this prompt when your system faces inputs near the edges of its operating envelope, when the cost of a wrong confident answer is high, and when you need an auditable signal for human-in-the-loop decisions. This prompt belongs in pre-generation guardrails, classification gateways, and agent safety layers.

To implement this effectively, you must provide the model with a clear [RISK_LEVEL] and an explicit [EPISTEMIC_THRESHOLD] for escalation. The prompt expects a [CONTEXT] block containing the system's operating boundaries, policy constraints, and any relevant few-shot [EXAMPLES] of known edge cases. The output must conform to a strict [OUTPUT_SCHEMA] that separates the two uncertainty types, provides a total risk score, and includes a binary escalation flag. This structured output is critical for downstream routing logic; without it, the human review queue cannot prioritize or diagnose the issue efficiently. You should validate the output against the schema before allowing any automated action to proceed, and log every uncertainty assessment for audit and calibration analysis.

Do not use this prompt as a substitute for statistical calibration tooling, embedding-based drift detection, or full distributional monitoring. It is a point-in-time reasoning tool, not a population-level monitor. It is also not suitable for low-risk, high-volume classification where the cost of human review exceeds the cost of an occasional error. If your system requires real-time anomaly detection across millions of events, pair this prompt with a lightweight statistical pre-filter to reduce the volume of inputs sent to the LLM. Start by running this prompt against a golden dataset of known edge cases and safe cases to calibrate your [EPISTEMIC_THRESHOLD] before deploying it in a production guardrail.

This prompt template is designed for AI reliability engineers who need structured uncertainty quantification on edge-case inputs before a system acts autonomously. It forces the model to separate irreducible noise (aleatoric uncertainty) from knowledge gaps (epistemic uncertainty) and to produce a calibrated confidence breakdown rather than a single vague score. Replace every square-bracket placeholder with your domain-specific values before sending the prompt to the model. The output is a structured JSON object suitable for downstream routing logic—if epistemic uncertainty exceeds the configured threshold, the system should escalate to a human review queue with the full diagnostic payload.

textCopy
You are an uncertainty quantification engine operating inside a production AI system. Your job is to analyze a given input and decompose your uncertainty into structured components. You must separate aleatoric uncertainty (inherent randomness or ambiguity in the input itself) from epistemic uncertainty (gaps in your knowledge, training data, or reasoning capability). Do not guess. Do not fabricate confidence. If epistemic uncertainty is high, you must recommend escalation.

## INPUT
[INPUT]

## CONTEXT
- Domain: [DOMAIN]
- Normal operating boundaries: [NORMAL_BOUNDARIES]
- Known edge-case patterns: [KNOWN_EDGE_CASES]
- Available tools or retrieval sources: [TOOLS]

## OUTPUT SCHEMA
Return a single JSON object with the following fields:
- `aleatoric_uncertainty`: object with `score` (float 0-1), `sources` (array of strings describing irreducible ambiguity in the input), and `rationale` (string).
- `epistemic_uncertainty`: object with `score` (float 0-1), `knowledge_gaps` (array of strings describing what you don't know), `nearest_training_boundary` (string describing the closest known pattern), and `rationale` (string).
- `combined_risk_level`: one of "low", "medium", "high", "critical".
- `recommendation`: one of "proceed", "proceed_with_caveats", "escalate_to_human".
- `escalation_reason`: string, required if recommendation is "escalate_to_human".
- `diagnostic_context`: object containing `input_summary` (string), `boundary_violations` (array of strings), and `suggested_reviewer_role` (string).

## CONSTRAINTS
- If epistemic_uncertainty.score > [EPISTEMIC_THRESHOLD], recommendation must be "escalate_to_human".
- If the input contains patterns matching [ESCALATION_PATTERNS], recommendation must be "escalate_to_human" regardless of scores.
- Do not invent knowledge. If you are uncertain about a fact, classify it as epistemic uncertainty.
- If the input is ambiguous in ways that even a domain expert would find unclear, classify that as aleatoric uncertainty.
- Provide specific, actionable rationale in every field. No generic statements like "I'm not sure."

## EXAMPLES
[FEW_SHOT_EXAMPLES]

## RISK LEVEL
[RISK_LEVEL]

Adaptation guidance: Replace [INPUT] with the actual text, query, or structured payload under analysis. Set [EPISTEMIC_THRESHOLD] based on your risk tolerance—0.3 is a conservative starting point for high-stakes domains. Populate [ESCALATION_PATTERNS] with regex-like descriptions of known dangerous patterns (e.g., "requests to modify production configs", "inputs containing PII outside expected channels"). Include 2–4 [FEW_SHOT_EXAMPLES] that demonstrate the desired output shape for both low-uncertainty and high-uncertainty cases. Set [RISK_LEVEL] to "high" if the downstream action is irreversible, "medium" if it affects user-visible state, or "low" for read-only analysis. Wire the output into an application harness that parses the JSON, checks recommendation, and routes "escalate_to_human" cases to a review queue with the full diagnostic_context payload. Validate the output schema before acting—if the model returns malformed JSON, retry once with a repair prompt, then escalate.

This prompt is designed to sit at a critical decision point in your inference pipeline: after the model generates a candidate output on a boundary input, but before that output is delivered to the user or consumed by downstream systems. The harness should invoke the prompt only when a preliminary classifier or heuristic flags an input as a potential edge case—running uncertainty quantification on every request is wasteful and adds latency. The prompt expects a structured [INPUT] containing the original user query or task, the model's proposed [OUTPUT], and any relevant [CONTEXT] such as retrieved documents, tool call results, or conversation history. The harness must assemble these fields programmatically from the current request state.

The core implementation loop follows a threshold-gate-escalate pattern. After receiving the structured uncertainty breakdown from the model, your harness must parse the JSON output and extract the epistemic uncertainty score. Compare this score against a configurable threshold (start with 0.7 on a 0–1 scale and tune based on your tolerance for false escalations). If epistemic uncertainty exceeds the threshold, the harness should block the output, package the uncertainty breakdown alongside the original input into a structured escalation payload, and route it to your human review queue. If the score is below threshold but aleatoric uncertainty is high, consider appending a calibrated confidence disclaimer to the output rather than blocking entirely. The harness must also validate the prompt's output schema before acting on it—if the model returns malformed JSON or missing required fields like epistemic_uncertainty_score or uncertainty_source, retry once with a repair prompt, then escalate as a system error if the retry fails.

For production deployment, instrument the harness with structured logging that captures the input hash, both uncertainty scores, the threshold decision, and the escalation outcome. This log becomes your calibration dataset. Every two weeks, sample escalated cases and compare human reviewer dispositions against the model's uncertainty scores to measure calibration drift. If you observe that escalated cases are consistently approved by reviewers, lower the epistemic threshold; if reviewers are overwhelmed with obvious cases, raise it. Store the prompt version, model identifier, and threshold value in the log so you can attribute behavior changes to prompt updates versus threshold tuning. Never deploy this prompt without a human review queue ready to receive escalations—an uncertainty gate with no reviewer is just a silent failure mode.