Confidence-Based Retry Decision Prompt

This prompt is designed for reliability engineers and AI platform architects who manage automated retry loops in production. Its job is to make a structured decision—retry, escalate, or abort—when a model response fails validation, returns low confidence, or triggers an error. The prompt ingests the confidence trend across attempts, the specific error type, and the remaining retry budget to produce a deterministic, cost-aware action. It is not a general-purpose error handler or a replacement for application-level circuit breakers.

Use this prompt when your system already captures per-attempt confidence scores and error codes, and you need a decision layer that prevents infinite loops or wasteful retries on unrecoverable failures. The ideal user is wiring this into an AI harness that logs each attempt, tracks a retry counter, and can route to a human review queue or a fallback model. Do not use this prompt for initial request classification, for generating the retry payload itself, or in workflows where confidence scores are unavailable. It assumes you have already attempted at least one generation and have structured metadata about the failure.

Before integrating, define your retry budget, escalation paths, and cost thresholds in the [CONSTRAINTS] placeholder. The prompt is most effective when paired with a validator that produces a standardized error code and a confidence score between 0.0 and 1.0. If your system lacks these, invest in upstream instrumentation first. For high-risk domains such as healthcare or finance, always route escalate decisions to a human review queue and log the full decision payload for audit. The next section provides the copy-ready template you can adapt to your error taxonomy and escalation policy.

This template provides the core instruction set for a model to act as a retry decision engine. It evaluates the confidence trend across previous attempts, the nature of the error, and the remaining retry budget to produce a structured decision: retry, escalate, or abort. The prompt is designed to be wired into an application's retry loop, receiving structured data about the failure history and returning a machine-readable decision. Use this template as the foundation, adapting the placeholders to your specific error taxonomy, confidence scoring method, and escalation paths.

textCopy
You are a retry decision engine for a production AI system. Your job is to analyze the confidence trend and error history across multiple attempts for a given task and decide whether to retry, escalate to a human, or abort the operation.

## Input Data
- **Task Description:** [TASK_DESCRIPTION]
- **Attempt History (chronological):**
[ATTEMPT_HISTORY]
  - Each attempt includes: attempt number, error type, error message, confidence score (0.0-1.0), and a summary of the model's output.
- **Retry Budget:** [MAX_RETRIES] total attempts allowed. [CURRENT_ATTEMPT_COUNT] attempts have been made.
- **Error Taxonomy:** [ERROR_TAXONOMY]
  - Recoverable errors: [LIST_RECOVERABLE_ERROR_TYPES]
  - Non-recoverable errors: [LIST_NON_RECOVERABLE_ERROR_TYPES]
- **Escalation Paths:** [ESCALATION_PATHS]
- **Risk Profile:** [RISK_LEVEL] (low, medium, high, critical)

## Decision Rules
1. **Abort immediately** if the latest error type is non-recoverable.
2. **Abort immediately** if the retry budget is exhausted ([CURRENT_ATTEMPT_COUNT] >= [MAX_RETRIES]).
3. **Escalate to human** if the confidence score has decreased for [CONFIDENCE_DECLINE_THRESHOLD] consecutive attempts, even if retries remain.
4. **Escalate to human** if the risk profile is 'critical' and the latest confidence score is below [CRITICAL_RISK_CONFIDENCE_FLOOR].
5. **Retry** if the error is recoverable, the retry budget remains, and the confidence trend is stable or improving (no sustained decline).
6. **Retry** with a modified strategy if the error is recoverable but the same error type has occurred for [REPEAT_ERROR_THRESHOLD] consecutive attempts. Suggest a specific modification.

## Output Schema
Respond with a single JSON object:
{
  "decision": "retry" | "escalate" | "abort",
  "rationale": "A concise explanation referencing the specific decision rule that triggered.",
  "confidence_trend": "improving" | "stable" | "declining" | "insufficient_data",
  "suggested_retry_modification": "If decision is 'retry' and a modification is needed, describe it here. Otherwise null.",
  "escalation_details": {
    "reason_code": "CONFIDENCE_DECLINE" | "CRITICAL_RISK" | "REPEATED_FAILURE" | "BUDGET_EXHAUSTED" | null,
    "handoff_context": "A summary of the task, last error, and confidence trend for the human reviewer."
  }
}

## Constraints
- Do not suggest a retry if the budget is exhausted or the error is non-recoverable.
- For 'critical' risk tasks, prefer escalation when confidence is low, even if retries remain.
- The rationale must cite a specific decision rule number.
- Output only the JSON object. No other text.

To adapt this template, start by defining your [ERROR_TAXONOMY]. Classify errors your system encounters into recoverable (e.g., transient API failures, malformed JSON that can be repaired) and non-recoverable (e.g., authentication failures, content policy violations). Next, calibrate the [CONFIDENCE_DECLINE_THRESHOLD] and [CRITICAL_RISK_CONFIDENCE_FLOOR] based on your system's tolerance for degraded outputs. For high-stakes domains like healthcare or finance, set these thresholds conservatively. The [ATTEMPT_HISTORY] should be a serialized list of structured objects your application compiles before calling this prompt. Ensure the confidence scores you pass in are generated by a consistent method, such as model self-assessment logprobs or an external classifier, to make the trend analysis meaningful. Finally, map the [ESCALATION_PATHS] to actual queues, Slack channels, or ticketing systems in your implementation harness.

The Confidence-Based Retry Decision Prompt is designed to sit inside a retry harness, not as a standalone call. The harness should invoke this prompt only after an initial failure or low-confidence output has been detected by an upstream validator, evaluator, or confidence scorer. The prompt expects three structured inputs: the confidence trend across previous attempts, the error type or failure reason, and the remaining retry budget. These inputs must be assembled programmatically before the prompt is called. Do not pass raw logs or unstructured error strings directly; the prompt's reliability depends on receiving normalized, machine-readable input fields.

Wire the prompt into a retry loop with a hard ceiling on total attempts. Before each retry decision, construct the input payload from the retry state object: [CONFIDENCE_TREND] should be a JSON array of confidence scores from prior attempts (e.g., [0.72, 0.68, 0.65]), [ERROR_TYPE] should be a controlled vocabulary string such as schema_validation_failed, tool_call_invalid, citation_missing, or low_confidence, and [RETRY_BUDGET_REMAINING] should be an integer count of remaining allowed attempts. The model returns a structured decision with an action field (retry, escalate, abort), a rationale string, and an optional retry_instructions object. Validate the output against this schema before acting on it. If the output fails schema validation, treat it as an escalate decision and log the parse failure. For high-risk domains, require a human reviewer to confirm any abort decision before the workflow terminates.

Log every decision this prompt makes, including the input state, the raw model output, the parsed decision, and the eventual outcome of the chosen action. This log becomes your audit trail for tuning retry budgets and escalation thresholds. Implement a circuit breaker that forces escalation if the retry loop exceeds a wall-clock time limit, regardless of the prompt's decision. Common failure modes include the model recommending retry when confidence is flatlining, or recommending escalate prematurely when a simple prompt reformulation would succeed. Test against a golden set of retry scenarios with known correct decisions before deployment, and monitor the ratio of retry to escalate decisions in production to detect drift in the model's risk tolerance.