Tool Retry vs Abort Decision Prompt

This prompt is for reliability engineers and agent developers who need to encode retry judgment directly into an AI agent's decision loop. The job-to-be-done is producing a binary decision—retry or abort—backed by explicit, auditable rationale when a tool call fails. The ideal user is someone integrating an agent into a production system where tool dependencies (APIs, databases, internal services) fail in ways that naive retry loops make worse. They need the agent to distinguish a transient network blip from a systemic outage, respect a retry budget, consider idempotency guarantees, and avoid cascading failures. The required context includes the error signature (status code, body, headers), the remaining retry budget, the tool's idempotency contract, and the downstream impact of both retrying and aborting.

Do not use this prompt when the retry logic is better handled in the application layer with deterministic rules (e.g., a simple HTTP 429 with a Retry-After header). It is also inappropriate for agents operating in fully synchronous, low-latency environments where a retry loop of 2-3 attempts with exponential backoff is sufficient and the cost of a wrong decision is negligible. The prompt adds value when the decision is genuinely contextual: the error is ambiguous, the retry budget is nearly exhausted, the tool is not idempotent, or aborting triggers a complex fallback chain. In high-risk domains—payments, patient data, infrastructure provisioning—the prompt's output must be treated as a recommendation that requires human approval or a secondary policy check before execution.

Before wiring this into an agent loop, define the retry budget, idempotency guarantees, and abort fallback behavior as explicit configuration, not as hidden assumptions in the prompt. Test the prompt against a golden set of borderline cases: a 500 error on an idempotent GET with budget remaining (should retry), a 409 Conflict on a non-idempotent POST with no budget left (should abort), and a timeout where the request may or may not have succeeded on the server side (the hardest case). The next section provides the copy-ready template you can adapt to your tool ecosystem.

This prompt template encodes the judgment an agent needs when a tool call returns an error. Instead of a naive retry loop that can amplify load on a degraded dependency, the prompt forces the model to reason about the error type, the remaining retry budget, idempotency guarantees, and downstream impact before producing a binary decision. The output is a structured JSON object that your application harness can parse to either re-queue the tool call, switch to a fallback, or escalate to a human operator. Use this template as the core decision node inside a retry governance layer, not as a standalone chat prompt.

textCopy
You are a tool reliability decision engine. Your only job is to decide whether a failed tool call should be retried or aborted.

## INPUT
- Tool name: [TOOL_NAME]
- Error received: [ERROR_PAYLOAD]
- HTTP status or gRPC code (if available): [STATUS_CODE]
- Retry attempt number: [ATTEMPT_NUMBER]
- Maximum retries allowed: [MAX_RETRIES]
- Is this tool call idempotent? [IDEMPOTENT]
- Downstream impact of aborting: [DOWNSTREAM_IMPACT]
- Current circuit breaker state: [CIRCUIT_STATE]
- Latency of the failed call (ms): [LATENCY_MS]
- Recent failure rate for this tool (last 5 min): [FAILURE_RATE]

## OUTPUT_SCHEMA
Return ONLY valid JSON with this exact structure:
{
  "decision": "RETRY" | "ABORT",
  "confidence": 0.0-1.0,
  "rationale": "One-sentence explanation citing the strongest factor.",
  "retry_delay_ms": integer or null,
  "escalation_recommended": true | false,
  "fallback_tool": "[FALLBACK_TOOL_NAME]" | null
}

## CONSTRAINTS
1. If the error is a client error (4xx) and NOT a 429 rate limit, always ABORT. Do not retry bad requests.
2. If the error is a server error (5xx) and the retry budget remains, prefer RETRY with exponential backoff.
3. If the circuit breaker is OPEN, always ABORT regardless of error type.
4. If the tool call is NOT idempotent and a partial success is possible, prefer ABORT to avoid double execution.
5. If the downstream impact of aborting is CRITICAL and the error is transient, prefer RETRY even if confidence is moderate.
6. If the failure rate exceeds 50% in the recent window, prefer ABORT and recommend escalation.
7. Set retry_delay_ms using exponential backoff: min(1000 * 2^attempt, 30000). Set to null if ABORT.
8. Only suggest a fallback tool if one is known and appropriate for the error type.

## EXAMPLES

Example 1: Transient server error with budget remaining
Input: Tool=payment-capture, Error=503 Service Unavailable, Status=503, Attempt=2, Max=3, Idempotent=true, Impact=CRITICAL, Circuit=CLOSED, Latency=1200, FailureRate=0.1
Output: {"decision":"RETRY","confidence":0.9,"rationale":"Transient 503 with idempotent operation and critical downstream impact; retry budget available.","retry_delay_ms":4000,"escalation_recommended":false,"fallback_tool":null}

Example 2: Client error with no retry budget
Input: Tool=user-lookup, Error=400 Bad Request, Status=400, Attempt=3, Max=3, Idempotent=true, Impact=LOW, Circuit=CLOSED, Latency=200, FailureRate=0.05
Output: {"decision":"ABORT","confidence":1.0,"rationale":"400 Bad Request is a client error; retries will not resolve the issue.","retry_delay_ms":null,"escalation_recommended":true,"fallback_tool":null}

Example 3: Circuit breaker open
Input: Tool=inventory-check, Error=504 Gateway Timeout, Status=504, Attempt=1, Max=3, Idempotent=true, Impact=MEDIUM, Circuit=OPEN, Latency=30000, FailureRate=0.8
Output: {"decision":"ABORT","confidence":1.0,"rationale":"Circuit breaker is OPEN; all calls must abort until half-open probe succeeds.","retry_delay_ms":null,"escalation_recommended":true,"fallback_tool":"inventory-cache"}

## INSTRUCTIONS
Analyze the INPUT against the CONSTRAINTS. If constraints conflict, prioritize safety (ABORT) over availability (RETRY). Return only the JSON object.

To adapt this template for your own tool ecosystem, replace the placeholder values with data from your agent's runtime context. The [ERROR_PAYLOAD] should include the raw error body so the model can distinguish between, for example, a 503 caused by a deployment rollover versus a 503 caused by persistent database unavailability. If your system does not track circuit breaker state, remove that constraint and the [CIRCUIT_STATE] placeholder, but expect more false-positive RETRY decisions during outages. The examples are critical calibration data—add at least two examples that reflect your most common borderline cases, such as a non-idempotent payment retry or a rate-limit response that looks like a server error. After copying the template, run it through your evaluation harness with the test cases described in the Testing and Evaluation section of this playbook before deploying to production.

The retry vs abort prompt is not a standalone decision engine; it is a judgment module inside a broader tool-execution harness. The harness owns the retry budget, tracks attempt counts, enforces idempotency checks, and applies the model's decision. The prompt's job is to produce a structured, auditable rationale given the current failure context. The harness must never delegate retry budget enforcement to the model—the model advises, the harness governs. This separation prevents prompt injection, hallucinated budgets, and runaway retry loops when the model misclassifies a persistent failure as transient.

Wire the prompt into the agent's tool-call error handler. When a tool returns a failure, the harness assembles the prompt context: the tool name, the error type and body, the current retry count and maximum budget, whether the operation is idempotent, the downstream impact if the call fails permanently, and any circuit-breaker state. The prompt returns a JSON decision object with action (retry, abort, escalate), rationale, suggested_delay_ms, and confidence. The harness validates this output against a strict schema before acting. If action is retry but the budget is exhausted, the harness overrides to abort and logs the conflict. If confidence is below a configurable threshold (e.g., 0.7), the harness escalates to a human or a fallback path rather than trusting a low-confidence retry. This validation layer is critical because the cost of a wrong retry in production—duplicate charges, corrupted state, cascading timeouts—is far higher than the cost of an unnecessary abort.

For high-risk domains such as payments, provisioning, or clinical data writes, add a human-in-the-loop gate before executing a retry decision. The harness can pause the workflow, surface the prompt's rationale and the raw error to a review queue, and resume only after approval. Log every decision—model output, harness override, human approval—as structured audit events with trace IDs that link the tool call, the prompt invocation, and the final action. When testing, build a harness-level test suite that simulates common failure signatures: 429 with Retry-After headers, 5xx without headers, connection resets, timeout mid-response, and malformed error bodies. Verify that the harness correctly overrides the model when the retry budget is zero, when the error is non-retryable by policy, and when the model returns malformed JSON. The harness, not the prompt, is the reliability boundary.