Retry State Machine Prompt for Complex Agent Workflows

This prompt is for agent infrastructure developers who are building the orchestration layer for complex, multi-step agent workflows. The job-to-be-done is not simply deciding whether to retry a single failed tool call; it's about managing the entire lifecycle of a task through multiple failure domains. You need a prompt that acts as a deterministic state machine, consuming the current retry state, error context, and workflow progress to output the next valid state transition, associated actions, and guard conditions. The ideal user is an engineer wiring this prompt into a workflow engine (like Temporal, Prefect, or a custom async job runner) where the AI's output directly controls system behavior.

Use this prompt when your agent workflow has more than two retry stages (e.g., immediate retry, backoff retry, argument modification, fallback tool, human escalation, dead-letter queue). It is appropriate when the transition logic is too complex for a simple if/else block but must be auditable and free of hallucinated states. You must provide the prompt with a strict [STATE_DEFINITIONS] schema, the [CURRENT_STATE], [ERROR_CONTEXT], and [WORKFLOW_PROGRESS]. The prompt is not a general-purpose error handler; it requires a pre-defined state machine graph. Do not use this prompt for simple single-tool retries where a Retry Decision Prompt for Transient Failures would suffice, or as a replacement for a proper workflow engine's built-in retry policies.

Before implementing, map out your complete state machine on paper. The prompt's primary failure mode is suggesting a transition that is not in your defined state graph, which can cause the orchestrator to crash or loop infinitely. To mitigate this, the implementation harness must validate the output against your state definitions and reject invalid transitions. This prompt is a high-risk control-plane component; a bug here can deadlock an entire fleet of agents. Always pair it with a Retry Budget Exhaustion Escalation Prompt for terminal states and ensure every state has a defined timeout to prevent infinite loops.

This prompt template implements a deterministic retry state machine for complex agent workflows. It expects the current retry state, error context, workflow progress, and a defined state transition table as input. The model's job is not to invent retry logic but to apply the provided transition rules to the current context and output the next state, associated actions, and guard conditions in a structured format. This separation keeps retry policy auditable and testable outside the model while letting the model handle the messy work of matching real error signals to the right state transition.

textCopy
You are a retry state machine executor for an agent workflow. Your task is to evaluate the current state, error context, and workflow progress against a provided state transition table, then output the next state with associated actions and guard conditions.

## STATE TRANSITION TABLE
[STATE_TRANSITION_TABLE]

## CURRENT STATE
- Current retry state: [CURRENT_STATE]
- Retry count for current operation: [RETRY_COUNT]
- Total retries consumed in workflow: [TOTAL_RETRIES]
- Retry budget remaining: [RETRY_BUDGET]
- Workflow deadline: [DEADLINE]
- Current time: [CURRENT_TIME]

## ERROR CONTEXT
- Tool name: [TOOL_NAME]
- Error classification: [ERROR_CLASSIFICATION]
- Error message: [ERROR_MESSAGE]
- HTTP/gRPC status code (if applicable): [STATUS_CODE]
- Retry-After header value (if present): [RETRY_AFTER]
- Partial result available: [PARTIAL_RESULT_AVAILABLE]
- Idempotency status of operation: [IDEMPOTENCY_STATUS]

## WORKFLOW PROGRESS
- Completed steps: [COMPLETED_STEPS]
- Remaining steps: [REMAINING_STEPS]
- Dependency graph: [DEPENDENCY_GRAPH]
- Fallback chain for failed tool: [FALLBACK_CHAIN]
- Circuit breaker status for [TOOL_NAME]: [CIRCUIT_BREAKER_STATUS]

## OUTPUT SCHEMA
Return a valid JSON object with these fields:
{
  "next_state": "string (must match a state defined in the transition table)",
  "confidence": "float between 0.0 and 1.0",
  "actions": [
    {
      "action_type": "string (retry | fallback | escalate | abort | wait | circuit_break | partial_continue)",
      "target_tool": "string | null",
      "modified_arguments": "object | null",
      "wait_duration_seconds": "integer | null",
      "reason": "string"
    }
  ],
  "guard_conditions": [
    {
      "condition": "string (condition that must be true before executing actions)",
      "evaluation_context": "string (what to check to verify this condition)"
    }
  ],
  "escalation_packet": {
    "summary": "string (human-readable failure summary)",
    "attempted_recoveries": ["string"],
    "partial_results_summary": "string | null",
    "recommended_human_action": "string | null"
  },
  "audit_log_entry": "string (machine-readable log line for observability)"
}

## CONSTRAINTS
- Do not transition to a state not defined in [STATE_TRANSITION_TABLE].
- If [RETRY_BUDGET] is zero, do not output a retry action.
- If [DEADLINE] minus [CURRENT_TIME] is less than the estimated retry duration, prefer abort or escalate.
- If [IDEMPOTENCY_STATUS] is "non-idempotent" and [PARTIAL_RESULT_AVAILABLE] is true, include a guard condition that checks for duplicate side effects before retrying.
- If [CIRCUIT_BREAKER_STATUS] is "open", do not output a retry action for [TOOL_NAME].
- Never include credentials, tokens, or secrets in any output field.
- If confidence is below 0.7, include an escalation_packet even if the action is not escalate.

## EXAMPLES
[EXAMPLES]

To adapt this template, replace each square-bracket placeholder with data from your agent runtime. The [STATE_TRANSITION_TABLE] is the most critical input: define it as a structured object mapping each valid state to its permitted next states, transition criteria, and associated actions. The [EXAMPLES] placeholder should contain 2-4 worked examples showing correct state transitions for common failure patterns (transient timeout, auth failure, rate limit, partial result). Wire the output through a JSON schema validator before acting on it. For high-risk workflows, add a human approval gate when next_state is escalate or abort. Test this prompt against a golden set of error scenarios where the correct next state is known, and measure exact-match accuracy on next_state plus structural validity of actions and guard_conditions.

The retry state machine prompt is not a standalone decision-maker; it is a deterministic state transition engine wrapped in a language model call. The implementation harness must treat the prompt output as a structured state transition record that the orchestrator enforces, not as a suggestion. The harness owns the state machine definition, validates every transition against the allowed state graph, enforces guard conditions, and logs every transition for debugging and audit. The prompt's job is to interpret the current state, error context, and workflow progress to propose the next state and associated actions. The harness's job is to accept or reject that proposal based on hard rules.

Wire the prompt into your agent orchestrator as a dedicated state transition function. The function receives the current retry state (e.g., INITIAL_ATTEMPT, FIRST_RETRY, BACKOFF_WAIT, FALLBACK_ACTIVE, CIRCUIT_OPEN, ESCALATE_TO_HUMAN, ABORT), the error context (tool name, error classification, attempt count, retry budget remaining, deadline timestamp), and workflow progress (completed steps, remaining steps, partial results). The prompt returns a JSON object with next_state, actions (an array of tool calls, wait instructions, or escalation payloads), guard_conditions (boolean checks the harness must verify before applying the transition), and rationale. Before executing the transition, the harness validates that next_state is a legal transition from current_state using a predefined adjacency matrix. If the proposed transition is illegal, the harness logs a state machine violation and falls back to a safe default (typically ESCALATE_TO_HUMAN or ABORT depending on risk level).

Deadlock detection is critical. The harness must track the state transition history and detect cycles that indicate the agent is oscillating between states without making progress. Common deadlock patterns include FIRST_RETRY → BACKOFF_WAIT → FIRST_RETRY loops without argument modification, or FALLBACK_ACTIVE → FALLBACK_ACTIVE chains where fallback tools repeatedly fail. Implement a maximum transition count per workflow step and a cycle detector that flags repeated state sequences. When deadlock is detected, the harness forces an ESCALATE_TO_HUMAN transition with the full transition history attached. For high-risk workflows, require human approval on any transition to ABORT or CIRCUIT_OPEN states. Log every transition with a correlation ID, timestamp, current state, proposed state, validation result, and the full prompt input/output for traceability. Use structured logging that your observability stack can query—this is essential for debugging production retry storms and auditing agent behavior.

Model choice matters for this prompt. The retry state machine requires strict schema adherence and low-latency responses, not creative reasoning. Use a fast, instruction-tuned model with strong JSON mode support (e.g., GPT-4o with structured outputs, Claude 3.5 Sonnet with tool use, or a fine-tuned smaller model if your state space is limited). Set temperature=0 to eliminate variance in state transition decisions. Implement a response timeout of 2-5 seconds; if the model call exceeds this, treat it as a transient failure and retry the prompt call itself once before falling back to a hardcoded safe transition. Cache the state adjacency matrix and guard condition evaluators in the harness, not in the prompt—these are deterministic rules that should never depend on model interpretation. The prompt should only handle the fuzzy parts: interpreting error context, assessing workflow progress, and generating rationale.

Before shipping, build a test harness that exercises every legal state transition and at least three illegal transitions per state. Verify that the harness rejects illegal transitions and that deadlock detection triggers within the expected cycle count. Test with real tool failure scenarios: timeouts, 429 rate limits, 5xx server errors, malformed JSON responses, and empty result sets. Measure end-to-end latency from failure detection to next action; if the prompt call adds more than 500ms to your retry loop, consider caching common transition patterns or using a smaller model for high-frequency retry decisions. Finally, ensure your audit logs capture enough context to answer 'why did the agent retry this call seven times?' without reconstructing state from scattered log lines.