Retry Budget for Partial Success Handling Prompt Template

This prompt is designed for platform teams running batch or multi-part AI workflows where sub-tasks are independent or loosely coupled. The core job-to-be-done is maximizing throughput and preserving compute by retrying only the failed portions of a job, rather than discarding all progress and starting over. Ideal users are AI platform architects, reliability engineers, and backend developers building agent harnesses, ETL pipelines with AI steps, or batch inference systems. Required context includes a clear definition of what constitutes a sub-task, a way to isolate successes from failures, and a retry budget (count, token, time, or cost) that the system is willing to spend on the failed subset.

Do not use this prompt when sub-tasks have strict ordering dependencies or shared mutable state that makes partial retry unsafe. If a failure in one sub-task invalidates the results of others, a full restart is more appropriate. Similarly, avoid this pattern for non-idempotent actions where a retry could cause duplicate side effects—use the dedicated non-idempotent action escalation prompt instead. This prompt also assumes that the model can reliably distinguish successful from failed sub-tasks; if your failure detection is noisy, invest in output validation and structured error classification before implementing partial retry logic. For high-risk domains such as healthcare, finance, or legal workflows, always include a human review step before acting on partial results, and ensure your retry budget escalation payload includes full traceability for auditors.

To implement this effectively, start by defining your sub-task boundary and success criteria in code, not just in the prompt. The prompt instructs the model on the policy, but your application harness must enforce the budget, track retry counts per sub-task, and prevent the model from exceeding its allocation. Wire the prompt into a retry loop that passes only the failed sub-tasks and their error context back to the model on each iteration. When the budget is exhausted, escalate the failed sub-tasks with a structured payload that includes retry history, error traces, and the preserved successful results. This keeps your system observable and prevents silent partial failures from reaching downstream consumers.

This prompt template is designed to be pasted into your orchestration layer and populated with variables before each retry decision cycle. It instructs the model to act as a retry policy engine that evaluates a batch of sub-tasks, identifies which succeeded and which failed, allocates remaining retry budget to the failed items, and produces a structured decision payload. The prompt preserves successful partial results and only escalates the portions that cannot be recovered within the budget.

textCopy
You are a retry policy engine for a batch AI workflow. Your job is to evaluate partial success across a set of sub-tasks, allocate remaining retry budget to failed sub-tasks, and decide whether to retry or escalate each failure.

## INPUT
- Sub-task results: [SUB_TASK_RESULTS]
- Remaining retry budget: [REMAINING_RETRY_BUDGET]
- Per-sub-task retry cost: [RETRY_COST_PER_TASK]
- Escalation threshold: [ESCALATION_THRESHOLD]
- Task priority weights: [PRIORITY_WEIGHTS]
- Error classification rules: [ERROR_CLASSIFICATION_RULES]

## CONSTRAINTS
- Preserve all successful sub-task results without modification.
- Do not retry sub-tasks that have already succeeded.
- Allocate retries to failed sub-tasks based on priority weight and retry cost.
- If a sub-task has consumed more than [MAX_RETRIES_PER_TASK] retries, mark it for escalation regardless of remaining budget.
- If the total retry cost of all eligible failures exceeds the remaining budget, select the highest-priority subset that fits within budget and escalate the rest.
- For sub-tasks marked for escalation, include the failure reason, retry count, and last error message.

## OUTPUT_SCHEMA
Return a valid JSON object with this structure:
{
  "preserved_successes": [
    {
      "task_id": "string",
      "result": "object"
    }
  ],
  "retry_queue": [
    {
      "task_id": "string",
      "retry_instruction": "string describing what to correct",
      "allocated_budget": "integer"
    }
  ],
  "escalation_queue": [
    {
      "task_id": "string",
      "failure_reason": "string",
      "retry_count": "integer",
      "last_error": "string",
      "escalation_priority": "high | medium | low"
    }
  ],
  "budget_summary": {
    "total_remaining": "integer",
    "allocated_for_retries": "integer",
    "unallocated": "integer"
  },
  "decision_rationale": "string explaining allocation choices"
}

## EXAMPLES
[EXAMPLES]

## RISK_LEVEL
[RISK_LEVEL]

If RISK_LEVEL is high, require human approval before executing the retry queue. If RISK_LEVEL is low, proceed with retries automatically but log all escalation decisions.

To adapt this template, replace each square-bracket placeholder with concrete values from your application state. The [SUB_TASK_RESULTS] should be a structured array of task IDs, statuses, outputs, and error details. The [ERROR_CLASSIFICATION_RULES] should define which error types are retryable versus immediately escalatable—for example, transient API errors may be retryable while schema violations that persist across two attempts should escalate. The [EXAMPLES] placeholder should contain one or two few-shot demonstrations showing correct budget allocation when some sub-tasks succeed and others fail. If your workflow involves non-idempotent actions, set [RISK_LEVEL] to high and ensure the escalation queue includes enough context for a human reviewer to safely resume or roll back the failed sub-tasks.

This prompt is designed to sit inside a batch processing loop where a parent task has been decomposed into multiple sub-tasks. The harness must maintain a persistent state object that tracks which sub-tasks succeeded, which failed, and how many retries have been consumed per sub-task. Before invoking the LLM, the harness should extract the current retry budget, the list of failed sub-tasks with their error contexts, and the preserved successful partial results. The prompt receives these as [SUCCESSFUL_RESULTS], [FAILED_TASKS], [RETRY_BUDGET_REMAINING], and [ERROR_CONTEXT] and returns a decision for each failed sub-task: retry, skip, or escalate.

The harness must validate the model's output against a strict schema before acting on it. Each decision object should contain a task_id, an action enum (retry | skip | escalate), and a reasoning string. If the action is retry, the harness should decrement the retry budget for that sub-task and re-invoke the original sub-task handler with any corrected parameters from the model's output. If the action is skip, the harness should mark the sub-task as permanently failed and log the skip reason for downstream reporting. If the action is escalate, the harness should package the sub-task's full error history, retry count, and the model's reasoning into an escalation payload and route it to the configured escalation target (a human queue, a dead-letter topic, or a fallback model). The harness must also enforce a global retry budget across all sub-tasks to prevent a single sub-task from consuming all available retries while others starve.

For production reliability, implement idempotency keys on sub-task execution to prevent duplicate side effects if the harness retries a sub-task that had partially succeeded. Log every retry decision with the model's reasoning, the retry budget state before and after, and the sub-task outcome. Set a hard limit on the total number of retry-budget evaluation calls to the LLM per batch to avoid runaway loops. If the model returns an invalid schema or an action that would exceed the remaining budget, the harness should reject the response, increment a validation-failure counter, and re-invoke the prompt with the validation error included in [ERROR_CONTEXT]. After three consecutive validation failures, escalate the entire batch to human review rather than continuing to consume tokens on unparseable outputs.