Retry Policy

The retry condition is the rule that determines whether a failed call should be retried. It is evaluated after each attempt. Common conditions include:

• Transient errors: HTTP status codes like 429 (Too Many Requests), 500 (Internal Server Error), 502 (Bad Gateway), 503 (Service Unavailable), and 504 (Gateway Timeout).
• Network timeouts: When a call exceeds its configured timeout threshold without receiving a response.
• Specific exception types: Such as connection resets or temporary DNS failures.

Permanent errors (e.g., HTTP 400 Bad Request, 404 Not Found) should typically not trigger retries, as they indicate a client-side issue unlikely to resolve on its own.

The maximum attempts parameter defines the absolute upper limit on retries for a single operation. This is a critical guardrail to prevent:

• Cascading failures: Infinite retry loops that can overwhelm a recovering dependency.
• Resource exhaustion: Thread pools, memory, or network connections being consumed by hung operations.
• User experience degradation: Excessive latency for the end-user.

A typical configuration might be 3 to 5 total attempts (1 initial + 2-4 retries). This value should be tuned based on the error budget for the dependency and the user's tolerance for latency.

The backoff strategy dictates the waiting period between consecutive retry attempts. Its purpose is to reduce load on a failing service and increase the probability of successful recovery.

Key strategies include:

• Constant backoff: A fixed delay (e.g., 1 second) between all attempts. Simple but can cause thundering herd problems.
• Linear backoff: The delay increases by a fixed amount each attempt (e.g., 1s, 2s, 3s).
• Exponential backoff: The delay increases exponentially (e.g., 1s, 2s, 4s, 8s). This is the most common strategy for distributed systems, often combined with jitter.
• Jitter: Adding random variation to the backoff delay to prevent synchronized retry storms from multiple clients.

Idempotency handling ensures that retrying an operation does not cause unintended duplicate side effects (e.g., charging a credit card twice, creating two database records).

Mechanisms include:

• Idempotency keys: A unique client-generated key (e.g., UUID) sent with the request. The server uses this key to deduplicate and return the same result for identical requests.
• Safe HTTP methods: Retrying idempotent HTTP methods like GET, PUT, and DELETE is generally safe, while POST is not inherently idempotent.
• Application-level checks: The agent's logic or the tool's API may provide mechanisms to check if an operation was already completed before retrying.

This component is essential for agentic systems interacting with state-changing APIs.

A retry policy is often integrated with a circuit breaker pattern. The circuit breaker monitors failure rates and, when a threshold is exceeded, opens to fail fast for all subsequent calls for a period.

Integration points:

• The circuit breaker's state (open, half-open, closed) can be a retry condition. If the circuit is open, no retry is attempted; the call fails immediately.
• Successful retries can contribute to the circuit breaker's health checks when in a half-open state.
• This combination prevents retry storms from hammering a completely unresponsive dependency, allowing it time to recover and protecting the calling agent's resources.

A retry policy must be fully instrumented to provide actionable telemetry. Key observability signals include:

• Span Events: Adding events like retry.attempted and retry.succeeded to the span representing the tool call, capturing the attempt number and delay.
• Metrics: Incrementing counters for retries.attempted_total and retries.succeeded_total, tagged with the tool name and error type.
• Attributes: Adding span attributes such as retry.max_attempts, retry.backoff_ms, and the final error.retryable flag.
• Logs: Structured logs for each retry decision, including the triggering error and calculated wait time.

This data is critical for agent performance benchmarking, tuning policy parameters, and understanding dependency health.

A resilience design pattern that programmatically fails fast when calls to a tool or service are likely to fail, preventing cascading system failures. Unlike a Retry Policy which attempts to overcome transient faults, a circuit breaker stops calls entirely when a failure threshold is met, allowing the downstream service time to recover. It operates in three states:

• Closed: Calls pass through normally.
• Open: Calls fail immediately without attempting the operation.
• Half-Open: A limited number of test calls are allowed to probe for recovery. This pattern is essential for protecting an agent's runtime from being overwhelmed by retrying against a completely unavailable dependency.

A specific backoff strategy commonly used within a Retry Policy where the wait time between consecutive retry attempts increases exponentially. This is calculated using a formula like delay = base_delay * (2 ^ attempt_number). Its purpose is to:

• Reduce load on a potentially failing or overwhelmed service.
• Increase the probability that a transient issue (e.g., a throttled API, temporary network glitch) resolves before the next attempt.
• Prevent retry storms that can exacerbate an outage. For example, a policy might retry after 1 second, then 2 seconds, then 4 seconds, and so on, often with a jitter factor added to randomize delays and prevent synchronized retries from multiple clients.

A unique identifier (often a UUID) sent with a request to an external API to ensure that performing the same operation multiple times yields the same, non-duplicative result. This is a critical companion to a Retry Policy. When a retry occurs due to a timeout or network error, the original request may have already been processed by the server. An idempotency key allows the server to recognize and return the result of the first successful request, preventing side effects like:

• Double-charging a payment.
• Creating duplicate database records.
• Sending duplicate notifications. It transforms a potentially non-idempotent operation (like POST /charge) into an idempotent one for the client.

The maximum duration an agent will wait for a response from a tool or API before aborting the call. This is a foundational configuration parameter that directly interacts with a Retry Policy. A timeout defines what constitutes a 'failed' call that may be eligible for a retry. Key considerations include:

• Setting appropriate values based on the expected latency of the dependency (e.g., 100ms for a cache, 10s for a complex database query).
• Preventing resource exhaustion in the agent's runtime (e.g., thread pool starvation).
• Differentiating between a slow response (timeout) and an explicit error response (HTTP 5xx). Timeout thresholds are often layered, with a shorter connection timeout and a longer overall request timeout.

The primary Service Level Indicators (SLIs) used to measure the reliability of tool calls and trigger or evaluate Retry Policies.

• Error Rate: The ratio of failed invocations to total invocations, measured by HTTP status codes (e.g., 5xx, 429), network errors, or timeouts. A spike often indicates a problem with a dependency.
• Success Rate: The inverse, calculated as (Successful Calls / Total Calls) * 100%. This is a direct measure of reliability from the agent's perspective. These metrics are critical for:
• Defining Service Level Objectives (SLOs) for agentic systems (e.g., 99.9% success rate).
• Calculating Error Budgets to guide reliability investments.
• Automatically adjusting retry behavior (e.g., being more aggressive with retries when the success rate is above target, less aggressive when the error budget is depleted).

A persistent holding queue for messages or tool call requests that cannot be processed successfully after exhausting all retries defined by the Retry Policy. It serves as a last-resort observability and recovery mechanism. When a request fails permanently, instead of being discarded, it is placed in the DLQ with its full context and error details. This allows for:

• Manual inspection and debugging of pathological failures.
• Analysis of failure patterns to identify buggy payloads or incompatible API changes.
• Safe replay of the request once the root cause (e.g., a downstream API bug) is fixed. In agentic systems, DLQs are essential for auditing and ensuring no task is silently lost due to persistent external failures.