Fallback Strategies

This strategy involves calling a different, functionally equivalent tool or API endpoint when the primary one fails. It is a core pattern for achieving high availability.

• Implementation: A function registry is queried for tools tagged with the same semantic capability. The agent selects the next highest-ranked option.
• Use Case: Switching from a primary payment gateway (e.g., Stripe) to a secondary provider (e.g., Braintree) during an outage.
• Consideration: Requires maintaining multiple integrations and handling potential differences in response schemas.

The system serves a previously stored, valid response instead of making a new, failing API call. This is critical for maintaining user experience during backend outages.

• Mechanism: Responses are cached with a Time-To-Live (TTL) based on data freshness requirements. On a primary call failure, the latest valid cache entry is retrieved.
• Best For: Read-heavy operations with tolerable staleness, such as product listings, reference data, or weather information.
• Limitation: Not suitable for mutable operations (POST, PUT) or highly dynamic data where staleness is unacceptable.

The agent completes the task with reduced functionality or precision when a required tool is unavailable, rather than failing entirely.

• Process: The system identifies which sub-tasks are non-critical and skips them, or uses a less accurate internal method (e.g., a model's parametric knowledge instead of a real-time search).
• Example: A travel agent cannot access live flight prices. It provides itinerary planning using known airline routes and generic pricing, explicitly stating the data is estimated.
• Design Principle: Requires careful task decomposition to isolate fallible components from core workflow logic.

Upon a tool failure, the agent is re-prompted to reformulate its plan or break the problem down differently, often without the failed tool.

• Execution: The failure and error context are injected into a new prompt, instructing the model to "step back" and reason about an alternative approach.
• Logic: "The stock API failed with a timeout error. Given the user's request to analyze Company X, what is a different way to gather or estimate the necessary financial data?"
• Advantage: Leverages the LLM's reasoning for adaptive recovery without pre-programming every contingency.

The system halts automated execution and escalates the task, along with context and the error, to a human operator for completion or triage.

• Workflow: A failed tool call triggers the creation of a ticket in a system like Jira or a message in a Slack channel, containing the user request, error logs, and agent state.
• Critical For: High-stakes operations in finance, healthcare, or customer support where incorrect automation poses significant risk.
• Integration: Requires robust audit logging and secure handoff channels between the autonomous agent and human oversight systems.

When a call to retrieve a specific parameter fails, the system substitutes a safe, predefined default value to allow progression.

• Application: Common in configuration or personalization services. If a user's profile API fails, default preferences (e.g., temperature units, region) are used.
• Safety: Defaults must be chosen to avoid harmful actions. A default for a transfer_amount should be 0, not null or a high value.
• Notification: The user should be informed that a default was applied (e.g., "Using standard settings as your profile is temporarily unavailable").

Error propagation is the strategy of forwarding exceptions or failure states from a failed tool call back to the AI agent or orchestration layer. This allows the system to reason about the error context and decide on a recovery path.

• Critical for fallback logic: The agent cannot trigger a fallback if it is unaware a primary call has failed.
• Structured error context: Propagated errors often include HTTP status codes, timeout flags, or service-specific error messages that inform the fallback decision.
• Example: A DatabaseConnectionError propagated to an agent may trigger a fallback to a cached query result, while a PermissionDeniedError may trigger a request for user credentials.

A retry policy is a set of rules governing the automatic re-attempt of a failed API call before declaring a definitive failure and triggering a fallback. These are the first line of defense against transient issues.

• Exponential backoff: The time between retries increases exponentially (e.g., 1s, 2s, 4s, 8s) to avoid overwhelming a recovering service.
• Jitter: Random variation is added to backoff intervals to prevent synchronized retry storms from many agents.
• Conditional retries: Policies are often configured to retry only on specific error types (e.g., HTTP 429 Too Many Requests, 503 Service Unavailable) but not on others (e.g., 400 Bad Request, 404 Not Found).

A circuit breaker is a resilience pattern that temporarily blocks calls to a failing service after a predefined failure threshold is met. It prevents cascading failures and allows the downstream service time to recover.

• Three states: Closed (normal operation), Open (requests fail immediately, triggering fallbacks), Half-Open (allows a test request to see if the service has recovered).
• Fallback enabler: When the circuit is Open, all requests are short-circuited to a predefined fallback strategy without attempting the doomed call.
• System-wide protection: A circuit breaker on a critical payment API would protect the financial backend from being overwhelmed during an outage, forcing all agents to use a cached ledger or queue transactions.

Agent-side caching is the temporary storage of API responses and computed results within an agent's session or memory. It is a foundational technique for implementing performance and availability fallbacks.

• Stale-while-revalidate: A common pattern where a cached, possibly stale, response is returned immediately to the user while a fresh API call is made in the background to update the cache.
• Time-to-live (TTL): Cache entries are invalidated after a period, ensuring data does not become too outdated for the use case.
• Fallback source: If a primary real-time API (e.g., live stock price feed) fails, the agent can immediately fall back to the most recent cached value with an appropriate disclaimer.

Workflow orchestration is the automated coordination, sequencing, and state management of multiple tool calls and conditional logic. It provides the framework in which fallback strategies are defined and executed.

• Conditional branching: Orchestrators manage if-then-else logic, such as "if the primary CRM API call fails, then call the legacy SOAP service."
• State persistence: Maintains context across retries and fallback attempts, ensuring the overall task goal is not lost.
• Compensation actions: In complex workflows, a fallback may require "undoing" or compensating for a partially completed action from a previous step (e.g., rolling back a provisional booking).

Tool selection is the decision-making process where an AI agent evaluates available tools against the current context. Advanced selection logic can incorporate fallback planning at the point of choice.

• Redundancy-aware selection: An agent may be aware of multiple tools that perform the same logical function (e.g., get_weather_openweathermap and get_weather_weathergov). Its selection heuristic can include reliability scores.
• Cost/quality trade-offs: A primary tool may offer high-quality data at a cost, while a fallback tool offers free, lower-quality data. The agent's selection or fallback logic can factor in the user's tolerance for quality versus cost.
• Dynamic registry: A function registry that includes metadata on tool health or latency allows for proactive selection of the most reliable tool, reducing the need for reactive fallbacks.