Error Handling

These errors occur when the language model fails to produce a valid, schema-compliant function call request.

• Malformed JSON: The model outputs text that is not valid JSON syntax.
• Schema Violation: The generated JSON does not adhere to the defined function signature (e.g., missing required parameters, incorrect data types).
• Hallucinated Tools: The model invokes a function that was not defined in the available tool list.
• Incorrect Intent: The model selects the wrong tool for the user's request, despite having the correct one available.

Mitigation: Use strict output parsing with validation against the JSON Schema, implement guardrails to filter invalid calls, and improve prompt instructions with clear examples.

These errors happen during or after the tool invocation itself, when the external service is contacted.

• Network Failures: Timeouts, DNS errors, or connection refused when calling an external API.
• Authentication/Authorization: Invalid API keys, expired tokens, or insufficient permissions.
• Service Unavailability: The external tool or API returns a 5xx HTTP status code or is offline.
• Rate Limiting: The request is throttled due to exceeding quota limits.
• Resource Not Found: The API endpoint or a specific resource (e.g., a user ID) does not exist (404 error).

Mitigation: Implement robust retry logic with exponential backoff for transient errors, validate credentials, and monitor external service health.

These errors stem from invalid, missing, or malformed data provided as arguments to the function, even if the JSON structure is correct.

• Type Mismatch: Passing a string like "abc" to a parameter expecting an integer.
• Out-of-Bounds: Providing a numerical value outside an acceptable range (e.g., a negative age).
• Invalid Format: Supplying an email address or date string in an incorrect format.
• Semantic Nonsense: Arguments that are syntactically valid but logically impossible for the tool to process.

Mitigation: Apply rigorous input sanitization and validation before execution. Use type coercion where safe and implement pre-execution parameter checks within the tool wrapper.

These are systemic errors related to the flow and decision-making of the multi-step function calling process.

• Circular Dependencies: Tool A calls Tool B, which then calls Tool A, creating an infinite loop.
• State Corruption: The system's internal state becomes inconsistent between sequential tool calls.
• Non-Idempotent Retries: Retrying a failed call (e.g., a payment API) without an idempotency key, causing duplicate side effects.
• Deadlocks in Multi-Tool Orchestration: Two parallel tool calls waiting on each other's output to proceed.
• Incorrect Fallback Path: The fallback logic itself triggers an error or provides an unhelpful response.

Mitigation: Design workflows with cycle detection, manage state immutably, enforce idempotency keys for critical operations, and thoroughly test fallback pathways.

Fallback logic is a system design pattern that defines an alternative action or response when a primary function call fails or when a model cannot confidently select a tool. This is essential for maintaining user experience and system uptime.

• Primary Use: Provides a graceful degradation path when a core service is unavailable.
• Implementation Example: If a weather API call fails, the system might fall back to a cached forecast or a simpler, rule-based estimation.
• Key Consideration: Fallback logic should be deterministic and not introduce new failure modes.

Retry logic is an error-handling strategy that automatically re-attempts a failed function call. It is designed to handle transient failures, such as network timeouts or temporary service throttling.

• Common Patterns: Implement exponential backoff (e.g., wait 1s, then 2s, then 4s) to avoid overwhelming the downstream service.
• Circuit Breaker Pattern: Often paired with retries; if failures persist, the circuit 'opens' to stop calls for a period, allowing the service to recover.
• Critical Setting: A maximum retry count must be defined to prevent infinite loops from permanent failures.

In function calling, guardrails are software constraints that validate, sanitize, or block tool invocations before execution to prevent unsafe, unauthorized, or malformed operations.

• Input Validation: Checks if parameter values are within expected ranges (e.g., a zip_code parameter must be 5 digits).
• Authorization Checks: Verifies the user or session has permission to call a specific tool.
• Safety Filters: Blocks calls to tools with dangerous parameters (e.g., a database DELETE operation without a confirmed WHERE clause).

Input sanitization is the process of cleansing and validating user-provided data before it is passed as parameters to a function call. Its primary goal is to prevent security vulnerabilities.

• Prevents Injection Attacks: Critical for calls to databases, shells, or other interpreters. Sanitization strips or escapes control characters.
• Type Normalization: Converts strings to the proper type (e.g., ensuring a numeric ID is an integer, not a string) to match the function signature.
• Example: A user-provided search term '; DROP TABLE users;-- would be escaped to become a harmless literal string.

An idempotency key is a unique client-generated value (like a UUID) sent with a function or API request. It ensures that retrying the same call does not result in duplicate side effects.

• Core Purpose: Essential for reliable retry logic. The server uses the key to recognize and return the result of the original request.
• Use Case: A payment processing tool call. If a network timeout occurs after the charge is processed, a retry with the same idempotency key will not create a second charge.
• Implementation: The key is typically passed as an HTTP header (e.g., Idempotency-Key: <key>).

An execution trace is a detailed, structured log of all steps in a function calling workflow. It is the primary data source for debugging errors and implementing observability.

• Contents Includes: The user's original input, the model's reasoning/selection, the exact parameters sent, the tool's raw response, and any errors encountered.
• Debugging Value: Allows engineers to pinpoint whether a failure originated in the model's parameter extraction, the external API, or the network.
• Observability Link: Traces are fed into monitoring systems to track metrics like latency, error rates, and cost per tool call.