Tool Use Policy

The foundational component that explicitly defines which external tools an agent is permitted or prohibited from calling. This is a primary safety control.

• Allowlist (Positive Security Model): The agent can only call tools explicitly listed (e.g., get_weather, calculate_shipping). This is the most secure approach.
• Denylist (Negative Security Model): The agent can call any tool except those explicitly banned (e.g., execute_shell_command, delete_database). This is less secure but offers flexibility.

Policies often combine both, using an allowlist for core functions and a denylist for absolute prohibitions.

Rules designed to manage operational expenses and prevent system overload by constraining the frequency or cumulative cost of tool calls.

• Per-Tool Rate Limits: Maximum calls per minute/hour for expensive or shared APIs (e.g., call_llm: 10/min, search_web: 100/hour).
• Session Budgets: A total cost or call limit for an entire agent session or task (e.g., total_tool_cost < $0.10).
• Cost-Aware Routing: Policy logic that selects a cheaper tool when multiple options exist for similar functionality.

These rules prevent runaway agents from incurring unexpected bills or triggering API throttling.

Pre-execution checks on the parameters an agent attempts to pass to a tool. This prevents injection attacks and malformed requests.

• Schema Enforcement: Validates that parameters match the expected data types and structures defined in the tool's OpenAPI or JSON schema.
• Value Range Checking: Ensures numeric parameters fall within safe, predefined bounds (e.g., limit between 1 and 100).
• Content Moderation: Scans string inputs for prohibited content (PII, profanity, malicious code) before passing to the tool.

This component acts as a firewall, ensuring only well-formed, safe data reaches downstream systems.

Rules that grant or restrict tool access based on the agent's identity, the user's role, or the broader task context.

• Role-Based Access Control (RBAC): A support_agent may only call lookup_customer_info, while an admin_agent may also call update_account.
• Contextual Gates: A purchase_item tool may only be callable after a verify_inventory tool has returned a successful observation.
• User Consent Checks: For tools that perform external actions (e.g., send email), the policy may require explicit user approval or confirmation from a prior step.

This moves beyond simple allowlisting to dynamic, state-aware permissioning.

Instructions defining the agent's behavior when a tool call fails, times out, or returns an unexpected result. This ensures robustness.

• Retry Logic: Rules for how many times to retry a transient failure (e.g., retry http_500 errors up to 3 times).
• Alternative Tool Selection: Specifies a fallback tool if the primary is unavailable (e.g., if search_vector_db fails, try search_keyword_index).
• Graceful Degradation: Instructions for the agent to proceed with partial information or notify the user if a critical tool is unreachable.
• Error Observability: Mandates that all tool errors are logged to a specific telemetry system for analysis.

Policy mandates that all tool-use decisions and executions are recorded in an immutable audit trail to support debugging, compliance, and oversight.

• Structured Logging: Every tool call must log: timestamp, agent_id, tool_name, parameters (sanitized), result_status, cost_incurred, and reasoning_trace_id.
• Explainability Links: The log must link the tool call to the specific reasoning step (Thought) and user query that prompted it, creating a full chain of causality.
• Retention Periods: Defines how long logs are kept for different tool categories (e.g., financial tools: 7 years, general tools: 90 days).

This component is non-negotiable for production deployments in regulated industries.

A policy enforces strict budgets and API quotas to prevent runaway costs. For example, a customer support agent may be limited to 5 paid API calls per conversation.

• Budget Caps: The agent is halted if its cumulative tool costs exceed a session or daily limit.
• Rate Limits: Tool calls are throttled (e.g., max 10 calls/minute) to avoid overwhelming backend services.
• Fallback Logic: If a paid search tool hits its limit, the policy triggers a fallback to a free, cached knowledge base.

Policies prevent agents from using tools in ways that could generate harmful or inappropriate content. This is critical for public-facing applications.

• Pre-Call Validation: Before executing a web search or image generation, the agent's query is checked against a blocklist of unsafe topics.
• Post-Call Filtering: Raw results from a retrieval tool are scanned by a safety classifier before being passed to the agent's context.
• Tool Blacklisting: Specific tools (e.g., unfiltered web browsers) are entirely prohibited for agents operating in high-trust environments like healthcare.

Policies ensure tool use complies with data residency laws (e.g., GDPR, EU AI Act) and internal privacy rules.

• Geofencing: A policy restricts database query tools to only connect to servers in specific geographic regions.
• PII Scrubbing: Before an agent can send data to an external analytics API, a pre-processing tool must redact all personally identifiable information (PII).
• Tool Whitelisting: Agents handling sensitive financial data are only permitted to call internal, audited tools and are blocked from any external SaaS APIs.

Policies dictate the mandatory order of tool calls to enforce business logic and ensure process integrity.

• Precondition Checks: An agent must call an inventory check tool and receive a confirmed_in_stock observation before it is allowed to invoke the order placement API.
• Atomic Transactions: For a multi-step workflow like booking a trip, the policy may require the flight booking tool to succeed before the hotel booking tool is unlocked, ensuring rollback capability.
• Audit Trail: Every tool call is logged with a session ID and timestamp before execution proceeds.

A robust policy defines explicit responses to tool failures, timeouts, or unexpected outputs, ensuring system resilience.

• Retry Logic: On a network timeout, the policy may allow up to 3 automatic retries with exponential backoff before declaring a failure.
• Graceful Degradation: If a primary LLM-based analysis tool is down, the policy instructs the agent to use a simpler, rule-based classifier instead.
• Human Escalation: After two consecutive tool errors, the policy triggers a human-in-the-loop step, pausing the agent and notifying an operator.

Policies grant or restrict tool access based on the agent's assigned role, the user's permissions, or the specific task context.

• Role-Based Access Control (RBAC): A SupportAgent may only use knowledge base and ticket update tools, while an AdminAgent can also access user account modification APIs.
• Dynamic Scope: For a coding assistant, the policy may allow file read/write tools only for files in the current project directory, blocking access to system files.
• Just-in-Time Approval: For high-stakes actions like database deletion, the policy requires the agent to generate a summary and get explicit user approval before the tool is enabled.

Capability grounding is the process of providing an agent with an accurate understanding of the functions, limitations, and input/output schemas of its available tools. It is a prerequisite for effective policy design.

• Tool Documentation: Involves embedding descriptions, parameter types, error conditions, and cost/rate limits into the agent's context.
• Prevents Misuse: A well-grounded agent is less likely to call tools with incorrect parameters or for unsuitable tasks, reducing policy violations.
• Dynamic Updates: In advanced systems, grounding can be updated in real-time as new tools are registered or existing ones are deprecated.

A verification step is a stage where an agent or an intermediary system checks the validity, correctness, or safety of a generated action against predefined rules before execution. It is a core technical component of a Tool Use Policy.

• Pre-Execution Check: Validates parameters for type, range, and sanity (e.g., preventing a 'delete_all' command without confirmation).
• Policy Compliance: Cross-references the intended action against an allow/deny list and checks for required human approvals.
• Fallback Trigger: A failed verification halts execution and triggers an error correction loop or a fallback mechanism.

A fallback mechanism is a predefined alternative strategy an agent executes when its primary tool call is blocked by policy, fails, or times out. Policies must define acceptable fallbacks.

• Graceful Degradation: Ensures the agent can continue operating with reduced capability rather than failing completely. Example: Using a public API if an internal one is unavailable.
• Policy-Driven: The fallback path itself is subject to policy rules (e.g., "if Tool A is denied, you may use Tool B, but not Tool C").
• User Notification: Often involves informing the user that a constrained or alternative action was taken.

A human-in-the-loop step is a deliberate pause where an agent requests input, approval, or clarification from a human before proceeding. It is a critical policy enforcement mechanism for high-stakes actions.

• Explicit Authorization: For actions with irreversible consequences (e.g., financial transactions, data deletion) or high cost.
• Policy Configuration: Policies define which tool categories or specific parameters mandate a human review.
• Breakglass Override: Allows a human operator to approve an action that would normally be blocked by automated policy, with an audit trail.

Agentic threat modeling is the security practice of identifying risks unique to autonomous systems, such as prompt injection, unintended tool cascades, and data exfiltration via tool outputs. It directly informs Tool Use Policy creation.

• Identifies Policy Requirements: Uncovers needs for input sanitization, output filtering, and strict tool sequencing rules.
• Considers Novel Attacks: Addresses threats like indirect prompt injection through tool-returned data or resource exhaustion via recursive tool calls.
• Proactive Defense: Moves beyond static API security to model the unique risks of LLM-driven, goal-directed autonomy.