Tool Call Token Overhead Audit Prompt

This prompt is for agent developers and infrastructure engineers who need to understand exactly where tokens are spent when an LLM interacts with external tools. The job-to-be-done is an audit: you have production trace spans that include tool definitions, function call requests, and tool response payloads, and you need a structured breakdown of token consumption across three distinct categories—tool descriptions and parameter schemas, function call formatting, and actual tool output tokens. This is not a prompt for designing tool schemas or debugging why a tool call failed. It is a forensic cost analysis prompt that answers the question: 'How much of my context window and token budget is consumed by the machinery of tool use itself, before the model even sees the tool's response?'

The ideal user has access to raw trace data from an observability platform or logging pipeline and can extract the specific spans that represent tool definitions sent to the model, the model's function call request, and the tool's response. You should use this prompt when you suspect tool definitions are bloated, when you are evaluating whether to trim parameter descriptions or switch to a more compact function-calling format, or when you are building a FinOps dashboard that attributes costs to tool integration overhead. Do not use this prompt for real-time latency analysis, for debugging incorrect tool selection, or for evaluating the semantic quality of tool descriptions. It is strictly a token accounting tool.

Before running this prompt, assemble the required inputs: the full tool definition JSON schema as sent to the model, the raw function call request from the model's response, and the complete tool output payload. If your trace system captures these as separate spans, you will need to join them by trace ID and span sequence. The prompt expects these inputs in the [TOOL_DEFINITIONS], [FUNCTION_CALL], and [TOOL_OUTPUT] placeholders respectively. The output is a structured JSON report with token counts for each category, a total overhead calculation, and an efficiency ratio comparing tool output tokens to overhead tokens. This ratio is the key metric: a low ratio indicates you are spending more tokens on the tool interface than on the actual data the tool provides.

This prompt is designed for single-tool-call analysis within a trace. If you need to audit an entire agent loop with multiple sequential tool calls, run this prompt once per tool invocation and aggregate the results. For multi-step agent workflows, pair this with the 'Cost Attribution for Agent Loops Prompt Template' to get loop-level economics. The prompt does not require human review for low-risk internal cost analysis, but if the tool definitions or outputs contain sensitive data, apply your standard PII redaction or data handling controls before passing traces to the model. The output is deterministic enough for automated pipeline use—you can wire this into a scheduled trace analysis job that flags tool definitions exceeding a token budget threshold.

This prompt template is designed to analyze a production trace containing tool calls and produce a structured breakdown of token consumption attributable to tool definitions, parameter schemas, and function call formatting versus the actual tool output tokens. Use it when you need to quantify the overhead of your tool ecosystem and identify bloated or underutilized tool definitions that silently inflate per-request costs. The prompt expects a trace span or log entry that includes the full tool call payload—definitions sent to the model, arguments passed, and the tool's response.

textCopy
You are an AI cost auditor specializing in agent tool-call token analysis. Your task is to audit a single production trace span containing a tool call and produce a structured token overhead report.

## INPUT
Trace span JSON:
[TRACE_SPAN]

## TASK
Analyze the provided trace span and break down the token consumption into the following categories:

1. **Tool Definition Overhead**: Tokens consumed by the tool's name, description, and parameter schema (JSON Schema) that were sent to the model as part of the function calling definition.
2. **Function Call Formatting Overhead**: Tokens consumed by the model's function call request itself—the function name and serialized arguments—before the tool executes.
3. **Tool Output Tokens**: Tokens in the actual response returned by the tool after execution.
4. **Total Tool-Related Tokens**: Sum of the above three categories.
5. **Overhead Ratio**: (Definition Overhead + Formatting Overhead) / Total Tool-Related Tokens, expressed as a percentage.

## OUTPUT SCHEMA
Return a valid JSON object with this exact structure:
{
  "trace_id": "string",
  "tool_name": "string",
  "timestamp": "string (ISO 8601 if available)",
  "token_breakdown": {
    "tool_definition_overhead": 0,
    "function_call_formatting_overhead": 0,
    "tool_output_tokens": 0,
    "total_tool_related_tokens": 0,
    "overhead_ratio_percent": 0.0
  },
  "schema_complexity": {
    "parameter_count": 0,
    "nested_depth": 0,
    "has_enum_constraints": false,
    "description_length_chars": 0
  },
  "findings": [
    {
      "severity": "HIGH | MEDIUM | LOW",
      "category": "SCHEMA_BLOAT | REDUNDANT_DESCRIPTION | OVER_SPECIFICATION | UNDERUTILIZED_TOOL | FORMAT_OVERHEAD",
      "detail": "string explaining the finding"
    }
  ],
  "recommendation": "string with actionable advice to reduce overhead"
}

## CONSTRAINTS
- Estimate token counts using the standard approximation of 1 token ≈ 4 characters for English text. State this assumption in your findings if exact token counts are unavailable in the trace.
- If the trace span does not contain the full tool definition, note this as a limitation and estimate from the function call arguments and tool name only.
- Flag any tool definition where the overhead ratio exceeds 40% as HIGH severity.
- Flag any parameter schema with more than 5 top-level properties or nested depth greater than 2 as potential bloat.
- If the tool output is empty or an error, set tool_output_tokens to 0 and note the tool failure in findings.
- Do not hallucinate token counts. If data is missing, mark fields as null and explain in findings.

## EXAMPLES
[EXAMPLES]

## RISK LEVEL
[RISK_LEVEL]

To adapt this prompt for your environment, replace the square-bracket placeholders with your actual data and configuration. [TRACE_SPAN] should contain the raw JSON trace span from your observability platform (e.g., a LangSmith run, a Langfuse observation, or a custom trace log). [EXAMPLES] should include one or two representative trace spans with their expected audit outputs to ground the model's behavior—this is critical for consistent severity classification. [RISK_LEVEL] should be set to LOW for advisory cost analysis, MEDIUM if the output feeds into automated cost alerts, or HIGH if the audit results trigger automatic tool schema rollbacks or block deployments. For HIGH risk, always route the output through human review before taking action on production tool definitions.

This prompt is designed to be run offline against structured trace exports, not in the hot path of a user-facing application. The primary integration point is a batch processing job or a scheduled notebook that reads trace spans from your observability store (e.g., LangSmith, Arize, Datadog LLM Observability, or a custom logging pipeline). The prompt expects a JSON array of tool call spans, each containing the raw tool definition schema, the serialized function call arguments, and the tool's response payload. Before invoking the model, your harness must extract these spans from the trace and serialize them into the [TOOL_CALL_TRACES] placeholder. This is a read-only audit operation; the prompt does not modify any production configuration.

The implementation harness should enforce a strict pre-processing contract. Validate that each span object contains the required fields: tool_name, tool_definition (the full JSON schema sent to the model), function_arguments (the serialized arguments the model generated), and tool_output. If a span is missing any of these fields, log a warning and exclude it from the batch to prevent the model from hallucinating missing data. For cost attribution, the harness should also inject the model's per-token pricing into the [MODEL_PRICING] placeholder as a simple JSON object mapping model names to input and output costs per 1k tokens. This keeps pricing logic in the application layer, where it can be updated without changing the prompt. After the model returns its analysis, parse the JSON output and validate that the total_tokens field equals the sum of definition_tokens, argument_tokens, and formatting_overhead_tokens. If the arithmetic is inconsistent, discard the row and trigger a retry with a stricter output constraint.

For production-grade reliability, implement a two-stage validation pipeline. First, use a lightweight JSON schema validator to confirm the output conforms to the expected structure. Second, run a cross-check query against your trace database to verify that the token counts reported by the model are directionally consistent with the API's reported usage fields. Large discrepancies (greater than 15%) should be flagged for human review, as they may indicate the model miscounted tokens or the trace spans were truncated. Store the final audit results in a dedicated cost-governance table with a foreign key to the original trace ID, enabling downstream dashboards to track tool definition bloat over time. Avoid running this prompt on every single trace; instead, sample traces by tool name and version, or trigger the audit when a new tool definition is deployed.