Cost Attribution for Agent Loops Prompt Template

This prompt is designed for agent developers and AI infrastructure engineers who need to understand the cost dynamics of multi-step autonomous workflows. The core job-to-be-done is tracing token consumption and compute cost back to individual iterations of an agent loop, identifying which steps drove the most spend and whether the agent exceeded its planned step budget. You should use this prompt when you have access to production trace data—specifically spans that capture model inference, tool calls, and step boundaries—and you need a structured, loop-level cost attribution report rather than a flat per-request summary.

The ideal user is someone who can query an observability backend (such as LangSmith, Arize, or a custom trace store) and feed a structured trace object into the prompt. Required context includes the agent's step budget, the cost-per-token rates for the models used, and the raw trace spans with their parent step identifiers. This prompt is not a replacement for real-time cost monitoring dashboards; it is a diagnostic tool for offline analysis of completed traces. Do not use it when you lack reliable step-boundary metadata in your traces, or when the agent's execution model does not produce discrete, numbered iterations—without clear loop boundaries, the attribution will be misleading.

Before running this prompt, ensure your trace data includes step_id or equivalent parent span identifiers, token counts per span, and model identifiers. If your traces are missing these fields, invest in instrumentation first. The output is a structured cost breakdown, not a real-time alert; pair it with a separate monitoring system for production guardrails. For high-cost or customer-facing agents, always review the output for unproductive loops—iterations that consumed tokens without advancing the task—and use the findings to tighten step budgets, improve tool descriptions, or add early-exit conditions to your agent logic.

This prompt template is designed to analyze a multi-step agent trace and produce a loop-level cost attribution report. It takes raw trace spans—including model inferences, tool calls, and metadata—and breaks down token consumption and estimated cost for each iteration of the agent's reasoning loop. The output is a structured table that highlights iterations exceeding planned step budgets or entering unproductive cycles, enabling FinOps and engineering teams to pinpoint cost drivers in autonomous workflows.

textCopy
You are an AI cost analyst reviewing a production trace from an autonomous agent loop. Your task is to attribute token consumption and cost to each iteration of the agent's reasoning cycle.

## INPUT
[AGENT_TRACE]

## CONSTRAINTS
- Analyze the trace as a sequence of iterations. An iteration is defined by a complete cycle of: model reasoning → tool call(s) → tool response(s) → model reasoning.
- For each iteration, calculate:
  - `iteration_number`: The 1-based index of the loop.
  - `model_input_tokens`: Total tokens in the model request (system prompt + context + tool results).
  - `model_output_tokens`: Tokens generated by the model (reasoning + tool call arguments).
  - `tool_call_count`: Number of distinct tool invocations in this iteration.
  - `tool_output_tokens`: Total tokens returned by all tool calls in this iteration.
  - `total_tokens`: Sum of input, output, and tool output tokens.
  - `estimated_cost_usd`: Cost calculated using the pricing model specified in [PRICING_MODEL].
  - `planned_step_budget`: The expected step budget for this task from [PLANNED_BUDGET].
  - `budget_status`: `within_budget` if iteration_number <= planned_step_budget, `over_budget` otherwise.
  - `cycle_assessment`: `productive` if the iteration made observable progress toward the goal, `unproductive` if it repeated actions, corrected itself without progress, or looped without new information.
- Flag any iteration where `cycle_assessment` is `unproductive` AND `budget_status` is `over_budget` as a `critical_cost_risk`.
- If the trace contains no clear iteration boundaries, state that attribution is not possible and explain why.

## OUTPUT_SCHEMA
Return a JSON object with the following structure:
{
  "trace_id": "string",
  "total_iterations": number,
  "total_cost_usd": number,
  "planned_budget_steps": number,
  "budget_compliance": "within_budget" | "over_budget",
  "critical_cost_risks": number,
  "iterations": [
    {
      "iteration_number": number,
      "model_input_tokens": number,
      "model_output_tokens": number,
      "tool_call_count": number,
      "tool_output_tokens": number,
      "total_tokens": number,
      "estimated_cost_usd": number,
      "budget_status": "within_budget" | "over_budget",
      "cycle_assessment": "productive" | "unproductive",
      "assessment_rationale": "string"
    }
  ],
  "summary": "string"
}

## PRICING_MODEL
[PRICING_MODEL]

## PLANNED_BUDGET
[PLANNED_BUDGET]

## EXAMPLES
[EXAMPLES]

## RISK_LEVEL
[RISK_LEVEL]

To adapt this template, replace the square-bracket placeholders with concrete values. [AGENT_TRACE] should contain the raw trace data, typically in JSON or a structured log format that captures model requests, responses, and tool call spans with token counts. [PRICING_MODEL] must specify the per-token or per-request pricing for each model involved (e.g., gpt-4o: $2.50/1M input tokens, $10.00/1M output tokens). [PLANNED_BUDGET] is the expected maximum number of iterations for the task, drawn from your agent's design specification or runbook. [EXAMPLES] should include one or two annotated trace snippets with correct cost attributions to guide the model's output format and judgment. [RISK_LEVEL] should be set to high if the cost data will be used for financial reporting or departmental chargeback, triggering additional validation requirements.

Before deploying this prompt, validate the output against a golden set of manually annotated traces to ensure iteration boundary detection and cost calculations are accurate. For high-risk financial workflows, route outputs through a human review step before ingestion into billing systems. Common failure modes include misidentifying iteration boundaries when tool calls are asynchronous or overlapping, applying incorrect pricing when multiple models are used in a single trace, and classifying self-correction steps as unproductive when they were necessary for task completion. Test with traces that contain parallel tool calls, empty tool responses, and edge cases where the agent terminates early or exceeds the planned budget by a wide margin.

The Cost Attribution for Agent Loops prompt is designed to operate on structured trace data, not raw log streams. Before calling the prompt, extract the relevant spans from your observability store (e.g., LangSmith, Arize, Datadog LLM Observability, or a custom trace database). Filter traces to isolate a single agent session, then serialize the loop iterations as a JSON array where each element contains: loop_id, step_number, model (with provider and version), prompt_tokens, completion_tokens, tool_call_tokens (if tracked separately), latency_ms, tool_calls_made (array of tool names), and step_outcome (completed, failed, timed_out, retried). The prompt expects this as the [TRACE_DATA] input. If your observability platform does not natively track cost, precompute cost estimates using your provider's published pricing per 1K tokens before passing data to the prompt—the model should not be asked to memorize or guess pricing tables.

Wire the prompt into a batch analysis job or an on-demand diagnostic endpoint. For batch use, run the prompt nightly against sessions that exceeded a step-count threshold (e.g., more than 10 agent iterations) or a cost threshold (e.g., sessions above $2.00). For on-demand use, expose it behind an internal tool that accepts a session_id or trace_id, fetches the relevant spans, assembles the [TRACE_DATA] payload, and calls the LLM. Validate the output against a JSON schema that requires: a loops array with loop_id, total_tokens, total_cost, step_count, exceeded_budget (boolean), unproductive_cycles (array of step ranges), and cost_per_step; plus a summary object with total_session_cost, loops_over_budget, estimated_waste, and recommendations (array of strings). If validation fails, retry once with the validation error message appended to [CONSTRAINTS]. If the second attempt also fails, log the raw response and alert the on-call channel with the trace ID for manual review.

Model choice matters here. Use a model with strong structured output support (GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro) and enable structured output mode or function calling to enforce the schema. Avoid models smaller than ~70B parameters for this task—the prompt requires cross-loop comparison, budget arithmetic, and pattern recognition that smaller models handle unreliably. Set temperature=0 to maximize consistency across runs. For high-volume environments, consider caching the prompt prefix (the instructions and schema are static) to reduce per-request cost. If your agent runs hundreds of sessions daily, sample the top 5% by cost or step count rather than analyzing every session—this catches the expensive outliers without overwhelming your analysis budget.

The output of this prompt should feed directly into your FinOps dashboard and your agent reliability monitoring. Pipe loops_over_budget and estimated_waste into cost-tracking metrics. Route unproductive_cycles to the agent engineering team as candidate loops for step-limit tightening or early-termination rules. Flag any session where exceeded_budget is true and unproductive_cycles contains more than two entries for immediate human review—these are likely agent loops that entered a degenerate state. Do not use this prompt's output to automatically modify agent configurations in production; the cost attribution is diagnostic, not prescriptive. Human review is required before adjusting step budgets, tool timeout thresholds, or loop-termination policies based on these findings.