Token Consumption by Prompt Version Analysis Prompt

This prompt is designed for FinOps engineers, AI platform teams, and prompt engineers who need to compare token consumption across multiple deployed prompt versions in production. The core job-to-be-done is cost attribution: you have traces from several prompt versions running concurrently (or in an A/B test), and you need a structured breakdown showing which versions consume the most tokens, where those tokens are spent (system prompt, user messages, tool calls, output generation), and a ranked list of versions by token efficiency. The ideal user has access to production trace data that includes prompt_version, token_counts (broken down by category), and cost_per_token metadata. Without this granular trace data, the prompt cannot produce a meaningful comparison.

Do not use this prompt when you only have a single prompt version deployed—this is a comparative analysis tool, not a single-version audit. It is also not a substitute for real-time cost monitoring dashboards; this prompt is for periodic, structured analysis of version-to-version efficiency. The prompt assumes that trace data is already collected and normalized. If your traces lack per-category token breakdowns (system, user, tool, output), you must first enrich your observability pipeline before this prompt can deliver value. For regulated environments where cost data feeds into financial reporting, always require human review of the output before it enters a formal cost allocation or chargeback process.

After running this analysis, the next step is typically to investigate the highest-cost versions using a deeper trace review prompt (such as the Token Waste Identification Prompt or the Cost Efficiency Score per Prompt Version Prompt Template) to identify specific inefficiencies. Avoid treating the ranked list as a definitive 'best version' score without also considering task success rates and latency—a version that uses fewer tokens but fails more often is not truly efficient. Always pair token efficiency findings with quality metrics before making a rollback or promotion decision.

This template is designed to be dropped directly into your observability or analysis pipeline. It expects structured trace data as input and produces a ranked breakdown of token usage by prompt version, including cost attribution and efficiency flags. Adapt the placeholders to match your internal schema, cost model, and reporting needs before use.

textCopy
You are a FinOps analyst specializing in LLM infrastructure costs. Your task is to analyze production trace data and produce a token consumption report grouped by prompt version.

## INPUT DATA
[TRACE_DATA]

## ANALYSIS PARAMETERS
- Cost per 1K input tokens: [INPUT_TOKEN_COST]
- Cost per 1K output tokens: [OUTPUT_TOKEN_COST]
- Analysis period: [TIME_RANGE]
- Minimum request count per version for inclusion: [MIN_REQUESTS]

## OUTPUT SCHEMA
Return a JSON object with the following structure:
{
  "report_metadata": {
    "analysis_period": "string",
    "total_requests_analyzed": "integer",
    "total_cost": "float",
    "cost_model": {
      "input_per_1k": "float",
      "output_per_1k": "float"
    }
  },
  "version_breakdown": [
    {
      "prompt_version": "string",
      "request_count": "integer",
      "avg_input_tokens": "float",
      "avg_output_tokens": "float",
      "avg_total_tokens": "float",
      "total_cost": "float",
      "cost_per_request": "float",
      "token_efficiency_rank": "integer",
      "anomalies": ["string"]
    }
  ],
  "efficiency_leaderboard": ["string"],
  "cost_anomalies": [
    {
      "prompt_version": "string",
      "anomaly_type": "enum[cost_spike, token_bloat, low_efficiency]",
      "description": "string",
      "recommended_action": "string"
    }
  ]
}

## CONSTRAINTS
- Only include versions present in the input trace data.
- Calculate costs using the provided per-1K token rates.
- Flag any version where avg_total_tokens exceeds [TOKEN_BLOAT_THRESHOLD] as a token bloat anomaly.
- Flag any version where cost_per_request exceeds [COST_SPIKE_THRESHOLD] as a cost spike anomaly.
- Rank versions by token efficiency (lowest avg_total_tokens per successful request gets rank 1).
- If a version has fewer than [MIN_REQUESTS] requests, exclude it from ranking but note it in a separate "low_volume_versions" array.
- Do not invent data. If a field cannot be calculated, set it to null and add a note to the anomalies array.

## EXAMPLE OUTPUT
{
  "report_metadata": {
    "analysis_period": "2025-03-01 to 2025-03-07",
    "total_requests_analyzed": 15420,
    "total_cost": 342.18,
    "cost_model": {
      "input_per_1k": 0.003,
      "output_per_1k": 0.015
    }
  },
  "version_breakdown": [
    {
      "prompt_version": "v2.1.0",
      "request_count": 8200,
      "avg_input_tokens": 1250.5,
      "avg_output_tokens": 340.2,
      "avg_total_tokens": 1590.7,
      "total_cost": 187.45,
      "cost_per_request": 0.0229,
      "token_efficiency_rank": 1,
      "anomalies": []
    }
  ],
  "efficiency_leaderboard": ["v2.1.0", "v2.0.1", "v1.9.3"],
  "cost_anomalies": []
}

## INSTRUCTIONS
1. Parse the input trace data and group requests by prompt_version.
2. For each version, calculate the average input, output, and total tokens.
3. Apply the cost model to compute total and per-request costs.
4. Check each version against the anomaly thresholds.
5. Build the efficiency leaderboard sorted by avg_total_tokens ascending.
6. Return only the JSON object. No additional commentary.

To adapt this template, replace the cost parameters with your actual model pricing and adjust the anomaly thresholds to match your budget alerts. The output schema is designed for direct ingestion into a dashboard or cost allocation system. If your trace data uses different field names, add a mapping section or preprocess the data before passing it to this prompt. For high-stakes financial reporting, always route the output through a human review step before committing to chargeback or budget decisions.

This prompt is designed to be integrated into a batch trace analysis job, not a real-time user-facing endpoint. It expects a structured trace payload containing per-step token counts, model identifiers, and prompt version tags. The ideal execution environment is a scheduled workflow (e.g., a nightly FinOps aggregation job) that queries your observability backend (such as LangSmith, Arize, or a custom trace store), extracts the relevant spans, and feeds them into the prompt for comparative analysis.

Input Assembly: Before calling the model, you must flatten raw trace data into the [TRACE_BATCH] placeholder. This JSON array should contain one object per trace, with fields: trace_id, prompt_version, model, system_tokens, user_tokens, tool_tokens, output_tokens, and cost. To avoid token limit errors, implement a pre-filtering step that limits the batch to a specific time window or a maximum of 200 traces. If the dataset is larger, shard it and run the analysis in parallel, then use a secondary prompt to merge the results. Validation: Before passing the payload, validate that all required numeric fields are present and non-negative. A malformed trace object will cause the model to hallucinate cost figures or skip the version entirely.

Model Choice and Tool Use: Use a model with strong JSON reasoning and long-context capabilities (e.g., gpt-4o or claude-3.5-sonnet). Do not provide external tools for this analysis; the model should perform the ranking and breakdown internally. However, you should post-process the output with a deterministic script. Parse the model's JSON output, recalculate the total_tokens and total_cost from the raw input data, and cross-validate these against the model's summary. If a discrepancy is found, log a warning and fall back to the deterministic calculation for the final report, using the model's output only for the qualitative efficiency notes and ranking rationale.

Logging and Human Review: Log the full prompt, the raw model response, and the post-processed output to your prompt management system. This is critical for debugging version-tagging errors. If the analysis identifies a version with a cost spike exceeding a defined threshold (e.g., 20% increase over the baseline), route the report to a human FinOps reviewer via a ticketing system before automatically flagging the version for deprecation. This human-in-the-loop step prevents automated systems from blocking a production prompt version due to a transient traffic spike or a data logging error.