Tool Result Interpretation Accuracy Prompt Template

This prompt is designed for agent platform teams who need to evaluate whether their agent correctly interprets tool outputs before acting on them. Tool result misinterpretation is one of the most common failure modes in multi-step agent workflows. An agent calls the right tool with the right arguments, receives a valid response, and then draws the wrong conclusion, extracts the wrong field, or ignores a critical error code. This prompt template produces a structured interpretation fidelity score by comparing the agent's stated understanding against the actual tool result. Use it as part of your evaluation harness when you need automated, model-graded assessment of interpretation accuracy across hundreds of agent traces.

Do not use this prompt for evaluating tool selection correctness or argument accuracy; those require separate evaluation surfaces covered by sibling playbooks like the Tool Selection Correctness Grading Prompt Template and the Function Argument Accuracy Evaluation Rubric Prompt. This prompt assumes you already have the agent's reasoning trace, the raw tool output, and the agent's subsequent action or claim that depends on that output. The ideal user is an AI engineer or evaluation lead running regression suites, pre-release checks, or production monitoring on agent pipelines where tool output drives downstream decisions.

Before using this prompt, verify that you have captured three required inputs per trace: the raw tool response payload, the agent's stated interpretation of that response, and the agent's next action or claim derived from that interpretation. If your agent does not produce an explicit interpretation step, consider adding one to your agent's reasoning trace before applying this evaluation. For high-stakes domains such as finance, healthcare, or safety-critical operations, pair this automated evaluation with human spot-checking on a sample of low-confidence scores.

This template provides a copy-ready prompt for evaluating tool result interpretation fidelity. It is designed to be dropped into an evaluation harness where a judge model compares the agent's stated understanding and subsequent actions against the actual tool output. The prompt uses square-bracket placeholders that you must replace with real values before each evaluation run. The goal is to produce a consistent interpretation fidelity score and a structured justification that can be logged, reviewed, and trended over time.

textCopy
You are an evaluation judge grading how accurately an AI agent interpreted the result of a tool call.

# CONTEXT
[AGENT_TASK_DESCRIPTION]

# TOOL CALL THAT WAS MADE
Tool name: [TOOL_NAME]
Arguments: [TOOL_ARGUMENTS]

# ACTUAL TOOL RESULT
[TOOL_RESULT]

# AGENT'S STATED INTERPRETATION OF THE RESULT
[AGENT_INTERPRETATION]

# AGENT'S SUBSEQUENT ACTION OR REASONING BASED ON THAT INTERPRETATION
[AGENT_SUBSEQUENT_ACTION]

# EVALUATION INSTRUCTIONS
Compare the agent's stated interpretation against the actual tool result. Then evaluate whether the agent's subsequent action or reasoning was appropriate given what the tool actually returned.

Score the interpretation fidelity on a 1-5 scale:
- 5: Perfect interpretation. Agent correctly understood all relevant fields, values, and implications. Subsequent action is fully appropriate.
- 4: Minor misunderstanding that did not affect the subsequent action. Agent still acted correctly.
- 3: Partial misunderstanding. Agent captured some results correctly but missed or misread one material detail. Subsequent action may be slightly off.
- 2: Significant misunderstanding. Agent misread key fields or drew wrong conclusions. Subsequent action is inappropriate or risky.
- 1: Complete misinterpretation. Agent's understanding contradicts the actual result. Subsequent action is wrong or harmful.

# OUTPUT FORMAT
Return a JSON object with exactly these fields:
{
  "score": <integer 1-5>,
  "interpretation_accuracy": "<one sentence summarizing whether the agent read the result correctly, partially, or incorrectly>",
  "action_appropriateness": "<one sentence on whether the subsequent action made sense given the actual result>",
  "key_errors": ["<list specific misinterpretations or missed details; empty array if none>"],
  "justification": "<2-3 sentence explanation connecting the score to the evidence>"
}

# CONSTRAINTS
- Only use information present in the actual tool result to judge correctness.
- Do not penalize the agent for tool result quality issues; only grade interpretation.
- If the agent correctly identified an error or empty result from the tool, that counts as accurate interpretation.
- If the agent's interpretation is reasonable but the subsequent action is wrong for other reasons, note that in action_appropriateness but do not lower the interpretation score.

Before integrating this template into your evaluation pipeline, replace each bracketed placeholder with concrete values from your agent trace. [AGENT_TASK_DESCRIPTION] should capture what the agent was trying to accomplish so the judge has full context. [TOOL_RESULT] must be the exact, unmodified output from the tool—do not summarize or truncate it, as that would invalidate the fidelity check. [AGENT_INTERPRETATION] should be extracted from the agent's reasoning trace or the message where it acknowledges the tool result. [AGENT_SUBSEQUENT_ACTION] is the next tool call, message, or decision the agent made after interpreting the result. For batch evaluation, wrap this prompt in a harness that iterates over trace segments, collects scores, and logs per-call results alongside the full trace for later audit. If the agent's interpretation is implicit rather than explicitly stated, consider adding a pre-processing step that asks the agent to verbalize its understanding before acting—this makes interpretation grading more reliable and auditable.

This prompt is designed to operate as a post-execution evaluator, not a real-time guardrail. It should be wired into an offline or async evaluation pipeline that processes agent traces after a run completes. The typical integration point is a batch evaluation job that consumes structured trace logs containing the tool call, the raw tool result, and the agent's subsequent reasoning or action. The prompt expects these three artifacts to be aligned by trace ID and step index before invocation.

For a production harness, implement a trace preprocessor that extracts the relevant span from your observability store (e.g., LangSmith, Arize, or a custom trace database). The preprocessor must assemble the [TOOL_RESULT] and [AGENT_INTERPRETATION] fields by isolating the exact tool response payload and the immediate next reasoning step from the agent. Avoid passing the entire conversation history as interpretation context—this dilutes the signal and increases token cost. Instead, extract the specific reasoning block that references the tool output. If your agent framework doesn't cleanly separate reasoning from action, add a parsing layer that identifies the first model-generated text following the tool result observation.

Validation and retries should be built around the structured output schema. The prompt returns a JSON object with an interpretation_fidelity_score (0-1), a misinterpretation_flags array, and an evidence_alignment summary. Implement a post-processing validator that checks: (1) the score is a float between 0 and 1, (2) each flag in misinterpretation_flags references a specific claim from the agent's interpretation, and (3) the evidence_alignment field contains explicit citations to the tool result. If validation fails, retry once with the same inputs and a stricter temperature setting (0.0-0.1). If the retry also fails, log the trace as unevaluable and flag for human review. For high-stakes agent workflows—such as financial transactions, healthcare data interpretation, or safety-critical tool use—route all scores below 0.8 to a human review queue before accepting the evaluation.

Model choice matters for this evaluator. Use a model with strong instruction-following and structured output capabilities (e.g., GPT-4o, Claude 3.5 Sonnet, or Gemini 2.0 Flash with JSON mode enabled). Avoid smaller models for this task because interpretation fidelity grading requires nuanced comparison between the agent's understanding and the actual tool result. If cost is a concern, sample 20% of traces with a strong model and use the results to calibrate a lighter model on the remaining 80%, but monitor for score drift weekly. Do not use this prompt as a real-time gate—the latency and cost of running a full evaluation on every tool call will degrade agent responsiveness. Instead, use it for offline quality monitoring, regression testing during prompt changes, and periodic audit sampling.

Logging and observability should capture the full evaluation payload alongside the trace. Store the interpretation_fidelity_score, misinterpretation_flags, and the evaluator's model version as trace-level metadata. This enables downstream dashboards that track interpretation accuracy trends over time, per tool, and per agent version. Set up an alert that triggers when the rolling 7-day average interpretation fidelity drops below a defined threshold (e.g., 0.85) or when a specific tool shows a sudden increase in misinterpretation flags. This turns the prompt from a one-off check into a continuous reliability signal for your agent platform.