Tool-Call Sequence Diagnosis Prompt

This prompt is designed for platform engineers and AI SREs who need to diagnose a single agent trace where tool-call behavior deviated from expectations. The primary job-to-be-done is to take a raw, captured trace span from an agent session and produce a structured fault report. This report must identify incorrect tool selection, malformed arguments, or missing calls, and it must provide root-cause hypotheses. Use this prompt when you have isolated a problematic session and need a deep, forensic analysis before you aggregate data across multiple sessions or escalate to an incident. The ideal user has access to the raw trace data, understands the expected tool schemas, and needs a repeatable diagnostic process.

This prompt is not a replacement for real-time monitoring or alerting systems. Do not use it for live traffic analysis, aggregate trend detection, or as a substitute for a production observability dashboard. It is also not suitable for diagnosing issues that span multiple agent sessions, such as a systemic model drift problem, which would require a different, aggregation-focused analysis. The prompt assumes the fault is isolated to a single trace and that the root cause is likely within the tool-call sequence itself—not a broader infrastructure failure, a network timeout, or a model serving error. If you haven't already captured the full trace with tool-call arguments and responses, you must do that first; this prompt is for analysis, not data collection.

To get the most out of this prompt, you should provide the complete trace data, including the system prompt, user input, and the full sequence of tool calls with their arguments and responses. The prompt is most effective when you also supply the expected tool schemas as a [TOOLS] variable, allowing the model to validate the actual calls against the contract. The output will be a structured report, which you should validate against your own knowledge of the system. In high-stakes production environments, always treat the model's root-cause hypothesis as a starting point for your own investigation, not a final verdict. The next step after receiving the diagnosis is to reproduce the issue in a controlled test environment using the identified failure conditions.

This prompt template is designed to be pasted directly into your trace review workflow. It instructs the model to act as a platform engineer auditing a single agent trace, focusing exclusively on the sequence of tool calls, their arguments, and the responses they received. The goal is to produce a structured fault report that identifies incorrect tool selection, malformed arguments, or missing calls, along with root-cause hypotheses. Before using this prompt, ensure you have extracted the relevant spans from your observability platform (e.g., LangSmith, Arize, or a custom trace store) and have the expected tool schemas available for validation.

codeCopy
You are a platform engineer diagnosing an agent trace failure. Your task is to analyze the provided tool-call sequence and identify any faults in tool selection, argument formatting, or call ordering.

## INPUT

**Trace Spans (Chronological):**
[TRACE_SPANS]

**Expected Tool Schemas:**
[TOOL_SCHEMAS]

**Agent's Stated Goal:**
[AGENT_GOAL]

## INSTRUCTIONS

1.  **Reconstruct the Sequence:** Parse the [TRACE_SPANS] to build a chronological list of every tool call, including the tool name, input arguments, and the response or error received.
2.  **Validate Against Schemas:** For each tool call, compare the provided arguments against the expected schema in [TOOL_SCHEMAS]. Flag any missing required parameters, incorrect data types, or hallucinated parameters.
3.  **Evaluate Selection and Order:** Assess whether the correct tool was chosen for the step and if the sequence of calls is logical for achieving the [AGENT_GOAL]. Identify any missing but necessary calls or calls that are redundant or out of order.
4.  **Formulate Root-Cause Hypotheses:** Based on the identified faults, propose the most likely root cause for the failure. Prioritize hypotheses that are specific and testable (e.g., 'The system prompt omitted a required parameter for the search tool' rather than 'The agent made a mistake').

## OUTPUT FORMAT

Respond with a JSON object conforming to this schema:
{
  "reconstructed_sequence": [
    {
      "step": number,
      "tool_name": "string",
      "arguments": {},
      "response_or_error": "string",
      "schema_validation_errors": ["string"]
    }
  ],
  "faults_identified": [
    {
      "step": number,
      "fault_type": "incorrect_selection | malformed_arguments | missing_call | redundant_call | ordering_error",
      "description": "string",
      "evidence": "string"
    }
  ],
  "root_cause_hypotheses": [
    {
      "hypothesis": "string",
      "confidence": "high | medium | low",
      "rationale": "string",
      "suggested_test": "string"
    }
  ],
  "overall_assessment": "string"
}

## CONSTRAINTS
- Base your analysis strictly on the provided [TRACE_SPANS] and [TOOL_SCHEMAS]. Do not invent information.
- If the trace is incomplete or missing critical spans, note this in the `overall_assessment` and flag it as a limiting factor for your diagnosis.
- Do not suggest fixes that require changing the agent's core reasoning logic; focus on prompt, schema, and configuration adjustments.

To adapt this template, replace the square-bracket placeholders with real data. [TRACE_SPANS] should contain the raw JSON or a structured text representation of the tool-use spans from your trace. [TOOL_SCHEMAS] should be the complete JSON Schema definitions for every tool the agent had access to. [AGENT_GOAL] is the user's original request or the agent's top-level objective for that session. After running the prompt, you must validate the output JSON against the provided schema. For high-stakes debugging, pair this automated diagnosis with a manual review of the trace to confirm the model's hypotheses before implementing any prompt or configuration changes.

This prompt is designed to be triggered programmatically when an agent trace flags an error or when an engineer pulls a specific trace ID for manual review. The implementation harness should fetch the raw trace data from your observability platform—such as LangSmith, Arize, or a custom logging pipeline—and inject it into the [TRACE_JSON] placeholder. The prompt expects a structured JSON array of tool calls, including timestamps, function names, arguments, and responses. Before sending the prompt, validate that the trace payload contains all required fields; if the trace is incomplete, prepend a warning to the [CONTEXT] block so the model can flag missing data as part of its diagnosis.

The harness should enforce a strict output contract by appending the [OUTPUT_SCHEMA] directly to the prompt and validating the model's response against it before surfacing results to the engineer. Use a JSON Schema validator to check that the fault_report contains the required root_cause_hypotheses array, that each hypothesis includes a confidence score between 0 and 1, and that the sequence_analysis correctly references the tool call indices from the input trace. If validation fails, implement a retry loop with a maximum of two additional attempts, appending the validation error message to the prompt as feedback. For high-severity faults—such as malformed arguments that could cause downstream data corruption—route the validated report to a human reviewer via a ticketing system or Slack notification before any automated remediation is triggered.

Log every diagnosis attempt, including the raw trace, the prompt version, the model response, and the validation outcome, to an audit table. This creates a feedback dataset for evaluating the prompt's accuracy over time. When deploying changes to the prompt template, replay a sample of historical traces through the new version and compare the root_cause_hypotheses against the previous diagnoses to detect regressions. Avoid using this prompt for real-time agent intervention without human approval gates; the diagnosis is a forensic tool, not a live decision-maker. For traces exceeding the model's context window, preprocess the tool-call sequence to include only the calls immediately surrounding the failure point, and note the truncation in the [CONTEXT] field so the model can account for missing information in its confidence scores.