Call Sequencing Order Validation Prompt Template

Use this prompt when you need to programmatically verify that a multi-step agent trace executed its tool calls in the correct dependency order. The primary job-to-be-done is automated quality assurance for agent pipelines where step ordering is a hard constraint—for example, an agent must search a knowledge base before summarizing results, or must create a record before updating it. The ideal user is an AI engineer or evaluation lead building a regression suite or CI/CD gate for an agent that uses tools. You should have a trace of tool calls, a defined dependency graph or sequence specification, and a need for a structured, machine-readable correctness score rather than a human spot check.

This prompt is designed for post-execution evaluation, not real-time enforcement. It assumes you already have a complete trace of tool calls and a known correct sequence. It produces a sequence correctness score, identifies out-of-order calls, and flags missing prerequisites. Do not use this prompt for real-time agent guidance, for grading the semantic quality of tool outputs, or for evaluating whether the agent chose the right tools in the first place. For those concerns, use the Tool Selection Correctness Grading Prompt or the Tool Call Chaining Correctness Rubric Prompt instead. This prompt also assumes that the dependency graph is known and unambiguous; if the correct order is subjective or context-dependent, you will need a human-annotated ground truth and a calibration step before relying on automated scores.

Before deploying this prompt into a production evaluation pipeline, pair it with a validation harness that parses the JSON output and checks for schema compliance. For high-stakes agent workflows—such as those affecting financial transactions, healthcare operations, or infrastructure changes—always route low-confidence or flagged sequences to human review. Start by running this prompt against a golden dataset of known-correct and known-incorrect traces to calibrate your pass/fail threshold, then integrate it into your regression suite to catch ordering regressions when prompt or model versions change.

This template is designed to be copied directly into your evaluation harness. It accepts an agent trace containing a sequence of tool calls, a tool catalog defining each tool's purpose and input/output signatures, and a dependency specification that declares which calls must precede others. The prompt instructs the model to produce a structured sequence correctness score, identify out-of-order calls, and flag missing prerequisites. All placeholders use square brackets so you can substitute your own data without confusing the model's output format instructions.

textCopy
You are an evaluation judge for agent tool-call sequences. Your job is to validate that multi-step tool calls follow the required dependency order.

# INPUTS

## Agent Trace
[AGENT_TRACE]

## Tool Catalog
[TOOL_CATALOG]

## Dependency Rules
[DEPENDENCY_RULES]

# TASK
Analyze the agent trace against the dependency rules. For each rule, determine whether the required prerequisite calls were executed before the dependent calls. Identify any out-of-order calls, missing prerequisites, or circular dependencies.

# OUTPUT SCHEMA
Return a JSON object with this exact structure:
{
  "sequence_correctness_score": number, // 0.0 to 1.0, where 1.0 means all dependencies satisfied
  "total_rules_checked": number,
  "rules_passed": number,
  "rules_failed": number,
  "violations": [
    {
      "rule_id": string,
      "rule_description": string,
      "violation_type": "out_of_order" | "missing_prerequisite" | "circular_dependency",
      "expected_order": string,
      "actual_order": string,
      "offending_call_index": number,
      "severity": "critical" | "warning",
      "explanation": string
    }
  ],
  "missing_prerequisites": [
    {
      "required_call": string,
      "dependent_call": string,
      "dependent_call_index": number,
      "impact": string
    }
  ],
  "summary": string
}

# CONSTRAINTS
- Score 0.0 if any critical dependency is violated.
- Treat optional dependencies as warnings, not failures.
- If a prerequisite call failed in the trace, flag it as a missing prerequisite.
- Do not penalize parallel calls that satisfy dependencies.
- If the trace is empty, return score 1.0 with an explanation.
- Only flag violations that are explicitly defined in the dependency rules.

# EXAMPLES
[DEPENDENCY_RULE_EXAMPLES]

# RISK LEVEL
[RISK_LEVEL]

To adapt this template, replace each placeholder with your actual data. [AGENT_TRACE] should contain the full sequence of tool calls with timestamps or indices. [TOOL_CATALOG] should list each available tool with its name, description, and input/output types so the judge can verify call identity. [DEPENDENCY_RULES] is the critical input: define each rule with a unique ID, the prerequisite call name, the dependent call name, and whether the dependency is hard (critical) or soft (warning). [DEPENDENCY_RULE_EXAMPLES] should include 2-3 annotated examples showing correct and incorrect sequences so the judge calibrates its expectations. Set [RISK_LEVEL] to high if ordering violations cause downstream data corruption or safety issues, which triggers stricter scoring and requires human review of any score below 1.0. For production pipelines, always validate the output JSON against the schema before accepting the score, and log every violation for traceability.

This prompt is designed to be called programmatically after an agent trace is captured, not during live agent execution. The typical harness wraps the LLM call in a function that accepts a structured trace object and a dependency specification, then returns a validated sequence correctness report. Because the output is a JSON schema with numeric scores and categorical flags, the harness should validate the response against the expected schema before accepting it. If the model returns malformed JSON, missing required fields, or scores outside the defined range, the harness should retry with an explicit error message appended to the prompt context rather than silently falling back to a default score.

The implementation should enforce a strict contract: the input trace must be an ordered list of tool calls, each with a tool_name and timestamp field. The dependency specification must define prerequisite relationships as a list of {step, requires} pairs. Before calling the LLM, the harness should validate that every step and requires value references a tool name present in the trace. If the trace or dependency spec fails structural validation, the harness should return an error immediately rather than sending invalid input to the model. For high-stakes pipelines where sequencing errors could cause data corruption or unauthorized actions, add a human review gate when the sequence correctness score falls below a configurable threshold—for example, any score below 0.85 should route the trace to a review queue with the full violation details attached.

Model choice matters here. Use a model with strong structured output support and reasoning capabilities, such as GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro. Avoid smaller or older models that may struggle with multi-step dependency reasoning across long traces. Set temperature=0 to maximize deterministic scoring across evaluation runs. For batch evaluation of many traces, implement parallel calls with rate limiting and log every response—including raw model output, parsed scores, and any validation errors—to an observability store. This enables trace-level debugging when a specific sequence is scored inconsistently. If you're integrating this into a CI/CD pipeline for prompt releases, store the sequence correctness scores per trace and compare them against a baseline to detect regressions automatically. The next step is to wire the output into your evaluation dashboard, where per-trace violation details and aggregate sequence health metrics can trigger alerts when agent step-ordering reliability degrades.