Inferensys

Prompt

Agent Step Failure Classification Prompt Template

A practical prompt playbook for using Agent Step Failure Classification Prompt Template in production AI workflows.
Engineer reviewing agent handoff workflow on laptop, task routing diagrams visible, technical office setup.
PROMPT PLAYBOOK

When to Use This Prompt

Define the job, reader, and constraints for classifying agent step failures into actionable recovery categories.

This prompt is for agent runtime developers and platform engineers who need to turn raw step failures into structured, machine-actionable decisions. The core job is classification: given a failed step's context, error message, and execution history, the prompt must output a failure record that downstream recovery logic can consume without ambiguity. The ideal user is someone wiring this into an agent harness where retry policies, circuit breakers, and escalation paths depend on consistent failure categorization. Required context includes the step's intended purpose, the raw error output, the retry count so far, and any tool or API contract that defines normal behavior.

Do not use this prompt when you need a decision rather than a classification. If your system already knows the failure type and needs to choose between retry, skip, or abort, use the sibling Retry Decision Prompt for Failed Tool Calls or Skip vs Abort Decision Prompt for Agent Workflows instead. This prompt is also inappropriate for user-facing error messages—it produces structured records for machine consumption, not human-readable explanations. Avoid using it for failures where the root cause is already known and deterministic; classification adds latency and token cost without benefit when the error type is unambiguous from the status code alone.

Before integrating this prompt into your agent loop, define your failure taxonomy and wire the output into a recovery dispatch layer. The classification record should feed directly into retry logic, escalation queues, or monitoring dashboards. Start by running this prompt against a labeled dataset of known failure modes and measure inter-rater consistency across multiple runs—classification drift under temperature sampling is a common failure mode. If your agent operates in a regulated domain or on mutable resources, require human review for any failure classified as permanent with severity: critical before automated rollback or state mutation occurs.

PRACTICAL GUARDRAILS

Use Case Fit

Where the Agent Step Failure Classification Prompt Template delivers value and where it introduces risk. Use these cards to decide if this prompt belongs in your agent runtime before wiring it into production error-handling loops.

01

Good Fit: Structured Error Routing

Use when: your agent orchestrator needs a consistent, machine-readable failure record to decide retry, skip, or escalate actions. Guardrail: validate the output schema before the classification result touches any routing logic—never branch on raw model text.

02

Bad Fit: Real-Time Hard Latency Budgets

Avoid when: your step timeout is under 500ms and adding an LLM call for classification would violate the SLA. Guardrail: use a deterministic regex or error-code mapper for tight latency paths; reserve this prompt for async diagnosis or post-step analysis.

03

Required Inputs: Structured Error Context

What to watch: the prompt needs the raw error message, the tool or step name, the retry count so far, and the step's criticality flag. Missing any of these causes the model to guess. Guardrail: gate execution on a required-fields check; if fields are absent, default to a safe classification (permanent, do not retry) and log the gap.

04

Operational Risk: Classification Drift on Novel Errors

What to watch: new error types from API updates or infrastructure changes can be misclassified as transient when they are actually permanent, causing retry storms. Guardrail: maintain a fallback rule that caps retries per step regardless of classification, and monitor for spikes in the 'transient' category after deployments.

05

Operational Risk: Severity Inflation

What to watch: the model may assign high severity to low-impact failures when the error message contains alarming language, triggering unnecessary escalations. Guardrail: pair this prompt with a severity calibration eval that compares model-assigned severity against a human-labeled baseline, and enforce a maximum escalation rate threshold.

06

Bad Fit: Unsupervised Irreversible Actions

Avoid when: the failed step performed a destructive or irreversible action and classification alone is insufficient for recovery. Guardrail: for steps tagged as irreversible, bypass classification and route directly to human review with full context, regardless of what the prompt would return.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A reusable prompt template with square-bracket placeholders for classifying agent step failures into actionable categories with retry eligibility and recovery actions.

This prompt template is designed to be copied directly into your agent runtime's error-handling module. It takes raw failure context from a failed agent step and produces a structured classification record. The template uses square-bracket placeholders that you must replace with live data from your execution environment: the original step description, the error details, the current retry count, and your operational constraints. The output is a machine-readable JSON object that downstream retry logic, escalation routers, and monitoring dashboards can consume without additional parsing.

text
You are an agent step failure classifier. Your job is to analyze a failed agent step and produce a structured classification record that downstream systems can use to decide on recovery actions.

## INPUT
[STEP_DESCRIPTION]
[ERROR_DETAILS]
[RETRY_COUNT]
[TOKEN_BUDGET_REMAINING]
[STEP_CRITICALITY]

## CONSTRAINTS
[CONSTRAINTS]

## OUTPUT_SCHEMA
Return a single JSON object with these exact fields:
- failure_category: string, one of "transient", "permanent", "permission", "timeout", "malformed_output"
- retry_eligible: boolean
- severity: string, one of "critical", "high", "medium", "low"
- recommended_action: string, one of "retry", "retry_with_backoff", "skip", "abort", "escalate", "fallback_tool"
- confidence: float between 0.0 and 1.0
- justification: string, brief explanation of the classification
- blast_radius: string, one of "isolated", "downstream_steps", "full_plan"
- retry_strategy: string or null, if retry_eligible, specify "immediate", "exponential_backoff", or "fixed_delay"
- escalation_reason: string or null, required if recommended_action is "escalate"

## CLASSIFICATION RULES
- Transient: errors likely to resolve on retry (network timeouts, rate limits, temporary unavailability)
- Permanent: errors that will not resolve on retry (invalid arguments, unsupported operations, data corruption)
- Permission: authorization, access denied, or scope errors
- Timeout: step exceeded its time budget but may have partial results
- Malformed Output: step completed but produced unparseable or schema-invalid output

## RETRY ELIGIBILITY
- Retry only if failure_category is "transient" or "timeout" AND retry_count is below the limit specified in constraints
- Never retry "permission" errors without human intervention
- "malformed_output" may be retried once with stricter output format instructions

## SEVERITY ASSESSMENT
- Critical: step is required for plan completion and failure blocks all downstream work
- High: step failure degrades output quality significantly or causes cascading failures
- Medium: step failure has moderate impact, workaround possible
- Low: step is optional or has minimal impact on overall plan success

## INSTRUCTIONS
1. Analyze the error details against the step description
2. Consider the retry count and remaining token budget
3. Factor in step criticality when assigning severity
4. If confidence is below 0.7, lean toward escalation
5. Output only the JSON object, no markdown fences, no additional text

To adapt this template for your environment, replace each placeholder with live data from your agent runtime. [STEP_DESCRIPTION] should include the original objective and expected output schema for the failed step. [ERROR_DETAILS] must capture the raw error message, stack trace, HTTP status code, or tool response that triggered the failure. [RETRY_COUNT] tracks how many times this specific step has already been retried. [TOKEN_BUDGET_REMAINING] helps the classifier decide whether retrying is affordable. [STEP_CRITICALITY] should be a label from your plan's dependency graph indicating whether downstream steps depend on this step's output. The [CONSTRAINTS] placeholder should contain operational limits like maximum retries, latency budgets, and escalation thresholds. Before deploying, validate that your harness populates all placeholders completely—missing context produces unreliable classifications. For high-risk workflows, add a human review gate when severity is "critical" or confidence is below 0.7, and log every classification decision alongside the raw failure data for postmortem analysis.

IMPLEMENTATION TABLE

Prompt Variables

Replace each placeholder with concrete values before sending the prompt. Validation notes describe how to check that the input is well-formed and safe before it reaches the model.

PlaceholderPurposeExampleValidation Notes

[STEP_DESCRIPTION]

Natural-language description of the agent step that failed, including its intended outcome and role in the larger plan.

Call the billing API to charge the customer's default payment method for invoice INV-9821.

Must be non-empty string. Check length > 10 chars. Reject if it contains only error codes without human-readable context.

[ERROR_PAYLOAD]

Raw error object, exception message, stack trace, or tool response body returned by the failed step.

{"error": "timeout", "status": 504, "retry_after_ms": 2000}

Must be valid JSON string or plain text. If JSON, validate it parses without error. If empty or null, classification must fall back to UNKNOWN with low confidence.

[AGENT_STATE_SUMMARY]

Brief structured summary of what the agent has completed, what is in progress, and what remains in the current plan.

Completed: user auth, invoice lookup. In progress: payment capture. Remaining: receipt email, audit log write.

Must be non-empty string. Should contain at least one completed or in-progress item. If null, classification must note reduced confidence due to missing context.

[TOOL_CAPABILITY_REGISTRY]

List of available tools with their names, descriptions, parameter schemas, and known failure modes.

[{"name": "billing_api", "retryable_errors": ["timeout", "rate_limit"], "permanent_errors": ["invalid_card", "insufficient_funds"]}]

Must be valid JSON array. Each entry requires name and at least one of retryable_errors or permanent_errors. If empty, transient vs permanent classification will be less reliable.

[RETRY_BUDGET_REMAINING]

Number of retries already consumed and the maximum allowed for this step or plan.

{"retries_used": 2, "retries_allowed": 3, "token_budget_remaining_pct": 45}

Must be valid JSON with integer retries_used and retries_allowed fields. If retries_used >= retries_allowed, classification must never return RETRY. Validate budget exhaustion before prompt call.

[SEVERITY_THRESHOLDS]

Organization-specific definitions for critical, high, medium, and low severity with escalation rules.

{"critical": "data loss or billing error > $1000", "high": "user-visible failure blocking core flow", "medium": "degraded experience", "low": "cosmetic or retryable"}

Must be valid JSON with at least critical and high thresholds defined. If missing, use default thresholds and flag in output metadata. Thresholds should reference measurable impacts, not vague adjectives.

[OUTPUT_SCHEMA]

Expected JSON schema for the classification output, including required fields and enum values.

{"failure_category": "TRANSIENT|PERMANENT|PERMISSION|TIMEOUT|MALFORMED_OUTPUT", "retry_eligible": true, "severity": "high"}

Must be valid JSON Schema or example object. Validate that the model output conforms to this schema before accepting. Reject outputs missing required fields or using undefined enum values.

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the Agent Step Failure Classification prompt into a production agent runtime with validation, retry logic, and observability hooks.

Integrating the failure classification prompt into an agent runtime requires a thin harness that captures the failing step's context, invokes the model, validates the structured output, and routes the result to the appropriate recovery handler. The harness should be called immediately after a step failure is detected—before any retry, skip, or escalation logic executes. This ensures that every recovery path operates on a consistently classified failure record rather than raw error strings that vary across tools, APIs, and model providers. The harness is not a replacement for exception handling in your application code; it is a classification layer that translates heterogeneous failures into a uniform contract that downstream recovery prompts and decision nodes can consume reliably.

The harness must enforce a strict output schema. After calling the model with the classification prompt, validate that the response contains all required fields: failure_category (one of transient, permanent, permission, timeout, malformed_output), retry_eligible (boolean), severity (one of critical, high, medium, low), recommended_action (one of retry, skip, escalate, abort), and a justification string. If parsing fails or required fields are missing, implement a retry loop with a maximum of two additional attempts, appending the parse error to the prompt context so the model can self-correct. After three total failures, default to permanent / retry_eligible: false / severity: high / recommended_action: escalate and log the raw response for later debugging. For high-risk domains, route the escalation decision to a human review queue before any automated abort or skip action is taken.

Wire the harness into your agent's error-handling middleware so that every tool call, model inference, and data retrieval step is wrapped. Log the classification output alongside the step trace for observability. Use these structured records to build dashboards that track failure category distributions, retry eligibility rates, and severity trends over time. This data is essential for tuning retry budgets, identifying brittle integrations, and deciding when to invest in fallback tooling. Avoid the temptation to skip classification for 'obvious' errors—ad-hoc error handling creates drift between what your recovery prompts expect and what your runtime actually delivers, leading to silent stalls and misrouted escalations in production.

IMPLEMENTATION TABLE

Expected Output Contract

Defines the structured failure record that the prompt must produce. Use this contract to validate model output before passing it to retry logic, escalation queues, or monitoring dashboards.

Field or ElementType or FormatRequiredValidation Rule

failure_id

string (UUID v4)

Must parse as valid UUID v4; reject if missing or malformed

step_id

string

Must match the [STEP_ID] input exactly; reject on mismatch

classification

enum: transient | permanent | permission | timeout | malformed_output

Must be exactly one of the five allowed values; reject any other string

retry_eligible

boolean

Must be true for transient and timeout; false for permanent, permission, and malformed_output; reject contradictions

severity

enum: critical | high | medium | low

Must be exactly one of the four allowed values; reject if missing or invalid

recovery_action

string

Must be a non-empty string; reject if null, empty, or whitespace-only

evidence_summary

string

Must contain at least one direct quote or reference from [ERROR_MESSAGE] or [STEP_OUTPUT]; reject if purely speculative

confidence

number (0.0 - 1.0)

Must be a float between 0.0 and 1.0 inclusive; reject if out of range or non-numeric

PRACTICAL GUARDRAILS

Common Failure Modes

Agent step failure classification breaks when the model misjudges error types, misses transient patterns, or recommends unsafe recovery actions. These are the most common failure modes and how to guard against them before they reach production.

01

Transient vs. Permanent Misclassification

What to watch: The model classifies a permanent error (invalid credentials, missing resource) as transient, triggering retry loops that waste tokens and delay failure. Conversely, treating a brief rate limit as permanent causes unnecessary workflow abortion. Guardrail: Include explicit definitions and contrasting examples for each error class in the prompt. Validate classification against a golden set of known transient and permanent failures before deployment.

02

Permission Errors Treated as Data Errors

What to watch: The model misclassifies a 403 or access-denied response as a malformed output or timeout, recommending retry instead of escalation. This causes repeated forbidden requests that may trigger security alerts or account lockouts. Guardrail: Add a dedicated permission-error detection path with explicit escalation instructions. Test with simulated 401/403 responses and verify the classifier never recommends retry for auth failures.

03

Timeout Severity Underestimation

What to watch: The model classifies timeouts as low-severity transient errors without considering cumulative latency budget impact. Multiple sequential timeouts can exhaust SLA windows even though each individual failure appears minor. Guardrail: Include remaining latency budget and step deadline as required inputs. Add a severity escalation rule when timeout count exceeds a threshold within a single workflow execution.

04

Malformed Output Confused with Valid Empty Results

What to watch: The model cannot distinguish between a tool returning a valid empty result (no data found) and a genuinely malformed response (unparseable JSON, truncated output). Misclassification leads to retrying valid empty queries or accepting garbage as legitimate output. Guardrail: Require output schema validation before classification. Add explicit examples of valid-empty vs. malformed responses. Include a schema-check gate in the classification harness.

05

Recovery Action Mismatch with Error Type

What to watch: The model correctly identifies the error class but recommends an incompatible recovery action—suggesting retry for a permanent error or escalation for a trivial transient glitch. The classification is right but the downstream behavior is still wrong. Guardrail: Use a constrained output schema that maps each error class to its allowed recovery actions. Validate that the recommended action exists in the permitted set for the classified error type before executing recovery.

06

Context Window Truncation Hiding Error Details

What to watch: When the agent's context window is near capacity, error messages and stack traces get truncated before reaching the classifier. The model classifies based on partial information, often defaulting to a generic transient classification. Guardrail: Place error classification early in the context assembly order. Implement a pre-check that error payloads are complete before classification. If truncation is detected, escalate with a context-overflow flag rather than guessing.

IMPLEMENTATION TABLE

Evaluation Rubric

Use this rubric to test the Agent Step Failure Classification Prompt before deployment. Each criterion targets a known production failure mode. Run these tests against a golden dataset of 50+ labeled failure cases covering transient, permanent, permission, timeout, and malformed output categories.

CriterionPass StandardFailure SignalTest Method

Classification Accuracy

= 95% exact match on golden dataset labels across all 5 failure categories

Misclassifies transient errors as permanent (causes unnecessary aborts) or permanent errors as transient (causes infinite retry loops)

Run prompt against labeled golden dataset; compute precision/recall per category; flag any category below 90% for review

Retry Eligibility Correctness

100% correct retry eligibility flag on golden dataset where ground truth is known

Flags a permanent error as retry-eligible (wastes tokens and latency budget) or flags a transient error as not retry-eligible (causes premature escalation)

Binary check per test case: does [RETRY_ELIGIBLE] match ground truth? Any mismatch is a blocking failure

Severity Scoring Consistency

Inter-rater agreement >= 0.85 Cohen's kappa across 3 repeated runs on same inputs

Same failure receives critical severity in one run and low severity in another (causes inconsistent escalation behavior in production)

Run prompt 3x on identical inputs with temperature=0; measure kappa across severity field; flag if variance exceeds threshold

Output Schema Compliance

100% of outputs parse successfully against expected JSON schema with all required fields present

Missing [RECOVERY_ACTION] field, malformed JSON, or extra fields that break downstream parsers

Validate every output against schema definition; count parse failures; any failure is a blocking issue before deployment

Timeout Classification Precision

Correctly identifies >= 95% of timeout failures vs other transient errors in timeout-specific test subset

Classifies timeout as generic transient error instead of timeout-specific category (prevents proper backoff or scope reduction)

Isolate timeout test cases from golden dataset; measure precision of [FAILURE_CATEGORY] = timeout; flag if below threshold

Malformed Output Detection

Correctly identifies >= 95% of malformed output failures vs tool execution errors

Classifies malformed output as tool error (triggers wrong recovery path) or classifies valid output as malformed (causes false retries)

Test against curated malformed-output subset; check [FAILURE_CATEGORY] and [RECOVERY_ACTION] fields; measure false positive and false negative rates

Edge Case: Null Input Handling

Prompt returns structured output with category=unknown and severity=high when [ERROR_CONTEXT] is null or empty

Prompt crashes, returns unstructured text, or hallucinates a failure category from no evidence

Send null and empty string as [ERROR_CONTEXT]; verify output schema compliance and conservative classification behavior

Latency Budget Compliance

95th percentile response time < 2 seconds for single classification call

Prompt exceeds latency budget under load, causing cascading delays in agent recovery loop

Benchmark 100 classification calls; measure p95 latency; flag if exceeds threshold for production SLA

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Use the base prompt with a frontier model and a simple JSON schema. Focus on getting the classification logic right before adding infrastructure. Remove the [RETRY_BUDGET_REMAINING] and [LATENCY_BUDGET_MS] constraints initially. Replace [SEVERITY_SCALE] with a simple {low, medium, high} enum.

Watch for

  • The model classifying every timeout as transient without checking if the step is idempotent.
  • Overly verbose recovery_action fields that can't be parsed by a simple workflow engine.
  • Missing evidence fields when the error message is ambiguous.
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.