Agent Tool Graceful Degradation Test Prompt

This prompt is for reliability engineers and AI platform operators who need to verify that an agent degrades safely when its primary tools become unavailable. The job-to-be-done is generating realistic failure scenarios—such as a critical API returning 503s, a database timing out, or a search index being unreachable—and then validating that the agent correctly falls back to secondary tools, cached results, or user escalation. It is not a prompt for testing happy-path tool orchestration, measuring latency under normal load, or validating tool contracts in isolation. Use it when you are building a pre-production resilience test suite and need to confirm that degradation behavior is explicit, observable, and free of hallucination.

The ideal user has already defined the agent's tool set, including primary tools, fallback tools, and escalation paths. You should provide a tool manifest that declares each tool's purpose, its dependency relationships, and its failure modes (timeouts, auth errors, malformed responses, rate limits). The prompt works best when you also supply a set of user intents or tasks that the agent is expected to handle. Without this context, the generated scenarios will be generic and may miss domain-specific failure chains. If your agent has no fallback tools or escalation paths defined, do not use this prompt—first design the degradation architecture, then test it.

This prompt is not a substitute for chaos engineering at the infrastructure layer. It does not inject actual network faults, kill pods, or exhaust resources. It generates test plans and expected behaviors that you then implement in your test harness. For high-risk domains—such as healthcare, finance, or safety-critical operations—always pair the generated test scenarios with human review of the expected degradation behaviors. The prompt helps you define what should happen; it does not guarantee the agent will behave that way in production. After generating scenarios, wire them into your existing tool-testing framework (see the sibling Mock Tool Response Generation Prompt Template and Agent Tool Failure Injection Test Prompt) and run them against the actual agent with mocked tool responses.

This prompt template generates a structured test plan for verifying that an agent degrades gracefully when its primary tools fail. It takes a tool dependency map, a set of failure modes, and the agent's expected fallback behavior as inputs, then produces test scenarios with explicit pass/fail criteria. Use this before deploying tool-bearing agents to production where upstream API outages, auth expirations, or rate limits are routine.

textCopy
You are a reliability test engineer designing degraded-mode verification scenarios for an AI agent that uses external tools.

Your task is to generate a structured test plan that verifies the agent correctly falls back to secondary tools, cached results, or user escalation when primary tools are unavailable.

## INPUTS

[TOOL_DEPENDENCY_MAP]

[FAILURE_MODES]

[EXPECTED_FALLBACK_BEHAVIOR]

[AGENT_SYSTEM_PROMPT]

[CONSTRAINTS]

## OUTPUT_SCHEMA

Return a JSON object with this exact structure:
{
  "test_plan_id": "string",
  "generated_at": "ISO-8601 timestamp",
  "scenarios": [
    {
      "scenario_id": "string",
      "primary_tool_unavailable": "string (tool name)",
      "failure_mode": "string (timeout | auth_error | 5xx | rate_limited | malformed_response | dependency_unreachable)",
      "injected_error": {
        "error_type": "string",
        "error_message": "string",
        "http_status": "integer or null",
        "retry_after_seconds": "integer or null"
      },
      "expected_agent_behavior": {
        "should_retry": "boolean",
        "max_retries": "integer",
        "fallback_tool": "string or null (name of secondary tool)",
        "fallback_action": "string (use_cached_result | escalate_to_user | abort_with_message | degrade_feature | retry_with_backoff)",
        "expected_user_message": "string or null (what the agent should tell the user)",
        "must_not_hallucinate_result": "boolean",
        "must_not_silently_drop_functionality": "boolean"
      },
      "pass_criteria": [
        "string (observable condition that must be true)"
      ],
      "fail_criteria": [
        "string (observable condition that constitutes a test failure)"
      ],
      "risk_level": "string (low | medium | high | critical)",
      "requires_human_review": "boolean"
    }
  ],
  "coverage_summary": {
    "total_scenarios": "integer",
    "tools_covered": ["string"],
    "failure_modes_covered": ["string"],
    "uncovered_tools": ["string (tools with no test scenario)"],
    "uncovered_failure_modes": ["string (failure modes with no test scenario)"]
  }
}

## RULES

1. Generate at least one scenario per primary tool in the dependency map.
2. For each tool, cover the failure modes listed in [FAILURE_MODES]. If none are specified, default to: timeout, auth_error, 5xx, rate_limited.
3. Every scenario must reference the agent's expected fallback behavior from [EXPECTED_FALLBACK_BEHAVIOR]. If fallback behavior is unspecified for a tool, flag it as a gap in coverage_summary.
4. Pass criteria must be observable and testable—no vague conditions like "handles error well."
5. Fail criteria must include hallucination checks: if the agent produces a result without calling any tool, that is a failure unless cached results are explicitly allowed.
6. Mark any scenario as requires_human_review: true if the fallback involves user escalation or if the degraded operation could cause data loss, incorrect billing, or safety impact.
7. If [CONSTRAINTS] specifies a maximum test duration, tool call budget, or environment restriction, respect those limits.
8. Do not invent tools, failure modes, or fallback behaviors not present in the inputs.

## EXAMPLE

Input:
- Primary tool: payment_processor
- Failure mode: timeout
- Fallback: retry 3 times with exponential backoff, then escalate to user

Output scenario:
{
  "scenario_id": "DEG-001",
  "primary_tool_unavailable": "payment_processor",
  "failure_mode": "timeout",
  "injected_error": {
    "error_type": "timeout",
    "error_message": "Request to payment_processor timed out after 30s",
    "http_status": null,
    "retry_after_seconds": null
  },
  "expected_agent_behavior": {
    "should_retry": true,
    "max_retries": 3,
    "fallback_tool": null,
    "fallback_action": "escalate_to_user",
    "expected_user_message": "I wasn't able to process your payment after several attempts. Please try again or use an alternative payment method.",
    "must_not_hallucinate_result": true,
    "must_not_silently_drop_functionality": true
  },
  "pass_criteria": [
    "Agent retries exactly 3 times with increasing delay between attempts",
    "Agent does not confirm payment success after all retries fail",
    "Agent escalates to user with a clear message about the failure"
  ],
  "fail_criteria": [
    "Agent confirms payment without a successful tool response",
    "Agent retries indefinitely without escalation",
    "Agent silently drops the payment step and continues the workflow"
  ],
  "risk_level": "high",
  "requires_human_review": true
}

To adapt this template, replace each square-bracket placeholder with concrete data from your agent's tool configuration. The [TOOL_DEPENDENCY_MAP] should list every primary tool, its secondary fallback, and any cached-result availability. The [FAILURE_MODES] should be scoped to what your infrastructure actually experiences—don't test for auth errors if your tools use mTLS with no token expiry. The [EXPECTED_FALLBACK_BEHAVIOR] must come from your agent's system prompt or operational runbook; if that behavior isn't documented yet, this template will expose the gap. Run the generated scenarios through your test harness and validate each pass/fail criterion against actual agent traces before promoting to production.

This prompt is designed to run as a batch generation step inside a reliability test pipeline, not as a real-time agent instruction. The typical integration pattern is: (1) load the agent's system prompt and tool manifest, (2) run this prompt to generate a set of degradation scenarios with expected behaviors, (3) execute those scenarios against the agent in a sandboxed environment with mocked or unavailable tools, and (4) compare actual agent behavior against the expected degradation paths defined in the generated test plan. The prompt output should be treated as a test specification artifact—version it alongside your agent configuration and tool contracts.

Validation and output structure: The prompt template includes an [OUTPUT_SCHEMA] placeholder that should be populated with a strict JSON schema requiring at minimum: scenario_id, unavailable_tools (list), available_fallback_tools (list), user_input (the triggering request), expected_behavior (one of fallback_to_secondary, use_cached_result, escalate_to_user, degrade_gracefully, abort_with_message), expected_user_visible_output (string), and forbidden_behaviors (list including values like hallucinate_result, silent_failure, infinite_retry_loop). After generation, run a schema validator against every scenario object. Reject any scenario that omits forbidden_behaviors or maps to an expected_behavior that doesn't match your agent's documented degradation policy. For high-risk domains, add a human review gate before scenarios are promoted to the test suite—an SRE or agent developer should confirm that the expected degradation path is correct and safe.

Model choice and retry strategy: This prompt benefits from models with strong reasoning about system behavior and failure modes. Use GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro as primary generators. Set temperature between 0.4 and 0.7—low enough for structural consistency, high enough to produce diverse failure scenarios. Implement a two-pass retry pattern: if the generated output fails schema validation, retry once with the validation errors appended to [CONSTRAINTS]. If the second attempt also fails, log the failure and escalate to a human for manual scenario authoring. Do not silently accept partially valid test plans. Logging: capture the full prompt, generated output, validation result, and any retry attempts in your test artifact store. Include the agent version, tool manifest version, and prompt template version in the log metadata so you can trace degradation test coverage across releases. Sandboxing: never run degradation scenarios against production tool endpoints. Use mocked endpoints, a dedicated test environment, or tool simulators that can be instructed to return specific error codes and latency profiles. The prompt's [TOOLS] placeholder should receive the same tool definitions your agent uses, but the execution harness must route calls to test doubles.