Tool Degradation Detection Prompt Template

This prompt is designed for platform reliability engineers and agent developers who need an automated, structured assessment of tool health before cascading failures impact users. Use it inside an agent health monitor, a pre-flight check before critical tool calls, or as part of a circuit breaker decision pipeline. The prompt ingests recent tool response patterns, latency metrics, and error signatures, then produces a degradation assessment with a severity classification, supporting evidence, and a recommended action. It assumes you already collect tool-level telemetry and can feed that data into the prompt context.

Do not use this prompt as a replacement for metric-based alerting; it is a reasoning layer that interprets signals a threshold-based system might miss or misclassify. The prompt works best when paired with a validation harness that checks output schema compliance, severity classification consistency, and evidence grounding. For high-risk production systems, always route the assessment through a human review step before triggering automated circuit breaker state changes or load shedding decisions. The prompt is not designed for real-time sub-second decisions—expect latency appropriate for LLM inference and budget accordingly.

Before deploying, calibrate the prompt against your historical incident data. Run it against known degradation events to verify it correctly classifies severity and recommends appropriate actions. Pay special attention to false-positive degradation signals during maintenance windows, deployment rollouts, and known brownout periods. The prompt's value comes from catching subtle degradation patterns—gradual latency creep, intermittent error rate increases, or partial failure modes—that static thresholds routinely miss. Wire the output into your existing observability stack, not as a standalone alerting channel.

This prompt template is designed to be invoked by an agent health monitor or a pre-flight check system before a critical tool execution window. Its job is to produce a structured degradation assessment from raw telemetry, not to make the final circuit-breaking decision. The output should be consumed by a downstream policy engine that compares the assessment against defined thresholds. Do not use this prompt for real-time per-call decisions where latency is critical; it is intended for periodic health checks or state transition evaluations.

textCopy
You are an agent tool reliability analyzer. Your task is to assess the health of a specific tool based on recent telemetry data and produce a structured degradation assessment.

## INPUT DATA
- Tool Name: [TOOL_NAME]
- Current Circuit State: [CIRCUIT_STATE]
- Observation Window: [WINDOW_DURATION]
- Total Calls in Window: [TOTAL_CALLS]
- Failure Count: [FAILURE_COUNT]
- Error Signature Summary: [ERROR_SIGNATURES]
- P50 Latency: [P50_LATENCY]
- P99 Latency: [P99_LATENCY]
- Baseline P50 Latency: [BASELINE_P50]
- Baseline P99 Latency: [BASELINE_P99]
- Retry Budget Remaining: [RETRY_BUDGET_PCT]
- Dependency Health Status: [DEPENDENCY_HEALTH]

## OUTPUT SCHEMA
Return a valid JSON object with the following fields:
- "degradation_level": "healthy" | "degraded" | "unhealthy" | "unknown"
- "primary_signal": string (the single strongest indicator for the assessment)
- "evidence_summary": string (2-3 sentence summary of the telemetry patterns observed)
- "false_positive_risk": "low" | "medium" | "high" (likelihood this is a transient blip, not real degradation)
- "false_negative_risk": "low" | "medium" | "high" (likelihood real degradation is being masked)
- "recommended_action": "none" | "probe" | "degrade" | "circuit_break" | "escalate"
- "action_rationale": string (1 sentence explaining the recommended action)
- "confidence": float between 0.0 and 1.0

## CONSTRAINTS
- Do not recommend circuit_break unless failure rate exceeds 50% AND errors are non-transient (5xx, timeout, connection refused).
- A latency increase alone without errors should produce degradation_level "degraded", not "unhealthy".
- If the observation window has fewer than 10 calls, set degradation_level to "unknown" and confidence below 0.5.
- If dependency health status indicates upstream failures, note this in evidence_summary but assess the tool itself.
- Never hallucinate metrics not present in the input data.
- If error signatures are empty but latency is elevated, primary_signal must reference the latency anomaly.

## EXAMPLES
### Example 1: Clear Failure
Input: Tool=payment-api, Failures=45/100, P99=1200ms (baseline 200ms), Errors=timeout+connection_refused
Output: {"degradation_level": "unhealthy", "primary_signal": "45% failure rate with connection-refused errors", ...}

### Example 2: Latency-Only Degradation
Input: Tool=search-index, Failures=0/500, P99=800ms (baseline 150ms), Errors=none
Output: {"degradation_level": "degraded", "primary_signal": "P99 latency 5.3x baseline with zero errors", ...}

### Example 3: Low Sample Size
Input: Tool=audit-log, Failures=0/3, P99=100ms (baseline 90ms), Errors=none
Output: {"degradation_level": "unknown", "primary_signal": "insufficient sample size (3 calls)", "confidence": 0.3, ...}

Adapt this template by adjusting the thresholds in the CONSTRAINTS section to match your specific SLOs. If your tool has idempotency guarantees, add a constraint that permits retries for idempotent-safe errors. For tools with strict data freshness requirements, add a field to the INPUT DATA for [MAX_STALENESS_TOLERANCE] and include it in the assessment logic. Always test the prompt against your historical incident data to calibrate the false-positive and false-negative risk assessments before deploying to production.

This prompt is designed to be called by an agent health monitor, not directly by an end user. The typical integration point is a pre-flight check that runs before a critical tool execution, or a periodic background health assessment that feeds a circuit breaker. The harness must collect the required inputs—recent tool response patterns, latency metrics, and error signatures—from your observability stack and format them into the [TOOL_OBSERVABILITY_DATA] placeholder. Do not pass raw, unbounded log streams; the prompt expects a structured summary with a defined lookback window (e.g., last 60 seconds or last N calls).

The output is a structured degradation assessment that your application code must parse and act on. Implement a strict JSON schema validator on the model's response before accepting it. The schema should enforce the degradation_level enum (none, minor, major, critical), require a confidence score between 0.0 and 1.0, and validate that the evidence array contains at least one concrete observation. If validation fails, retry once with the validation error message appended to the prompt as additional [CONSTRAINTS]. If the second attempt also fails, default to a safe state: treat the tool as degraded and trigger a human review alert. Log every assessment—including the raw prompt, model response, validation result, and final action taken—to your agent trace store for post-incident analysis.

Calibrate the [DEGRADATION_THRESHOLDS] placeholder with values derived from your tool's historical performance baselines, not arbitrary defaults. A latency threshold that is too tight will generate false-positive degradation signals and trigger unnecessary circuit breaks; a threshold that is too loose will miss real failures. Run this prompt against a golden dataset of known-good and known-degraded tool behavior patterns before deployment. Measure precision and recall on degradation detection, and adjust thresholds until false-positive and false-negative rates are acceptable for your risk tolerance. In high-risk domains where a missed degradation could cause data loss or user harm, always require human review for major and critical assessments before automated action is taken.