Tool Response Latency Anomaly Detection Prompt

This prompt is designed for Site Reliability Engineers (SREs) and platform operators who need a consistent, evidence-based first pass at triaging a latency spike in a production tool. The core job-to-be-done is to augment an on-call workflow, not replace your observability stack. When a monitoring dashboard flags that a tool's p95 or p99 latency has crossed a threshold, this prompt takes a structured current latency distribution, a historical baseline, and any known seasonal patterns to produce a standardized anomaly assessment. The output includes a severity classification (e.g., 'Critical', 'Warning', 'Nominal') and a recommended investigation path, allowing the responder to quickly decide whether to escalate, continue observing, or dismiss the alert as a known pattern.

Use this prompt when you have already extracted the necessary metrics from your observability platform (such as Datadog, Prometheus, or Grafana) and can format them as structured input. The prompt assumes you can provide a current distribution with key percentiles (p50, p75, p90, p95, p99), a matching historical baseline distribution for the same window, and a description of known seasonal patterns like daily peak traffic or a weekend lull. Do not use this prompt if you lack a reliable historical baseline, as the model will be forced to guess at normal behavior. It is also inappropriate for real-time, sub-second decision-making or as a replacement for a statistical anomaly detection system. Its value lies in producing a human-readable, reasoned summary that standardizes the triage step across your on-call team, reducing cognitive load during an incident.

Before wiring this into an automated pipeline, validate that your input distributions are complete and correctly aligned in time. A common failure mode is providing a baseline from a different time window (e.g., a Tuesday baseline for a Saturday spike), which will produce a false severity classification. After using this prompt, the next step is to follow the recommended investigation path, which may involve checking for recent deployments, infrastructure changes, or upstream dependency saturation. Avoid the temptation to treat the AI's severity classification as a definitive answer; it is a triage aid that must be confirmed with direct system interrogation.

This prompt template is designed to be pasted into your agent's system or task prompt. It instructs the model to act as an SRE analyzing a stream of tool-call latency data. The model compares a provided current latency distribution against a historical baseline, classifies the severity of any deviation, and recommends a concrete investigation path. The prompt is structured to prevent the model from hallucinating metrics or jumping to conclusions without sufficient evidence.

textCopy
You are an SRE analyzing tool response latency for anomalies. Your task is to compare the current latency distribution against the provided historical baseline and produce a structured assessment.

## INPUT DATA
[CURRENT_LATENCY_DATA]
[BASELINE_LATENCY_DATA]

## CONTEXT
[TOOL_NAME]: The specific tool or endpoint being monitored.
[TIME_WINDOW]: The period for the current data (e.g., 'last 5 minutes').
[BASELINE_WINDOW]: The period for the baseline data (e.g., 'same 5-minute window over the last 14 days').
[SEASONALITY_NOTES]: Any known seasonal patterns, such as daily peaks or deployment windows.

## CONSTRAINTS
- Do not invent data points. If the provided data is insufficient for a high-confidence assessment, state that explicitly.
- Distinguish between a genuine latency spike and a shift in workload volume.
- Consider the [SEASONALITY_NOTES] before classifying an anomaly.
- If the anomaly is classified as 'critical', the investigation path must include an immediate check for cascading failures in dependent services.

## OUTPUT_SCHEMA
Return a single JSON object with the following keys:
- `anomaly_detected` (boolean): Whether a statistically significant deviation exists.
- `severity` (string): One of 'none', 'minor', 'major', 'critical'.
- `p95_increase_percent` (number or null): The percentage increase in P95 latency compared to the baseline.
- `key_evidence` (string): A concise summary of the data points that support the assessment.
- `likely_causes` (array of strings): A ranked list of 1-3 probable causes (e.g., 'resource saturation', 'network blip', 'upstream dependency slowdown').
- `investigation_path` (array of strings): An ordered list of 2-5 concrete next steps for an on-call engineer.
- `confidence` (string): One of 'low', 'medium', 'high', reflecting the certainty of the assessment given the data.

To adapt this template, replace the square-bracket placeholders with data from your monitoring stack. The [CURRENT_LATENCY_DATA] and [BASELINE_LATENCY_DATA] fields should be populated with pre-formatted statistical summaries (e.g., P50, P95, P99, mean, and sample count) rather than raw log streams. This prevents the model from performing error-prone arithmetic and keeps the context window focused on interpretation. Before deploying, run this prompt against a golden dataset of known anomalies and normal periods to calibrate the severity thresholds in your evaluation harness.

This prompt is designed to be called by an agent or a scheduled job when a monitoring alert fires for a specific tool endpoint. The calling system is responsible for gathering the required inputs before invoking the LLM. Do not pass raw alert payloads directly to the prompt; the agent must first query your time-series database (Prometheus, Datadog, Grafana, or equivalent) to fetch the [CURRENT_LATENCY_DISTRIBUTION] and [HISTORICAL_BASELINE] for the affected tool and time window. The prompt expects structured distribution data—percentiles, sample counts, and timestamps—not raw metric streams or dashboard screenshots.

Implement a pre-invocation guardrail: if the sample count in [CURRENT_LATENCY_DISTRIBUTION] is below 100, the agent should abort the LLM call entirely and instead log a LOW_SAMPLE_COUNT event with a request for more data. Low-sample assessments produce unreliable severity classifications and waste on-call attention. After the prompt returns, parse the structured output and route it based on the severity field. For critical or major classifications, the agent should automatically page the on-call engineer via your incident management tool (PagerDuty, Opsgenie) and attach the full assessment. For minor or cosmetic classifications, post the output as a non-urgent notification to your team's incident channel (Slack, Teams). Always include the [OBSERVABILITY_LINK] in every notification so the responder can jump directly to the source dashboard and verify the analysis independently.

Model choice matters here. Use a model with strong numerical reasoning and structured output support (GPT-4o, Claude 3.5 Sonnet, or equivalent). Set temperature=0 to maximize consistency across repeated assessments. Implement a retry with backoff only for transient API failures; do not retry on validation errors. Validate the output against the expected schema before routing—reject any response where severity is not one of the defined enum values or where p95_deviation_percent is missing. For high-severity production systems, consider running the same prompt against a second model as a consistency check before paging, but weigh the added latency against your incident response SLA. Log every invocation, including the input distributions, the model's raw output, and the routing decision, to enable post-incident review and prompt drift detection over time.