Alert Storm Correlation Prompt

This prompt is designed for Site Reliability Engineers (SREs) and on-call responders who are facing a high-volume alert storm and need to rapidly identify the single most probable triggering event. The core job-to-be-done is to cut through the noise of cascading, symptomatic alerts to produce a structured correlation graph and a prioritized investigation path. It is intended for use during an active incident, where time is critical and the operator is overwhelmed by data from multiple monitoring systems like PagerDuty, Datadog, or Prometheus Alertmanager. The prompt assumes you can provide a raw, time-ordered dump of firing alerts as the primary input.

This prompt is not a replacement for your monitoring system's built-in correlation engine or a long-term anomaly detection model. It is a tactical tool for a human operator to use when automated grouping has failed or is insufficient. You should not use this prompt if you have fewer than five simultaneous alerts, as the overhead of structuring the input outweighs the benefit. It is also unsuitable for post-incident root cause analysis where deep forensic data like logs and deployment diffs are available; for that, use a dedicated Incident Timeline Extraction or Deployment Change Correlation prompt. The output is a hypothesis, not a confirmed root cause, and must be validated against live system metrics before taking remediation actions.

To use this effectively, you must supply a list of alerts with at least a timestamp, a service name, and an alert description. The prompt works best when you include alert severity labels and any known dependency relationships between services. Before executing, ensure you have a clear understanding of your alert volume; if the storm contains hundreds of alerts, you may need to pre-filter for high-severity items to stay within context window limits. The next step after receiving the output is to validate the 'probable root alert' by checking its underlying metric and then silencing the identified symptomatic alerts to reduce noise for the rest of the response team.

This prompt template is designed to be dropped directly into your incident response tooling or AI orchestration layer. It accepts raw alert payloads, monitoring tool outputs, and service topology context, then produces a structured correlation graph and a ranked investigation plan. The template uses square-bracket placeholders that your application must populate before sending the request to the model. Do not send this prompt with unresolved placeholders—your harness should validate that every required field is present and that alert timestamps fall within a coherent time window before invoking the model.

textCopy
You are an SRE correlation analyst. Your task is to analyze a set of simultaneous alerts and produce a correlation graph with a prioritized investigation path.

## INPUT
Alert payloads (JSON array of alert objects):
[ALERT_PAYLOADS]

Service dependency graph (JSON adjacency list):
[SERVICE_DEPENDENCY_GRAPH]

Time window for correlation (ISO 8601 start and end):
[TIME_WINDOW_START] to [TIME_WINDOW_END]

## CONSTRAINTS
- Group alerts by shared infrastructure, service dependencies, and temporal proximity.
- Identify the probable root alert that triggered downstream symptomatic alerts.
- Suppress alerts that are direct consequences of the root alert.
- Flag any alerts that do not fit the correlation pattern as independent incidents.
- If the evidence is insufficient to identify a single root, state that explicitly and propose the top N candidates.
- Do not invent service relationships not present in the provided dependency graph.

## OUTPUT SCHEMA
Return a JSON object with the following structure:
{
  "correlation_graph": {
    "nodes": [
      {
        "alert_id": "string",
        "service": "string",
        "type": "root|symptomatic|independent",
        "summary": "string"
      }
    ],
    "edges": [
      {
        "from": "alert_id",
        "to": "alert_id",
        "relationship": "causes|correlates_with"
      }
    ]
  },
  "investigation_path": [
    {
      "priority": 1,
      "alert_id": "string",
      "action": "string",
      "rationale": "string"
    }
  ],
  "suppressed_alerts": ["alert_id"],
  "independent_incidents": ["alert_id"],
  "confidence": "high|medium|low",
  "uncertainty_notes": "string"
}

## EXAMPLES
Example input: Three alerts—database connection timeout, API 503 errors, and a deployment event.
Example output: Root is database timeout, API errors are symptomatic, deployment is independent but temporally coincident.

## RISK LEVEL
[RISK_LEVEL]

If RISK_LEVEL is "high", include explicit human review instructions in your output and flag any correlation that could lead to incorrect suppression of a critical alert.

To adapt this template for your environment, replace the placeholders with live data from your monitoring stack. The [ALERT_PAYLOADS] should be a JSON array of alert objects containing at minimum an alert_id, service, timestamp, and description field. The [SERVICE_DEPENDENCY_GRAPH] must reflect your actual service topology—stale or incomplete dependency data will cause the model to miss correlations or invent false ones. Set [RISK_LEVEL] based on the incident severity: use "high" when the alert storm affects revenue-critical paths or involves data integrity risks, which will trigger additional caution in the output. Before deploying this prompt, run it against a set of known historical alert storms where you have ground-truth root causes, and measure whether the model correctly identifies the root alert and suppresses the right downstream noise. If your alert payloads are large, consider summarizing or filtering them before insertion to stay within context limits.

The Alert Storm Correlation Prompt is designed to be called during an active incident, not as a post-hoc analysis tool. Wire it into your incident response pipeline so that when an on-call engineer acknowledges a storm, the system automatically gathers the last 15–30 minutes of alert data from PagerDuty, Opsgenie, or your monitoring aggregation layer and feeds it into the prompt as [ALERT_PAYLOADS]. The prompt expects a JSON array of alert objects, each containing at minimum an alert_id, service, timestamp, severity, title, and description. If your alerting system provides additional metadata like hostname, cluster, or runbook links, include those fields as well—the model uses them to strengthen correlation edges.

Build a thin orchestration layer that handles the full lifecycle: fetch alerts from your alerting API, deduplicate by alert_id, sort by timestamp, and inject them into the prompt template. After the model returns the correlation graph and prioritized investigation path, validate the output against a strict schema. The correlation_graph must be a valid DAG with nodes (alert_id, service, severity, timestamp) and edges (source_alert_id, target_alert_id, relationship_type, confidence). The investigation_path must be an ordered list of alert_ids starting with the probable root alert. If validation fails, retry once with a repair prompt that includes the schema violation details. Log every invocation—input alert count, output graph size, validation pass/fail, and the root alert candidate—so you can measure precision and recall against postmortem findings later.

Model choice matters here. Use a model with strong reasoning capabilities and a large context window (200k+ tokens) because alert storms can produce dozens of alerts with verbose descriptions. GPT-4o or Claude 3.5 Sonnet are good defaults. If you're running in a private environment, a fine-tuned open-weight model trained on historical incident data will outperform a generic model, especially on service topology awareness. Do not use a small or fast model for this task—the cost of a wrong root cause hypothesis during an incident far outweighs the latency or token savings. Set a timeout of 30 seconds and implement a fallback that surfaces the raw alert list sorted by severity if the model call fails.

This prompt is high-risk because it influences incident response decisions. Always present the output as a hypothesis, not a conclusion. The investigation path should be displayed alongside a one-click option to open the relevant service dashboard or runbook. Never auto-suppress alerts based solely on the model's correlation output—require explicit human confirmation before muting or downgrading any alert. If you integrate this into an automated runbook, add a mandatory approval step that logs who reviewed the correlation graph and when. For post-incident evaluation, store the model's output alongside the ground truth root cause determined during the postmortem, and use that dataset to tune future prompt versions and measure correlation accuracy over time.