Alert Fatigue Risk Assessment Prompt

This prompt is designed for SREs, platform engineers, and monitoring admins who need to audit their alert inventory and identify rules that are likely to generate noise, cause alert fatigue, or waste responder time. The job-to-be-done is proactive alert hygiene: evaluating a single alert rule's configuration against known fatigue factors before it triggers in production. The ideal user has access to the alert rule definition, its recent firing history, the associated runbook, and the service's SLO context. Required context includes the alert threshold, evaluation window, aggregation settings, and any suppression or grouping rules currently applied.

This is not a prompt for real-time incident response or live alert triage. Do not use it to evaluate an alert that is currently firing or to make a page/no-page decision during an active incident. It is also not a replacement for a full monitoring strategy review—it assesses one rule at a time. The prompt produces a structured risk score with sub-scores for threshold sensitivity, flapping potential, aggregation gaps, and runbook quality, along with specific tuning recommendations. It will flag rules that should be consolidated with other alerts, widened, or retired entirely. The output is designed to be consumed by a human reviewer or fed into an alert governance workflow, not to automatically modify production alert configurations.

Before using this prompt, gather the alert rule definition in a structured format (threshold, window, aggregation function, group-by keys, notification channels) and at least 48 hours of firing history if available. The prompt assumes you can provide the runbook link or content. If no runbook exists, the prompt will score that dimension as zero and flag it as a critical gap. After running the assessment, prioritize rules with high risk scores and missing runbooks first. Avoid batch-assessing hundreds of rules without human review of the recommendations—the prompt identifies patterns, but an engineer familiar with the service's failure modes must validate the tuning suggestions before applying them.

This prompt template evaluates an alert rule definition and its surrounding operational context to produce a structured fatigue risk score. It is designed for SREs and monitoring admins who need a repeatable, evidence-backed assessment before an alert goes live or during a noise-reduction initiative. The template expects you to supply the alert rule, recent firing history, runbook availability, and team context. It returns a risk score, a breakdown of contributing factors, and specific tuning recommendations.

textCopy
You are an SRE alert quality reviewer. Your task is to assess the fatigue risk of a monitoring alert rule and recommend improvements.

## INPUT
[ALERT_RULE_DEFINITION]
[RECENT_ALERT_HISTORY]
[RUNBOOK_URL_OR_NONE]
[TEAM_ON_CALL_ROTATION_CONTEXT]

## OUTPUT_SCHEMA
Return a single JSON object with this exact structure:
{
  "alert_name": "string",
  "fatigue_risk_score": number (0-100, where 0 = no fatigue risk and 100 = guaranteed noise),
  "risk_factors": [
    {
      "factor": "string (e.g., 'Low threshold sensitivity', 'No aggregation window', 'Missing runbook')",
      "severity": "high" | "medium" | "low",
      "explanation": "string (one sentence explaining why this factor contributes to fatigue)"
    }
  ],
  "flapping_risk": "high" | "medium" | "low",
  "recommendations": [
    {
      "action": "string (concrete tuning or process change)",
      "expected_impact": "string (how this reduces fatigue)",
      "effort": "low" | "medium" | "high"
    }
  ],
  "consolidation_candidates": [
    "string (names of other alerts this could be merged with, or 'none')"
  ],
  "overall_assessment": "string (2-3 sentence summary of risk and top priority action)"
}

## CONSTRAINTS
- Base your assessment on the provided alert definition and history. Do not invent missing data.
- If no runbook is provided, flag it as a high-severity risk factor.
- If alert history is empty or unavailable, note this as a limitation and score based on the rule definition alone.
- Consider flapping potential: alerts that fire and clear repeatedly within short windows.
- Check for aggregation: alerts without grouping or minimum-duration thresholds are higher risk.
- Evaluate threshold sensitivity: overly tight thresholds on noisy metrics increase fatigue.
- Assess whether the alert duplicates or overlaps with existing alerts.
- If the alert lacks a clear owner or playbook action, increase the risk score.

## EXAMPLES
Example input:
Alert rule: CPU > 80% for 1 minute on web servers
History: Fired 47 times in the last 24 hours, 42 self-resolved within 2 minutes
Runbook: None
Team context: 4-person on-call rotation, already receiving 200+ alerts/week

Example output:
{
  "alert_name": "High CPU - Web Servers",
  "fatigue_risk_score": 85,
  "risk_factors": [
    {"factor": "No aggregation window", "severity": "high", "explanation": "1-minute threshold causes transient spikes to trigger alerts that self-resolve quickly."},
    {"factor": "Missing runbook", "severity": "high", "explanation": "On-call responders have no documented remediation steps, increasing time-to-resolve and frustration."},
    {"factor": "High firing frequency", "severity": "high", "explanation": "47 firings in 24 hours with 89% self-resolution rate indicates the alert is too sensitive."}
  ],
  "flapping_risk": "high",
  "recommendations": [
    {"action": "Increase threshold duration to at least 5 minutes", "expected_impact": "Eliminates transient spike alerts", "effort": "low"},
    {"action": "Add a runbook with diagnostic steps and escalation path", "expected_impact": "Reduces responder uncertainty and resolution time", "effort": "medium"},
    {"action": "Consider a burn-rate alert on error budget instead of raw CPU", "expected_impact": "Alerts on sustained degradation rather than momentary spikes", "effort": "medium"}
  ],
  "consolidation_candidates": ["High Memory - Web Servers", "High Load Average - Web Servers"],
  "overall_assessment": "This alert is a major fatigue contributor due to high sensitivity, no runbook, and frequent self-resolving firings. Immediate priority: increase the threshold duration to 5 minutes and draft a minimal runbook. Consider replacing with a burn-rate alert tied to SLO."
}

## RISK_LEVEL
This assessment informs alert tuning decisions. Flag any recommendation that could mask real incidents. If the alert protects a critical user-facing SLO, note that tuning requires extra caution.

To adapt this template, replace each square-bracket placeholder with your actual data. The [ALERT_RULE_DEFINITION] should include the full PromQL, Datadog query, or monitoring config. The [RECENT_ALERT_HISTORY] can be a summary of firing counts, self-resolution rates, and timestamps. If you lack history, state that explicitly so the model can qualify its confidence. The [RUNBOOK_URL_OR_NONE] field accepts a URL, the runbook text, or the literal string "none". The [TEAM_ON_CALL_ROTATION_CONTEXT] should describe team size, current alert volume, and any known fatigue complaints. After pasting the template into your AI harness, validate the output against the schema before using it to drive alert tuning decisions. For high-severity production alerts, always have a human SRE review the recommendations before implementing threshold changes.

This prompt is designed to be integrated into a monitoring governance pipeline, not used as a one-off chat interaction. The typical integration point is a CI/CD check that runs against proposed alert rule changes, a periodic audit job that re-evaluates existing alert rules, or a review queue that surfaces high-risk alerts to the on-call team. The prompt expects structured input describing the alert rule, its thresholds, aggregation settings, and associated runbook metadata. The output is a structured risk score with recommendations, which your application should parse and act on programmatically.

To wire this into an application, construct the input payload by extracting alert rule definitions from your monitoring system's API (e.g., Prometheus rule files, Datadog monitor exports, PagerDuty alert configs). Map each rule to the [ALERT_RULE_DEFINITION] placeholder, including fields for alert_name, metric_query, threshold, evaluation_window, aggregation_method, runbook_url, and current_routing. Send the prompt to a model with strong structured output support (GPT-4o or Claude 3.5 Sonnet are good defaults). Parse the response against a JSON schema that expects risk_score (0-100), risk_factors (array of strings), recommendations (array of objects with action, rationale, and priority), and consolidation_candidates (array of alert names). If the model returns malformed JSON, retry once with a repair prompt that includes the schema and the raw output. Log every assessment with the alert rule ID, timestamp, model version, and risk score for auditability.

For high-severity production alerts, do not auto-apply tuning recommendations without human review. Route assessments with a risk_score above 70 to a review queue where an SRE can approve, reject, or modify the suggested changes. For lower-risk assessments, you can auto-apply non-destructive recommendations such as adding aggregation windows or linking missing runbooks, but threshold changes should always require explicit approval. Avoid running this prompt against alerts that are actively firing during an incident—the assessment is designed for steady-state evaluation, not real-time triage. The next step is to build a dashboard that tracks risk score trends over time so your team can measure whether alert hygiene is improving.