Inferensys

Prompt

Alert Severity Classification Rubric Prompt

A practical prompt playbook for using Alert Severity Classification Rubric Prompt in production AI workflows.
Operations team reviewing AI workflow automation on laptop, workflow builder visible, casual office setup.
PROMPT PLAYBOOK

When to Use This Prompt

Define the job, reader, and constraints for the Alert Severity Classification Rubric Prompt.

This prompt is for SREs, incident commanders, and platform engineering leads who need to replace ad-hoc severity calls with a consistent, defensible classification rubric. The job-to-be-done is producing a structured taxonomy that maps observable incident signals—user impact, data loss risk, blast radius, time sensitivity—to defined severity levels (SEV-1 through SEV-5), each with explicit response time expectations, escalation paths, and notification channels. Use this when your organization lacks a shared severity vocabulary, when cross-team incidents produce conflicting severity assignments, or when you are codifying an incident response program and need a reference document that survives an audit.

Do not use this prompt for real-time incident triage during an active outage. It is a design-time tool for creating the rubric, not a runtime classifier for live alerts. It is also not a replacement for an incident management platform's automation rules; the output is a policy document that humans and systems reference, not executable code. The prompt assumes you can provide context about your organization's scale, team structure, regulatory obligations, and existing incident tooling. Without that context, the model will produce a generic rubric that may not fit your operational reality. If you need a runtime severity classifier that acts on incoming alerts, pair this rubric output with a separate classification routing prompt that references the finalized taxonomy.

After generating the rubric, treat it as a draft for stakeholder review. Engineering managers, product owners, and customer support leads should validate that the severity definitions align with actual business priorities and that response times are achievable given on-call staffing. The next step is to load the rubric into your incident management tool, runbook system, and on-call training materials, then test it against a set of historical incidents to verify that classification is consistent across responders.

PRACTICAL GUARDRAILS

Use Case Fit

Where the Alert Severity Classification Rubric Prompt delivers value and where it introduces operational risk.

01

Good Fit: Standardizing Incident Response

Use when: Your organization lacks a consistent severity taxonomy, leading to confusion during incidents. Guardrail: Run the generated rubric through a review with incident commanders from each major service team to ensure the definitions map to real operational pain, not abstract ideals.

02

Good Fit: Onboarding New SREs

Use when: New on-call engineers need a clear, unambiguous guide to classify incidents without tribal knowledge. Guardrail: Pair the rubric with 3-5 canonical examples of past incidents for each severity level to calibrate judgment before the first live incident.

03

Bad Fit: Replacing Human Judgment in Novel Failures

Avoid when: The incident involves a never-before-seen failure mode that doesn't fit the pre-defined categories. Guardrail: Always include a 'SEV-0 / Unclassified' escalation path that requires a human incident commander to declare severity for novel events, preventing misclassification under pressure.

04

Required Input: Business Impact Definitions

Risk: Without clear revenue, legal, or compliance impact thresholds, the model generates generic severity levels that don't trigger the right executive response. Guardrail: Provide concrete business metrics (e.g., '>$10k revenue loss per minute' or 'PII exposure >100 records') as [BUSINESS_IMPACT_THRESHOLDS] in the prompt template.

05

Operational Risk: Alert Fatigue Amplification

Risk: A poorly calibrated rubric can classify too many events as SEV-1, increasing alert fatigue and desensitizing responders. Guardrail: After deployment, run a retroactive analysis on the last 30 days of incidents to measure the distribution across severity levels. If >5% of incidents land in the top severity tier, recalibrate the definitions.

06

Operational Risk: Cross-Team Inconsistency

Risk: Different teams interpret the same rubric language differently, leading to inconsistent severity assignment across services. Guardrail: Use the prompt's eval checks to run a calibration exercise where 5 engineers independently classify the same 10 incident scenarios. Address any scenario with <80% agreement by refining the rubric language.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A reusable prompt for generating a consistent alert severity classification rubric with definitions, response times, and escalation paths.

This prompt template is designed to produce a standardized severity classification rubric for your organization. It forces the model to define clear, mutually exclusive SEV levels, map them to operational response expectations, and provide concrete examples. The output is intended to be a starting point for an SRE team to debate, calibrate, and operationalize, not a final policy to be adopted without review.

text
You are an SRE architect designing an alert severity classification rubric for an engineering organization.

Your task is to produce a clear, actionable rubric that incident commanders and on-call engineers can use to consistently classify incidents.

[CONTEXT]
The following is information about our system and organizational requirements:
- System Description: [SYSTEM_DESCRIPTION]
- User Base: [USER_BASE_DESCRIPTION]
- Existing SLO/SLI Targets (if any): [SLO_TARGETS]
- Regulatory or Compliance Requirements: [COMPLIANCE_REQUIREMENTS]
- On-Call Team Structure: [ON_CALL_STRUCTURE]

[OUTPUT_SCHEMA]
Produce a JSON object with the following structure:
{
  "rubric_name": "string",
  "version": "1.0",
  "severity_levels": [
    {
      "level": "SEV-0 | SEV-1 | SEV-2 | SEV-3 | SEV-4",
      "label": "string (e.g., Critical, Major, Minor, Warning, Info)",
      "definition": "string (a precise, testable definition of the impact)",
      "user_impact": "string (description of what users experience)",
      "response_sla": {
        "acknowledge": "string (e.g., 5 minutes)",
        "resolve_or_mitigate": "string (e.g., 1 hour)"
      },
      "escalation_path": "string (who is paged and when escalation occurs)",
      "notification_channel": "string (e.g., PagerDuty, Slack #incidents, email)",
      "examples": ["string (a specific, realistic scenario for this system)"],
      "post_incident_action": "string (e.g., Requires postmortem, Requires root cause analysis, Logged for trend analysis)"
    }
  ],
  "classification_guidelines": [
    "string (a rule to resolve ambiguity between levels)"
  ]
}

[CONSTRAINTS]
- Severity levels must be mutually exclusive. No incident should fit two definitions.
- The rubric must cover the full spectrum from "no user impact" to "complete service outage."
- Response SLAs must be realistic for the on-call structure provided.
- Examples must be specific to the system description, not generic.
- If regulatory requirements exist, ensure at least one severity level captures compliance-reportable events.
- Do not invent SLOs if none are provided; base definitions on user impact instead.

To adapt this template, replace the placeholders in the [CONTEXT] block with your specific system details. The more precise you are about your user base and compliance requirements, the more tailored the examples and definitions will be. After generating the rubric, treat it as a draft for a collaborative review session with your on-call team. The classification_guidelines field is critical for resolving edge cases; if the model's suggestions are insufficient, add your own based on past incident ambiguity. Always run the output through the evaluation checks described in the "Testing and Evaluation Criteria" section before considering it production-ready.

IMPLEMENTATION TABLE

Prompt Variables

Required inputs for the Alert Severity Classification Rubric Prompt. Each placeholder must be populated before the prompt is sent. Validation notes describe how to check the input before execution.

PlaceholderPurposeExampleValidation Notes

[ALERT_SOURCE]

Identifies the originating system or service emitting the alert

payment-gateway-prod-us-east-1

Must match a known service in the CMDB. Reject if source is null or 'unknown'.

[ALERT_TRIGGER_CONDITION]

The raw condition or metric threshold that fired the alert

HTTP 5xx error rate > 2% for 5 minutes

Must contain a metric name, threshold, and evaluation window. Parse for completeness; flag if window is missing.

[CURRENT_INCIDENT_STATE]

Whether an incident is already open, recently resolved, or this is a new standalone alert

new" or "ongoing-incident-INC-00421" or "resolved-2h-ago

Must be one of: 'new', 'ongoing-incident-<ID>', 'resolved-<N>-<unit>-ago'. Reject ambiguous states.

[AFFECTED_CUSTOMER_PATHS]

List of user-facing features, APIs, or journeys impacted by the alert condition

['/checkout', '/api/payments', '/login']

Must be a non-empty array of strings. Each path must match a registered route in the service catalog. Null allowed only if impact is unconfirmed.

[SLO_TARGETS]

Relevant Service Level Objectives and their current error budget status for the affected service

{ 'latency-p99': { 'target': '200ms', 'burn_rate': 0.8 }, 'availability': { 'target': '99.95%', 'remaining_budget_pct': 12 } }

Must be a valid JSON object. Each key must map to an object with 'target' and at least one of 'burn_rate' or 'remaining_budget_pct'. Reject if SLO data is stale (>1 hour old).

[ON_CALL_ROTATION]

Current on-call tier and escalation policy in effect at the time of the alert

Tier1-EMEA: alice, bob; Tier2: charlie (manager: dave)

Must contain at least one tier with a non-empty list of names. Validate against the live pager rotation API. Reject if rotation is unpopulated or expired.

[HISTORICAL_SIGNAL_PATTERN]

Summary of whether this alert is a known flake, recurring pattern, or novel signal

Recurring: fires 3x/week during peak, usually self-resolves within 8 minutes. Last 10 occurrences: 0 escalated to SEV2.

Must be a string or null. If non-null, must cite a time window and count. Flag if historical data is older than 7 days or missing for a known service.

[OUTPUT_SCHEMA]

The exact JSON schema the model must use for its severity classification response

{ 'severity_level': 'SEV1'|'SEV2'|'SEV3'|'SEV4'|'SEV5', 'rationale': 'string', 'recommended_escalation': 'string', 'confidence': 0.0-1.0 }

Must be a valid JSON Schema object. Validate with a schema parser before injection. Reject if schema is missing required fields or uses unsupported types.

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the Alert Severity Classification Rubric prompt into an incident management or alerting pipeline with validation, retries, and human review gates.

This prompt is designed to be called programmatically, not used as a one-off chat interaction. The primary integration point is within an incident management or alerting platform where a raw alert payload or incident description needs to be classified into a consistent severity taxonomy. The application should construct the prompt by injecting the specific alert context into the [ALERT_PAYLOAD] placeholder and the organization's operational constraints into [ORG_CONTEXT]. The model's response must be parsed as structured JSON and validated before any downstream action, such as paging an on-call engineer or updating an incident ticket, is taken.

A robust implementation harness requires strict output validation. The model's JSON response must conform to a predefined schema that includes the severity level, a justification grounded in the provided rubric, and a confidence score. Implement a validation layer that checks for: (1) the presence of all required fields, (2) the severity level matching one of the allowed enum values from the rubric, and (3) the confidence score being a number between 0 and 1. If validation fails, implement a retry loop with a maximum of two additional attempts, feeding the validation error message back into the prompt as [PREVIOUS_ERROR] to guide self-correction. For high-severity classifications (e.g., SEV0 or SEV1), the system should bypass automatic execution and instead route the classification and justification to a human incident commander for explicit approval before triggering the notification workflow.

Logging is critical for auditability and debugging. Log the full prompt, the raw model output, the validated classification, and any retry attempts to a structured logging system, ensuring that any sensitive data within the alert payload is redacted according to your telemetry data governance policies. Model choice involves a trade-off between latency and reasoning quality; a faster model may suffice for clear-cut cases, but a more capable model should be used when the initial classification confidence is below a threshold (e.g., 0.85). To prevent alert fatigue, integrate a deduplication check before classification: if an active incident with a similar fingerprint already exists, the harness should attach the new alert as a note to the existing incident rather than creating a new one and re-running the classification prompt.

PRACTICAL GUARDRAILS

Common Failure Modes

Alert severity classification prompts fail in predictable ways. These cards cover the most common failure modes and how to guard against them before an incident commander relies on the rubric.

01

Severity Inflation Under Pressure

What to watch: During active incidents, the model elevates severity because the prompt contains urgent language or the incident description sounds dramatic. A slow database replica gets classified as SEV-0. Guardrail: Anchor the rubric with objective, measurable criteria (user impact percentage, revenue loss threshold, data corruption confirmed) and instruct the model to downgrade unless thresholds are explicitly met.

02

Inconsistent Classification Across Teams

What to watch: Two SRE teams using the same prompt get different severity levels for identical incidents because the rubric definitions are ambiguous. 'Critical' and 'major' mean different things to the database team and the frontend team. Guardrail: Define each severity level with a concrete, falsifiable condition. Include negative examples in the prompt: 'This is NOT a SEV-1 because fewer than 10% of users are affected.'

03

Ignoring Response Time and Escalation Paths

What to watch: The prompt produces a severity label but omits the required response time, notification channel, and escalation path. The classification is technically correct but operationally useless. Guardrail: Require the output schema to include acknowledgment_time, escalation_policy, and notification_channel for each severity level. Validate these fields are populated before accepting the output.

04

Hallucinated Severity Levels

What to watch: The model invents a SEV-4 level when the organization only defines SEV-0 through SEV-3, or it merges two levels into a hybrid. Guardrail: Provide the exact list of allowed severity levels as an enum in the prompt. Add a post-processing validation step that rejects any output containing undefined severity labels before it reaches the incident commander.

05

Context Window Truncation of Incident Details

What to watch: Long incident threads, multi-channel Slack transcripts, or verbose monitoring outputs exceed the context window. The model classifies severity based on only the most recent or truncated information, missing the root cause. Guardrail: Summarize incident context into a structured brief before classification. Include a [INCIDENT_SUMMARY] placeholder with explicit fields: affected service, user impact count, duration, and known root cause indicators.

06

Overfitting to Historical Incident Patterns

What to watch: The prompt includes few-shot examples from past incidents, and the model pattern-matches superficially. A database timeout gets classified the same as a previous SEV-0 outage even though this time it's a single read replica with automatic failover. Guardrail: Use counterexamples in few-shot demonstrations. Show pairs where similar symptoms produce different severities because of differing blast radius or mitigation availability. Test the prompt against synthetic edge cases that look like past incidents but have different impact.

IMPLEMENTATION TABLE

Evaluation Rubric

Use this rubric to test the Alert Severity Classification Rubric Prompt before shipping. Each criterion targets a specific failure mode common in severity taxonomy generation.

CriterionPass StandardFailure SignalTest Method

SEV Level Completeness

Output defines at least 4 distinct SEV levels (e.g., SEV-0 through SEV-3) with unique definitions

Fewer than 4 levels defined, or multiple levels share identical definitions

Count distinct SEV labels in output; check for semantic overlap in definitions using pairwise comparison

Response Time Specification

Each SEV level includes a concrete response time target (e.g., '5 minutes', '30 minutes')

Response times are missing, vague ('ASAP'), or identical across all severity levels

Regex match for time units in each SEV definition; verify values differ across levels

Escalation Path Clarity

Each SEV level specifies who is notified and by what channel (e.g., 'page on-call SRE via PagerDuty')

Escalation paths are missing, reference generic roles without channels, or use circular escalation logic

Check for presence of both role and channel per SEV level; verify no SEV level escalates to itself

Operational Definition Consistency

SEV definitions use observable, measurable criteria (e.g., '>5% error rate for 2 minutes') rather than subjective language

Definitions contain subjective terms like 'major', 'significant', or 'critical' without quantifiable thresholds

Scan for subjective adjectives; flag any SEV definition lacking a numeric threshold or binary condition

Cross-Team Applicability

Rubric includes guidance for adapting SEV definitions to different service contexts without breaking the taxonomy

Rubric assumes a single service type or fails to address how teams customize thresholds

Check for adaptation instructions or placeholder slots for service-specific thresholds in output

Incident Commander Role Definition

Output specifies who declares severity, who can change it during an incident, and under what conditions

No incident commander role defined, or severity change authority is unconstrained

Search for 'incident commander' or equivalent role; verify change conditions are documented

Runbook Linking Requirement

Each SEV level references a required runbook or playbook link (even if placeholder)

Runbook references are absent, or only some SEV levels include them

Verify each SEV level section contains a runbook reference field or explicit 'no runbook' statement

Fatigue Reduction Mechanism

Rubric includes at least one rule to prevent alert fatigue (e.g., 'auto-resolve after 1 hour if no change', 'deduplicate by incident ID')

No fatigue reduction rules present, or rules would suppress legitimate escalations

Search for deduplication, auto-resolution, or suppression rules; validate they include safety conditions

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Start with the base rubric prompt and a single SEV level definition. Use a lightweight model call without strict schema enforcement. Focus on getting the taxonomy language right before adding operational constraints.

  • Remove response time and escalation path fields from [OUTPUT_SCHEMA] initially.
  • Use a simple list format instead of nested JSON for early drafts.
  • Test with 3-5 historical incidents to calibrate severity boundaries.

Watch for

  • Severity levels that sound distinct but overlap in practice (SEV-2 vs SEV-3 blur).
  • Missing edge cases: degraded vs down, partial vs total, internal vs customer-facing.
  • Overly academic language that on-call engineers won't use at 3 AM.
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.