Inferensys

Prompt

Incident Severity Classification Prompt for Ops

A practical prompt playbook for using Incident Severity Classification Prompt for Ops in production AI workflows.
Developer doing prompt engineering on laptop, prompt variations visible on screen, casual coding session.
PROMPT PLAYBOOK

When to Use This Prompt

Understand the job-to-be-done, required inputs, and operational boundaries for the incident severity classification prompt.

This prompt is designed for SRE and operations engineers who need a consistent, auditable method for classifying production incidents into severity tiers (SEV0 through SEV4). The core job-to-be-done is translating an unstructured incident report—often written under pressure during an active event—into a structured severity assessment that directly controls paging policies, escalation paths, and response SLAs. Use this when your on-call workflow requires a deterministic severity label before automated alerting fires, or when you need to reassess severity during a postmortem to validate whether the initial classification matched actual impact. The prompt expects three categories of input: a free-text incident description, monitoring context (such as error rates, latency spikes, or alert annotations), and service catalog data that maps affected components to their criticality tier and owning team.

Do not use this prompt when the incident is still being triaged and impact is entirely unknown—severity classification requires at least a preliminary symptom description and some signal from monitoring systems. It is also inappropriate for non-production environments where severity tiers carry no operational consequence, or for classifying customer-reported issues that have not yet been confirmed as platform incidents. The prompt is not a replacement for your on-call engineer's judgment; it is a decision-support tool that produces a structured assessment with evidence alignment checks. In regulated environments where misclassification carries compliance risk, always route the prompt's output through a human reviewer before it triggers paging or external notifications. The classification must be traceable to specific evidence in the input, and the prompt includes explicit checks for missing information that would make a severity determination unreliable.

Before integrating this prompt into your incident response pipeline, ensure you have a well-defined service catalog with criticality mappings and documented on-call alerting thresholds for each severity tier. The prompt's effectiveness depends on these reference data being accurate and available at classification time. Start by testing against a golden dataset of 20-30 historical incidents where severity was correctly assigned and confirmed in postmortems, then measure classification agreement and identify systematic errors such as over-severity on noisy alerts or under-severity on silent failures. The next section provides the copy-ready prompt template with all required placeholders and adaptation guidance.

PRACTICAL GUARDRAILS

Use Case Fit

Where this prompt works and where it does not. Understand the operational boundaries before wiring it into your incident response pipeline.

01

Good Fit: Structured Incident Triage

Use when: production incidents arrive with structured metadata (service name, alert type, metrics) and you need a consistent severity tier mapped to on-call thresholds. Guardrail: validate the output severity against your runbook definitions before paging anyone.

02

Bad Fit: Ambiguous User Reports

Avoid when: the input is a vague user complaint with no service context, no metrics, and no reproducible steps. Guardrail: require a minimum set of structured fields (affected service, user impact count, error rate) before invoking classification; otherwise route to human triage.

03

Required Inputs

What you need: affected service name, observable degradation evidence, user-facing impact description, and current alert state. Guardrail: build a pre-processing check that rejects inputs missing two or more required fields and returns a structured clarification request instead of guessing severity.

04

Operational Risk: Threshold Misalignment

What to watch: the prompt assigns severity based on general impact language, but your on-call thresholds use specific SLO burn rates or error budget consumption. Guardrail: post-process the classification against your actual alerting rules; never let the prompt output directly trigger a page without threshold validation.

05

Operational Risk: Postmortem Drift

What to watch: severity assigned during incident response may not match the severity determined during postmortem review, creating audit gaps. Guardrail: store the prompt output with timestamp and input snapshot; flag any case where postmortem severity differs from initial classification for retrospective calibration.

06

Escalation Boundary

What to watch: the prompt confidently classifies an incident as low severity when evidence is thin, delaying response to a degrading situation. Guardrail: when confidence indicators are low or evidence is incomplete, default to one severity tier higher and require human acknowledgment within a short SLA window.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A copy-ready prompt for classifying production incidents into severity tiers with impact scope, affected service, and user-facing degradation evidence.

This prompt template is designed to be pasted directly into your AI system. It expects a defined severity taxonomy (e.g., SEV0-SEV4) and a service catalog as pre-loaded context. Replace all square-bracket placeholders with your organization's specific values before execution. The prompt instructs the model to produce a structured classification that includes the severity tier, a justification grounded in the provided incident description, the scope of impact, and any evidence of user-facing degradation.

text
You are an SRE incident classifier. Your task is to analyze the provided incident description and assign a severity level based on the defined taxonomy.

[SEVERITY_TAXONOMY]
[SERVICE_CATALOG]

Incident Description:
[INCIDENT_DESCRIPTION]

Classify this incident and return a single JSON object with the following keys:
- "severity": The assigned severity level (e.g., "SEV1").
- "impact_scope": A string describing the scope of impact (e.g., "Global", "Single Region", "10% of Users").
- "affected_service": The primary service from the catalog that is affected.
- "user_degradation_evidence": A boolean indicating if there is explicit evidence of user-facing degradation in the description.
- "justification": A concise explanation linking the incident details to the specific criteria in the severity taxonomy.

[CONSTRAINTS]
- If the incident description is insufficient to determine severity, set "severity" to "UNKNOWN" and explain why in the "justification".
- Do not invent details not present in the incident description.
- Prioritize user-facing impact over internal system metrics when determining severity.

To adapt this template, replace the placeholders with your actual data. [SEVERITY_TAXONOMY] should contain the full text of your severity definitions. [SERVICE_CATALOG] should list your known services. [INCIDENT_DESCRIPTION] is the raw text of the alert or incident report. The [CONSTRAINTS] section is critical for high-risk operations; it forces the model to acknowledge uncertainty rather than hallucinate a severity level. Before deploying, run this prompt against a golden dataset of historical incidents to calibrate its alignment with your on-call alerting thresholds and to ensure the UNKNOWN fallback triggers appropriately on ambiguous inputs.

IMPLEMENTATION TABLE

Prompt Variables

Each placeholder must be populated before the prompt is sent to the model. Missing or malformed variables produce unreliable classifications. Validate all inputs at the application layer before prompt assembly.

PlaceholderPurposeExampleValidation Notes

[INCIDENT_TITLE]

Brief incident name or summary line from the alerting system

Payment API returning 503 errors in us-east-1

Required. Non-empty string. Max 200 chars. Reject if null or whitespace-only.

[INCIDENT_DESCRIPTION]

Full incident body: symptoms, timeline, affected components, user reports

Since 14:22 UTC, Payment API latency spiked to 12s p95. Error rate at 34%. Customers reporting failed checkouts.

Required. Non-empty string. Min 20 chars. Truncate to 4000 tokens before sending. Flag if description is a duplicate of title.

[AFFECTED_SERVICE]

Canonical service name from the service catalog

payment-api

Required. Must match an entry in the service catalog lookup table. Reject unknown service names. Case-insensitive match recommended.

[USER_IMPACT_SIGNALS]

Structured field capturing user-facing degradation evidence

{"error_rate_pct": 34, "latency_p95_ms": 12000, "user_reports_count": 47, "workaround_available": false}

Required. Must parse as valid JSON. All numeric fields must be non-negative. workaround_available must be boolean. Reject if unparseable.

[ALERT_SOURCE]

Originating monitoring system or trigger mechanism

pagerduty

Required. Must be one of the allowed enum values: pagerduty, datadog, grafana, opsgenie, manual, victorops. Reject unknown sources.

[ON_CALL_POLICY_VERSION]

Version identifier for the current on-call severity definitions

v2.4

Required. Must match a deployed policy version in the policy registry. Reject if version not found. Prevents classification against stale severity definitions.

[CURRENT_TIMESTAMP_UTC]

ISO-8601 timestamp when classification is requested

2025-03-15T14:28:00Z

Required. Must parse as valid ISO-8601 UTC timestamp. Must not be in the future. Used for SLA clock start and time-sensitivity calculations.

[PREVIOUS_INCIDENT_IDS]

List of related or recent incident IDs for context, or empty array

["INC-1423", "INC-1418"]

Optional. Must parse as array of strings or empty array. Null allowed. If provided, each ID must match incident ID format. Used for deduplication and pattern detection.

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the incident severity classification prompt into an on-call or incident management workflow with validation, retries, and human review.

This prompt is designed to be called as part of an automated triage step when a new incident is declared or an alert fires. The typical integration point is a webhook receiver or a messaging bot that listens for incident creation events from PagerDuty, Opsgenie, or a custom monitoring system. The application layer should extract the raw incident title and description, populate the [INCIDENT_DETAILS] placeholder, and inject your organization's specific severity definitions into [SEVERITY_DEFINITIONS] before calling the model. Do not pass raw alert payloads with unredacted secrets, API keys, or internal hostnames directly into the prompt; strip those at the application layer first.

The model's output must be validated against a strict schema before any downstream action is taken. Expect a JSON object with severity_level, impact_scope, affected_service, and user_facing_evidence fields. Implement a validation guard that rejects any response missing these fields or containing a severity_level outside your defined enum (e.g., SEV1-SEV5). If validation fails, retry once with the same prompt and a stronger instruction to output valid JSON. If the second attempt also fails, escalate to a human on-call responder with the raw incident details and a note that automated classification failed. Log every classification attempt, including the raw model response and validation result, for postmortem analysis and prompt improvement.

For high-risk environments, add a human approval gate for any classification of SEV1 or SEV2 before the incident is broadcast or paged. The system should post the classification result to a review channel and wait for a human to confirm or override. This prevents a misclassification from waking up the wrong team or under-prioritizing a critical outage. For lower-severity classifications (SEV3-SEV5), you can auto-route to the appropriate queue, but always include the classification evidence in the incident ticket so responders can challenge it if needed. Avoid wiring this prompt directly to paging automation without a confirmation step for top-tier severities.

IMPLEMENTATION TABLE

Expected Output Contract

The model must return a JSON object matching this schema. Validate every field before the classification is used to trigger pages or create incidents.

Field or ElementType or FormatRequiredValidation Rule

severity_level

string (enum: SEV1, SEV2, SEV3, SEV4, SEV5)

Must match exactly one of the defined enum values. Reject any response where this field is missing, null, or contains an unlisted string. Map to on-call alerting thresholds before dispatch.

impact_scope

object

Must contain affected_users (integer or 'unknown'), affected_services (array of strings, min 1 item), and degradation_type (enum: full_outage, partial_outage, degradation, none). Reject if services array is empty or degradation_type is 'none' when severity is SEV1 or SEV2.

user_facing_evidence

array of objects

Each object must have source (string: monitoring, user_report, synthetic_check, internal_observation) and description (string, min 10 chars). Array must contain at least 1 item for SEV1-SEV3. Reject if evidence is missing for high-severity incidents.

confidence_score

number (0.0 to 1.0)

Must be a float between 0 and 1 inclusive. If confidence is below 0.7, the incident should be routed for human review before paging. Reject if score is outside range or not a number.

recommended_action

string (enum: page_on_call, create_ticket, monitor, no_action)

Must align with severity level: SEV1/SEV2 require 'page_on_call', SEV3 requires 'create_ticket', SEV4/SEV5 require 'monitor' or 'no_action'. Reject if action-severity mapping is violated.

classification_rationale

string

Must be a non-empty string (min 20 chars) explaining why this severity was chosen. Must reference specific evidence from user_facing_evidence. Reject if rationale is generic, circular, or fails to cite evidence.

timestamp_utc

string (ISO 8601)

Must be a valid ISO 8601 UTC datetime string (e.g., 2025-01-15T14:30:00Z). Reject if format is invalid, timezone is missing, or timestamp is in the future by more than 5 minutes.

escalation_path

string or null

If provided, must be a non-empty string identifying the escalation team or contact. Required when confidence_score is below 0.7 or severity is SEV1. Null allowed for SEV4/SEV5 with high confidence. Reject if null when confidence is low.

PRACTICAL GUARDRAILS

Common Failure Modes

What breaks first when classifying incident severity in production and how to guard against it. Each failure mode includes a detection method and a mitigation.

01

Vague Impact Descriptions Produce Wrong Severity

What to watch: The prompt receives 'things are slow' or 'users are complaining' without concrete scope, causing it to default to a middle severity or hallucinate impact numbers. Guardrail: Require the prompt to output a missing_evidence flag and refuse to classify above SEV3 when user impact scope, affected service, or degradation evidence is absent. Pair with a pre-processing step that extracts explicit metrics from the alert payload before classification.

02

Silent Misalignment with On-Call Thresholds

What to watch: The model classifies an incident as SEV2, but your on-call tooling pages only on SEV1 labels. The incident sits unacknowledged because the severity label doesn't match the routing rules. Guardrail: Add a routing_action field to the output schema that maps the severity tier to the exact paging policy name used by PagerDuty or Opsgenie. Validate this field against a known list of active escalation policies in your harness.

03

Postmortem Severity Reassessment Drift

What to watch: The real-time severity assigned during the incident is lower than what the postmortem process later determines, making the initial classification look under-reactive. Guardrail: Include a postmortem_severity field in the output that is initially null. Run a separate post-incident evaluation prompt that compares the initial classification against the final postmortem severity to measure drift and tune the prompt's sensitivity to early weak signals.

04

Overfitting to Specific Keywords

What to watch: The prompt learns that 'outage' always means SEV1 and 'degraded' always means SEV3, ignoring the actual scope. A full regional outage of an internal tool gets SEV1, while a partial degradation of a customer-facing payment system gets SEV3. Guardrail: Use a multi-factor rubric in the prompt instructions that weights user impact, revenue risk, and affected service tier independently. Test with adversarial examples where keyword severity contradicts actual business impact.

05

Confidence Collapse on Ambiguous Alerts

What to watch: The model encounters an alert from a new service or a vague monitoring message and produces a low-confidence SEV3 to avoid being wrong, under-escalating a potential SEV1. Guardrail: Require a confidence_score (0-1) in the output. When confidence is below 0.7, the prompt must escalate to a human on-call with a 'triage requested' label instead of auto-assigning a severity. Log all low-confidence classifications for review.

06

Ignoring User-Facing Degradation Evidence

What to watch: The prompt focuses on infrastructure metrics ('CPU spike on payment-service') and misses the user-facing evidence ('checkout flow returning 500 errors for 40% of users'), resulting in a lower severity classification. Guardrail: Structure the prompt to separate 'Infrastructure Signal' from 'User-Facing Evidence' and require explicit user-impact statements before finalizing severity. If user-facing evidence is missing, the prompt must note this as a limitation in the justification.

IMPLEMENTATION TABLE

Evaluation Rubric

Test output quality before shipping. Run these checks against a golden dataset of 20-50 incidents with known correct severity classifications. Each check should pass on at least 90% of test cases before the prompt is deployed to production.

CriterionPass StandardFailure SignalTest Method

Severity Tier Accuracy

Exact match with golden label on >= 90% of cases

Off-by-one errors (e.g., SEV2 vs SEV1) on > 10% of cases

Run prompt against golden dataset; compare [SEVERITY_TIER] output to expected label per incident

Impact Scope Completeness

[IMPACT_SCOPE] field populated with non-null value on 100% of cases

Null or empty [IMPACT_SCOPE] on any test case

Parse output JSON; assert [IMPACT_SCOPE] is non-null string with >= 10 characters

Affected Service Identification

[AFFECTED_SERVICE] matches golden service label on >= 90% of cases

Wrong service name or missing service when incident clearly references one

Exact string match against golden [AFFECTED_SERVICE] field; allow for alias normalization

User-Facing Degradation Evidence

[DEGRADATION_EVIDENCE] contains at least one quoted excerpt from incident body on >= 85% of cases

Paraphrased or hallucinated evidence with no source grounding

Check that [DEGRADATION_EVIDENCE] substring appears verbatim in [INCIDENT_BODY] or is marked as inferred with confidence qualifier

On-Call Alerting Threshold Alignment

[SEVERITY_TIER] triggers correct alerting rule per runbook on >= 95% of cases

SEV1 assigned to incident that runbook classifies as SEV3 or vice versa

Cross-reference [SEVERITY_TIER] against on-call runbook threshold table; flag mismatches

Postmortem Severity Reassessment Consistency

Severity does not change during postmortem review for >= 90% of cases

Postmortem review downgrades or upgrades severity on > 10% of incidents

Simulate postmortem review by re-running prompt with full incident timeline; compare initial vs reassessed [SEVERITY_TIER]

Confidence Calibration

[CONFIDENCE] score >= 0.8 correlates with correct classification on >= 90% of high-confidence cases

High-confidence predictions (>0.8) with wrong severity tier on > 10% of cases

Bin outputs by [CONFIDENCE] value; compute accuracy per bin; flag bins where accuracy < confidence

Edge Case Handling: Multi-Service Incidents

Correctly identifies primary affected service when multiple services mentioned on >= 85% of cases

Lists all services without identifying primary or picks wrong primary service

Select golden cases with multi-service incidents; verify [AFFECTED_SERVICE] matches golden primary service label

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Start with the base prompt and a single model call. Use a simple severity enum (SEV1–SEV5) without strict schema enforcement. Accept free-text output and manually review classifications against your on-call runbook.

code
Classify this incident into one severity level: SEV1, SEV2, SEV3, SEV4, SEV5.

Incident: [INCIDENT_DESCRIPTION]
Affected service: [SERVICE_NAME]
User reports: [USER_REPORTS]

Watch for

  • Inconsistent severity labels across similar incidents
  • Model inventing impact scope without evidence
  • No alignment with your actual alerting thresholds
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.