Error Severity Classification Prompt

This prompt is designed for incident response teams and SREs who need to triage production errors at scale. It classifies raw error logs, alert payloads, or incident reports into severity levels (SEV1 through SEV5) based on user impact, system scope, and recoverability. Use this prompt when you need a consistent, auditable triage queue that aligns with your SLA definitions and historical incident patterns. The prompt expects structured input containing error details, affected services, and current observability signals.

The ideal user is an on-call engineer or incident commander facing a high volume of alerts who needs to separate critical outages from minor degradations before human investigation begins. The prompt requires specific context to function reliably: a defined error taxonomy, current SLA thresholds, affected service criticality ratings, and recent deployment or config change logs. Without this context, severity assignments will be generic and potentially misaligned with business impact. The prompt is not a replacement for on-call judgment in novel catastrophic failures, nor does it handle real-time paging decisions without human review. It also cannot classify errors from services or failure modes absent from its provided taxonomy.

Before deploying this prompt, ensure you have a calibrated severity framework that defines what constitutes each level in your organization. Run the prompt against a labeled dataset of at least 50 historical incidents and measure agreement with human-assigned severities. Pay special attention to SEV1/SEV2 boundary errors—these are the most costly to misclassify. If agreement falls below 90% on critical severities, refine your input context or add few-shot examples drawn from your actual incident history. Never use this prompt to automatically trigger paging or escalation without a human approval step for SEV1 and SEV2 classifications.

This template is designed to be dropped into your incident triage pipeline. It expects structured input about a production error and returns a severity classification with justification, impact assessment, and recommended response actions. Replace every square-bracket placeholder with live data from your monitoring systems, alert payloads, or incident management tools before each inference call. The template includes calibration anchors against your historical incident data and SLA definitions to prevent severity inflation or underestimation.

textCopy
You are an incident severity classifier for a production engineering team. Your task is to classify the severity of a production error based on impact, scope, and recoverability. You must produce a structured classification with justification and recommended response actions.

## INPUT
[ERROR_DETAILS]

## CLASSIFICATION RULES
Use the following severity levels:
- **SEV0 (Critical):** Complete service outage affecting all users, data loss or corruption in progress, security breach with active exploitation, or revenue loss exceeding [$CRITICAL_REVENUE_THRESHOLD] per minute.
- **SEV1 (High):** Major feature unavailable for >[MAJOR_FEATURE_THRESHOLD]% of users, significant performance degradation blocking core workflows, or data inconsistency requiring immediate rollback.
- **SEV2 (Medium):** Partial feature degradation affecting <[PARTIAL_FEATURE_THRESHOLD]% of users, non-blocking errors with workaround available, or delayed processing within SLA but requiring attention.
- **SEV3 (Low):** Cosmetic issues, non-critical warnings, or errors affecting <[LOW_IMPACT_THRESHOLD]% of users with no workflow blockage.

## CALIBRATION ANCHORS
Use these historical examples to calibrate your classification:
[HISTORICAL_INCIDENT_EXAMPLES]

## SLA CONTEXT
Current SLA definitions:
[SLA_DEFINITIONS]

## OUTPUT SCHEMA
Return a JSON object with this exact structure:
{
  "severity": "SEV0|SEV1|SEV2|SEV3",
  "confidence": 0.0-1.0,
  "impact_summary": "One-sentence description of user-facing impact",
  "affected_scope": {
    "user_percentage": "estimated percentage or 'unknown'",
    "services": ["list of affected services"],
    "regions": ["list of affected regions or 'global'"]
  },
  "recoverability": "immediate|workaround_available|manual_intervention_required|unknown",
  "justification": "Evidence-based reasoning linking the error to the severity level, referencing specific calibration anchors where applicable",
  "recommended_actions": ["ordered list of immediate response steps"],
  "escalation_needed": true/false,
  "escalation_target": "on-call team or individual if escalation_needed is true, otherwise null"
}

## CONSTRAINTS
- If the error affects a revenue-critical path, bias toward higher severity.
- If the error has a known workaround that restores full functionality, reduce severity by one level.
- If insufficient information exists to classify confidently, set confidence below 0.7 and flag escalation_needed as true.
- Do not invent impact data. If user percentage is unknown, state 'unknown'.
- Reference specific calibration anchors in your justification when the error pattern matches a historical incident.

Adaptation guidance: Replace [ERROR_DETAILS] with structured error data from your observability pipeline—include the error message, stack trace, affected endpoint, user impact signals, and any alert context. Populate [HISTORICAL_INCIDENT_EXAMPLES] with 3-5 real incidents from your postmortem database, each showing the error signature and the severity level that was ultimately assigned. Fill [SLA_DEFINITIONS] with your organization's actual response time targets per severity tier. The revenue and percentage thresholds in the severity definitions should be tuned to your specific service's scale—a 5% user impact might be SEV0 for a small B2B product but SEV2 for a consumer app with millions of users. Test the prompt against 10-20 historical incidents where you know the correct severity, and adjust the calibration anchors until the model's classifications match your team's actual severity assignments at least 85% of the time before deploying to production triage.

The Error Severity Classification Prompt is designed to sit at the entry point of your incident triage pipeline, consuming raw error telemetry and producing a structured severity assessment before the incident reaches a human responder or automated runbook. In production, this prompt should be wrapped in a thin service layer that handles input normalization, output validation, and routing. The prompt expects a structured input payload containing the error signature, affected service, user impact indicators, and any available recovery context. Before calling the model, your harness should assemble this payload from your observability stack—pulling error codes from your APM, user impact counts from your analytics pipeline, and recovery status from your orchestration layer. Never pass raw, unnormalized log lines directly to the prompt; the classification quality depends on consistent input structure.

After the model returns a severity classification, your harness must validate the output against a strict schema before acting on it. The expected output is a JSON object with fields for severity_level (one of SEV0, SEV1, SEV2, SEV3, SEV4), confidence_score (0.0 to 1.0), justification (a concise explanation tied to the input evidence), and recommended_escalation (a boolean indicating whether this should bypass normal triage queues). Implement a post-processing validator that rejects any response where severity_level is not in the allowed enum, where confidence_score falls below a configurable threshold (start at 0.7 and tune based on your false-positive tolerance), or where the justification field is empty or fails to reference specific input fields. For SEV0 and SEV1 classifications, route the incident to an on-call responder immediately but always include the model's justification and confidence score so the human can override. For SEV2 and below, queue the incident in your ticketing system with the classification attached as metadata. Log every classification decision—including the raw prompt, model response, validation result, and final routing action—for auditability and prompt drift monitoring.

Model choice matters here. Use a model with strong instruction-following and structured output capabilities; GPT-4o, Claude 3.5 Sonnet, or equivalent are appropriate for this task. Avoid smaller or older models that may struggle with the nuanced trade-offs between SEV1 and SEV2 classifications, where the difference often hinges on subtle impact-scope distinctions. Set temperature to 0 or a very low value (0.1 maximum) to minimize variance in severity calls. If your incident volume is high, consider implementing a lightweight pre-filter: errors with zero user impact and known transient signatures can skip the LLM classification entirely and auto-resolve as SEV4. For high-risk domains—financial services, healthcare, critical infrastructure—always require human acknowledgment for SEV0 and SEV1 classifications within a defined SLA window. The prompt includes a [RISK_LEVEL] parameter; set this to high for regulated environments to instruct the model to prefer over-classification when uncertain, reducing the risk of a critical incident being misclassified as low severity.

Calibration is an ongoing process, not a one-time setup. Maintain a golden dataset of 50–100 historical incidents with expert-assigned severity labels, and run the prompt against this dataset weekly or after any prompt change. Track precision and recall per severity level, paying special attention to SEV0 recall (you cannot afford to miss a critical incident) and SEV3/SEV4 precision (you do not want low-severity noise flooding your on-call rotations). When the model disagrees with the historical label, log the disagreement for human review—these cases often reveal either prompt weaknesses or inconsistencies in your historical labeling. Integrate these evals into your CI/CD pipeline for the prompt itself, treating prompt changes with the same rigor as code changes. If your SLA definitions or incident taxonomy change, update the [CONSTRAINTS] block in the prompt template and re-run the full calibration suite before deploying.