Inferensys

Prompt

Anomaly Severity Classification Prompt Template

A practical prompt playbook for using Anomaly Severity Classification Prompt Template in production AI workflows. Includes a copy-ready prompt, variable definitions, output contract, evaluation rubric, and failure mode analysis for security and fraud operations teams building consistent triage pipelines.
Operations team reviewing AI workflow automation on laptop, workflow builder visible, casual office setup.
PROMPT PLAYBOOK

When to Use This Prompt

A practical guide to deciding when the Anomaly Severity Classification Prompt is the right tool for your triage workflow.

This prompt is designed for security operations, fraud operations, and incident response teams that need to apply a consistent, repeatable severity label to anomalies that have already been detected and enriched with context. The core job-to-be-done is standardizing the triage process so that a 'Critical' alert from one analyst or detection system carries the same weight and urgency as a 'Critical' alert from another. You should use this prompt when your team has pre-defined severity criteria and business impact definitions, but you are experiencing inconsistent labeling that leads to misrouted escalations, alert fatigue, or missed service level agreements (SLAs). The ideal user is a detection engineer, SOC analyst, or automation engineer who is building a triage pipeline and needs a classification step that is auditable, explainable, and calibration-ready.

This prompt is not a replacement for a risk scoring engine or an anomaly detection system. It does not compute raw risk scores, detect new anomalies, or ingest raw telemetry. It assumes that an upstream process has already identified an anomaly and enriched it with the necessary context, such as the affected asset, its criticality, the type of anomaly, and any relevant timestamps. The prompt's value is in its ability to apply your organization's specific business logic to that enriched context, producing a structured severity classification with an impact assessment, urgency justification, and a confidence indicator. For example, an input might be a JSON object describing a failed login spike on a production database server, and the output would classify it as 'High' severity with a justification citing the asset's criticality and the potential for a brute-force attack.

Avoid using this prompt for real-time blocking decisions where latency is measured in milliseconds; a model inference step adds unacceptable delay for inline transaction blocking. Similarly, do not use it when your severity definitions are vague or constantly changing—the prompt requires stable, documented criteria to produce reliable outputs. If you lack clear definitions, invest time in creating a severity matrix first. After classification, the structured output should be used to route the anomaly to the correct queue, trigger the appropriate SLA timer, and provide the human reviewer with an immediate understanding of why the alert matters. The next step is to wire this prompt into your SOAR or case management system and begin tracking inter-rater reliability to calibrate its performance against human analysts.

PRACTICAL GUARDRAILS

Use Case Fit

Where the Anomaly Severity Classification Prompt Template delivers value and where it introduces operational risk.

01

Good Fit: Structured Triage Queues

Use when: security or fraud operations teams need consistent severity labels across a high volume of heterogeneous anomalies. Guardrail: lock the severity criteria to a defined business impact rubric and validate inter-rater reliability against human analyst consensus before production deployment.

02

Bad Fit: Novel Zero-Day Exploit Analysis

Avoid when: the anomaly type is unprecedented and lacks historical precedent for impact calibration. Risk: the model will force-fit a novel threat into an existing severity bucket, masking true uncertainty. Guardrail: route first-seen anomaly patterns to a human-in-the-loop investigation queue instead of auto-classifying.

03

Required Inputs: Defined Impact Taxonomy

What to watch: the prompt requires a pre-defined severity taxonomy with concrete criteria per level (e.g., financial exposure, data sensitivity, blast radius). Guardrail: if the taxonomy is missing or ambiguous, the prompt will produce inconsistent labels. Maintain the taxonomy as a versioned configuration artifact separate from the prompt template.

04

Operational Risk: Severity Drift Over Time

What to watch: analyst teams gradually shift their interpretation of severity levels, causing the model's classifications to diverge from current operational expectations. Guardrail: run monthly calibration exercises comparing model outputs to human-labeled samples and retrain or adjust the rubric when drift exceeds 10% disagreement on critical severity cases.

05

Operational Risk: Over-Confidence in Ambiguous Cases

What to watch: the model assigns a definitive severity label even when evidence is contradictory or incomplete, suppressing the uncertainty signal. Guardrail: require the output schema to include a confidence score and an evidence-gap field. Escalate any classification where confidence falls below the defined threshold to a human reviewer.

06

Bad Fit: Real-Time Blocking Decisions

Avoid when: the severity classification directly triggers automated blocking or quarantining without human review. Risk: misclassification of a legitimate transaction or event as critical severity causes immediate business disruption. Guardrail: use severity labels for prioritization and routing only. Require explicit human approval before any destructive or blocking action executes.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A copy-ready system prompt for classifying anomaly severity with structured output, inter-rater calibration, and business impact justification.

This prompt template provides a complete set of instructions for an LLM to act as a severity classification engine. It is designed for triage workflows that need consistent, explainable severity labels across different anomaly types. The template forces the model to apply defined criteria, assess business impact, and justify its urgency rating, making the output suitable for downstream automation or human review queues. Before using this prompt, ensure you have defined your severity levels, business impact categories, and the specific anomaly data schema your detection systems produce.

text
You are an anomaly severity classifier for a security and fraud operations triage system. Your task is to analyze an anomaly record and assign a consistent severity classification based on defined criteria.

## INPUT
You will receive an anomaly record in the following JSON format:
[ANOMALY_RECORD]

## CLASSIFICATION CRITERIA
Apply the following severity definitions strictly:
- **CRITICAL**: Active compromise confirmed, financial loss in progress, or core service outage. Requires immediate human intervention.
- **HIGH**: Strong indicators of malicious activity, policy violation, or significant business impact expected within [HIGH_IMPACT_TIMEFRAME]. Requires human review within [HIGH_SLA_TIMEFRAME].
- **MEDIUM**: Anomalous behavior deviating from baseline but no immediate threat confirmed. Potential policy violation or operational inefficiency. Requires review within [MEDIUM_SLA_TIMEFRAME].
- **LOW**: Minor deviation, likely benign, or known false-positive pattern. Log for audit; no immediate action required.
- **INFO**: Informational event, no anomaly detected, or expected system behavior. No action required.

## BUSINESS IMPACT ASSESSMENT
Evaluate the potential business impact across these dimensions:
- **Financial**: Direct monetary loss, fraud exposure, or revenue impact.
- **Operational**: Service degradation, resource availability, or process interruption.
- **Reputational**: Customer trust, public perception, or partner confidence.
- **Compliance**: Regulatory violation, audit failure, or legal exposure.

## OUTPUT SCHEMA
Return a single JSON object with the following structure. Do not include any text outside the JSON object.
{
  "severity": "CRITICAL | HIGH | MEDIUM | LOW | INFO",
  "confidence_score": 0.0-1.0,
  "primary_impact_category": "Financial | Operational | Reputational | Compliance",
  "impact_summary": "A concise, evidence-based summary of the potential business impact.",
  "urgency_justification": "Specific reasons why this severity level was chosen, referencing the criteria above.",
  "recommended_action": "The immediate next step for a human reviewer or automated system.",
  "escalation_required": true/false,
  "inter_rater_calibration_notes": "Any edge cases or ambiguity that could cause another analyst to classify this differently."
}

## CONSTRAINTS
- Do not invent evidence not present in the anomaly record.
- If the anomaly record is incomplete or ambiguous, note this in `inter_rater_calibration_notes` and assign a confidence score below 0.7.
- If the anomaly matches a known false-positive pattern from [KNOWN_FALSE_POSITIVE_PATTERNS], classify as LOW and explain the match.
- Prioritize precision over recall; it is better to escalate a borderline case than to miss a critical event.

## EXAMPLES
[FEW_SHOT_EXAMPLES]

## RISK LEVEL
[RISK_LEVEL]

To adapt this template for your environment, replace the square-bracket placeholders with your operational data. [ANOMALY_RECORD] should be populated with the JSON output from your detection system. [HIGH_IMPACT_TIMEFRAME] and the SLA placeholders must reflect your team's actual response commitments. [KNOWN_FALSE_POSITIVE_PATTERNS] is critical for reducing alert fatigue; provide a list of signatures or conditions that your team has previously determined to be benign. [FEW_SHOT_EXAMPLES] should include 3-5 representative anomaly records with their correct severity classifications, chosen to illustrate boundary cases between severity levels. Set [RISK_LEVEL] to 'high' if this prompt is used in a regulated industry, which will activate additional caution in the model's reasoning. Always use a structured output API or strict JSON mode when calling the model to prevent parsing failures in your triage pipeline.

IMPLEMENTATION TABLE

Prompt Variables

Each placeholder the Anomaly Severity Classification prompt requires, its purpose, an example value, and validation notes for reliable classification.

PlaceholderPurposeExampleValidation Notes

[ANOMALY_DESCRIPTION]

The raw text or structured log describing the detected anomaly to be classified.

A user downloaded 2.3TB of data to an external IP between 02:00-04:00 UTC, which is 15x their daily average.

Must be non-empty. If null, the prompt should return a MISSING_INPUT error instead of guessing severity.

[ANOMALY_SOURCE]

The system or sensor that generated the anomaly alert (e.g., DLP, CASB, SIEM).

DLP_Endpoint_Agent_v3

Must match a known source in the valid_sources registry. Unknown sources should trigger an UNKNOWN_SOURCE warning and default to a lower-confidence classification.

[ASSET_CRITICALITY]

The business criticality rating of the affected asset or user.

HIGH

Must be one of the defined enum values: LOW, MEDIUM, HIGH, CRITICAL. Invalid values should cause a retry with a corrected schema prompt.

[BASELINE_PROFILE]

A summary of the user's or system's normal behavior for comparison.

User typically downloads 150MB/day during business hours from internal IPs.

If empty, the prompt must note that the severity assessment is made without a baseline, increasing uncertainty. The output confidence_score should be capped at 0.6.

[BUSINESS_IMPACT_DEFINITIONS]

A dictionary mapping severity levels to concrete business impact criteria.

{"CRITICAL": "Confirmed data exfiltration or system compromise", "HIGH": "Potential data exfiltration..."}

Must be a valid JSON object. Schema check required. If missing, the prompt cannot calibrate severity and should escalate to a human reviewer with an UNCALIBRATED_POLICY flag.

[TIME_SENSITIVITY]

The operational urgency, often derived from the SLA of the monitoring system.

P1_SLA_15MIN

Must be one of the defined SLA tiers. A null value should default to the lowest tier (P4_STANDARD) to avoid false urgency.

[RELATED_ALERTS]

A list of other recent alerts for the same user or asset to provide context.

[{"alert_id": "ALT-992", "type": "Geo-Impossible Travel", "time": "01:55 UTC"}]

Must be a valid JSON array. An empty array is valid. If the JSON is malformed, the prompt should request a correction rather than silently ignoring the context.

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the anomaly severity classification prompt into a production triage pipeline with validation, retry, logging, and human review gates.

The anomaly severity classification prompt is designed to sit inside a triage pipeline, not as a standalone chat interaction. In production, this prompt receives structured anomaly records from upstream detection systems—such as a log anomaly detector, a transaction monitoring engine, or a user behavior analytics platform—and returns a severity label with impact assessment and urgency justification. The application layer is responsible for assembling the prompt inputs, calling the model, validating the output against the expected schema, and routing the result to either an automated response workflow or a human review queue based on the classified severity.

Start by defining a strict output schema that the application can validate before the classification enters any decision path. The expected output should include at minimum: severity (an enum constrained to your defined levels, e.g., CRITICAL, HIGH, MEDIUM, LOW, INFORMATIONAL), impact_assessment (a short structured object with affected_assets, potential_blast_radius, and data_sensitivity), urgency_justification (a free-text field with explicit reasoning tied to business impact definitions), and confidence (a numeric score between 0.0 and 1.0). After the model responds, run a schema validator. If the output fails validation, retry once with the validation error message appended to the prompt as additional context. If the retry also fails, escalate the raw anomaly record and both failed responses to a human triage queue rather than guessing.

Logging is critical for calibration and inter-rater reliability. Capture the full prompt, the model response, the validation result, the final severity label, and any human override at decision time. Store these in a structured format that allows you to compare model classifications against human reviewer decisions over time. This log becomes your dataset for recalibrating severity definitions, tuning the prompt's few-shot examples, and measuring drift. For high-severity classifications (CRITICAL or HIGH), always route to a human review queue with the full classification payload, the source anomaly record, and a pre-built approval interface. For MEDIUM classifications, consider a staged approach: allow automated response within defined guardrails but require post-action review within a time window. LOW and INFORMATIONAL classifications can proceed automatically but should still be logged for periodic audit sampling.

Model choice matters for consistency. Use a model with strong instruction-following and structured output capabilities. If your latency budget allows, consider running the classification twice—once with a primary model and once with a smaller, faster model—and flagging disagreements for human review. This dual-model pattern catches edge cases where severity boundaries are ambiguous. For regulated environments, ensure that every CRITICAL classification includes a human approval step before any downstream action is taken, and that the approval decision is logged with the reviewer's identity, timestamp, and any override justification. Never allow automated execution on CRITICAL anomalies without a human-in-the-loop gate.

IMPLEMENTATION TABLE

Expected Output Contract

Defines the exact fields, types, and validation rules for the anomaly severity classification output. Use this contract to build a parser, validator, or evaluation harness before deploying the prompt.

Field or ElementType or FormatRequiredValidation Rule

severity_level

enum: CRITICAL | HIGH | MEDIUM | LOW | INFO

Must match one of the defined enum values exactly. Case-sensitive check.

severity_score

number (0.0 - 100.0)

Must be a float within the inclusive range. Score must be consistent with severity_level band (e.g., CRITICAL >= 85.0).

impact_assessment

string (max 500 chars)

Must be non-empty. Should contain a reference to a specific business asset, user segment, or service. Null or whitespace triggers a retry.

urgency_justification

string (max 500 chars)

Must contain a temporal component (e.g., 'within 4 hours'). If missing, escalate for human enrichment.

anomaly_type

string

Must map to a known category in the [ANOMALY_TAXONOMY] list. Unknown types trigger a low-confidence flag.

confidence

number (0.0 - 1.0)

Must be a float. If confidence < [CONFIDENCE_THRESHOLD], the entire output should be routed to a human review queue.

evidence_ids

array of strings

If present, each string must match a UUID pattern from the [EVIDENCE_LOG]. Empty array is allowed but triggers a completeness warning.

inter_rater_flags

array of strings

If present, must only contain values from the [DISAGREEMENT_FLAGS] list. Used to signal boundary cases for calibration review.

PRACTICAL GUARDRAILS

Common Failure Modes

What breaks first when classifying anomaly severity in production and how to guard against it.

01

Severity Inflation Under Uncertainty

What to watch: The model defaults to 'Critical' or 'High' when evidence is ambiguous, causing alert fatigue and wasted reviewer time. Guardrail: Require a confidence score alongside the severity label. Route classifications with confidence below 0.7 to a 'Needs Review' queue with explicit uncertainty flags.

02

Business Impact Misalignment

What to watch: The model applies generic severity logic that contradicts your organization's actual impact definitions (e.g., treating a P4 as a P2 because the pattern looks unusual). Guardrail: Inject your severity definitions directly into the prompt as a [SEVERITY_RUBRIC] variable. Run weekly calibration checks against a golden set of 20 pre-labeled anomalies reviewed by your incident team.

03

Temporal Context Blindness

What to watch: The model classifies an anomaly in isolation without considering that it's part of a larger campaign or a known maintenance window, leading to incorrect severity assignment. Guardrail: Include a [TEMPORAL_CONTEXT] field with recent related anomalies and scheduled events. Add a pre-processing step that fetches the last 24 hours of related alerts before classification.

04

Output Schema Drift

What to watch: The model returns a severity string like 'high' when your downstream system expects 'HIGH', or adds extra commentary that breaks JSON parsing. Guardrail: Use a strict [OUTPUT_SCHEMA] with an enum for severity levels. Implement a post-generation validator that rejects any output not matching the exact schema and triggers a retry with a stronger format constraint.

05

Inter-Rater Reliability Collapse

What to watch: The same anomaly receives different severity labels across multiple runs or when minor prompt wording changes, eroding trust in the automation. Guardrail: Run each classification three times with a temperature of 0. If the severity labels disagree, escalate to a human and log the disagreement for prompt debugging. Track Cohen's Kappa weekly.

06

Evidence Grounding Failure

What to watch: The model fabricates plausible-sounding impact justifications that reference non-existent data or misattribute evidence sources. Guardrail: Require that every justification sentence cites a specific field from the [ANOMALY_PAYLOAD]. Add a fact-checking step that verifies cited values exist in the input before the classification is accepted.

IMPLEMENTATION TABLE

Evaluation Rubric

Run these checks against a golden dataset of 50-100 pre-classified anomalies with known severity labels from multiple human analysts. Use the results to measure inter-rater reliability, identify systematic misclassifications, and set a minimum pass threshold before shipping the prompt.

CriterionPass StandardFailure SignalTest Method

Severity Label Accuracy

Model-assigned severity matches the majority human label for >= 90% of cases

Accuracy falls below 90% or systematic bias toward over/under-classification is detected

Compare model output to golden dataset labels using exact match; compute F1 per severity class

Inter-Rater Reliability (Model vs. Humans)

Cohen's Kappa >= 0.8 between model and each human analyst

Kappa < 0.8 for any analyst, indicating the model disagrees with that analyst more than expected by chance

Calculate pairwise Cohen's Kappa between model labels and each human analyst's labels in the golden dataset

Impact Assessment Completeness

Impact field is non-null and contains a specific business impact statement for 100% of outputs

Null, empty, or generic impact statements such as 'impact unknown' appear in any output

Parse [OUTPUT_SCHEMA] field; assert non-null and minimum character length > 20 for impact statement

Urgency Justification Grounding

Urgency justification references at least one concrete signal from [INPUT] in >= 95% of cases

Justification contains only generic reasoning with no reference to input data in > 5% of outputs

Keyword match check: justification text must contain at least one entity, metric, or timestamp present in the input payload

Confidence Score Calibration

Model-reported confidence score correlates with actual correctness (Brier score < 0.15)

High confidence on incorrect classifications or low confidence on correct classifications

Bin predictions by confidence decile; plot observed accuracy per bin; compute Brier score against binary correct/incorrect outcomes

Output Schema Compliance

100% of outputs parse successfully against [OUTPUT_SCHEMA] without missing required fields

Any JSON parse failure, missing required field, or type mismatch in required field

Validate each output with a JSON Schema validator configured with [OUTPUT_SCHEMA]; count strict failures

Over-Classification Rate on Benign Samples

Severity is 'none' or 'low' for >= 95% of known-benign anomalies in the golden dataset

More than 5% of known-benign samples classified as 'medium', 'high', or 'critical'

Filter golden dataset to samples with human consensus label 'benign'; measure model severity distribution on this subset

Under-Classification Rate on Critical Samples

Severity is 'critical' or 'high' for 100% of known-critical anomalies in the golden dataset

Any known-critical sample classified as 'medium', 'low', or 'none'

Filter golden dataset to samples with human consensus label 'critical'; assert model severity is 'critical' or 'high' for all

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Start with the base severity classification prompt but relax strict schema enforcement. Use plain-text output with labeled sections instead of JSON. Focus on getting the severity logic right before locking down format.

Replace the [OUTPUT_SCHEMA] placeholder with:

code
Return your classification as:
SEVERITY: [Critical/High/Medium/Low/Informational]
JUSTIFICATION: [2-3 sentence reasoning]
IMPACT: [Brief impact statement]

Watch for

  • Inconsistent severity labels across runs
  • Missing impact assessment when anomalies are novel
  • Overly verbose justifications that bury the actual reasoning
  • No inter-rater consistency measurement
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.