Inferensys

Prompt

Triage Priority Scoring Prompt Template

A practical prompt playbook for using the Triage Priority Scoring Prompt Template in production AI workflows to decompose priority into urgency, impact, and risk factors before routing to human review queues.
Developer doing prompt engineering on laptop, prompt variations visible on screen, casual coding session.
PROMPT PLAYBOOK

When to Use This Prompt

Define the job, reader, and constraints for the Triage Priority Scoring Prompt Template.

This prompt is for triage engineers and operations teams who need to score and rank AI-generated work items before they enter a human review queue. The job-to-be-done is consistent, explainable prioritization: turning a stream of review items into an ordered list where the most critical work surfaces first. Use it when your review queue receives more items than reviewers can immediately handle and you need the AI to pre-score each item so humans can work from the top of the queue with confidence. The ideal user is someone who can define what 'urgent,' 'high-impact,' and 'high-risk' mean in their operational context and can provide those definitions as part of the prompt's input variables.

Do not use this prompt when the review queue is small enough for a human to triage everything in a single pass, or when the cost of a mis-prioritized item is catastrophic and no automated scoring should influence queue order. It is also inappropriate when priority depends on real-time external signals—such as a live incident dashboard or a customer's current emotional state—that the model cannot access. This prompt is a static scoring tool, not a dynamic monitoring system. If your triage workflow requires SLA timers, breach predictions, or multi-stakeholder routing, pair this prompt with the sibling playbooks for SLA breach prediction or stakeholder identification rather than overloading the priority score with those concerns.

Before using this prompt, you must define the scoring dimensions (urgency, impact, risk), the scale for each dimension, and the rules for combining them into a final priority score. You must also prepare a calibration set: at least 20–30 items that humans have already prioritized, so you can test whether the prompt's scores align with human judgment. Without calibration data, you cannot detect systematic scoring errors such as consistently underweighting risk or overweighting urgency. Start by running the prompt against your calibration set, compare scores to human-assigned priorities, and adjust the dimension definitions or combination rules until the prompt's output is reliable enough to trust in production.

PRACTICAL GUARDRAILS

Use Case Fit

Where the Triage Priority Scoring Prompt works, where it breaks, and what you must provide before relying on it in a production review queue.

01

Good Fit: Structured Triage Pipelines

Use when: you have a defined set of work items (support tickets, alerts, review tasks) flowing through a queue and need consistent, explainable priority scores. Guardrail: define a fixed priority scale (P0-P4) and calibrate the prompt against at least 50 human-scored examples before production use.

02

Bad Fit: Real-Time Safety Decisions

Avoid when: the priority score directly triggers an irreversible action (shutting down a system, blocking a user, escalating to emergency services) without human review. Guardrail: use this prompt only for queue ordering and reviewer attention; pair with a Human Approval Request Prompt for any action that changes system state.

03

Required Input: Complete Item Context

Risk: scoring quality collapses when the prompt receives only a title or truncated description. Guardrail: require at minimum the full item body, creation timestamp, customer or system impact signals, and any SLA metadata. Validate input completeness before calling the model.

04

Required Input: Calibrated Priority Definitions

Risk: without explicit definitions for each priority level, the model defaults to generic urgency language that drifts across model versions. Guardrail: include concrete, operational definitions for each priority tier (e.g., 'P1: customer-facing outage, revenue impact, requires response within 15 minutes') directly in the prompt template.

05

Operational Risk: Score Drift Over Time

Risk: priority score distributions shift as input patterns change, causing either alert fatigue (too many high-priority items) or missed critical items (everything scored medium). Guardrail: monitor the distribution of assigned scores weekly and recalibrate when any priority tier exceeds 40% of total volume or drops below 2%.

06

Operational Risk: Misclassified Critical Items

Risk: items requiring immediate human attention (security incidents, data loss, regulatory deadlines) are scored as low priority due to missing signals in the input. Guardrail: implement a pre-screening rule that bypasses the scoring prompt entirely for items matching known critical patterns (keywords, source systems, customer segments) and routes them directly to P0/P1 queues.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A reusable prompt for scoring the priority of a review item by decomposing urgency, impact, and risk factors.

This prompt template is the core instruction set for a triage priority scoring system. It is designed to be dropped into an AI harness where the placeholders are populated by application logic before the model call. The prompt forces the model to produce a structured, calibrated priority score by evaluating three independent dimensions—urgency, impact, and risk—and then synthesizing them into a final score with explicit reasoning. This decomposition prevents the model from defaulting to a gut-feeling 'high' or 'low' and instead builds a defensible, auditable trail for why an item landed in a specific queue position.

code
You are a triage priority scoring agent. Your task is to evaluate a work item and produce a structured priority score.

## INPUT
[INPUT]

## CONTEXT
[CONTEXT]

## CONSTRAINTS
- Do not guess missing information. If a factor cannot be determined, mark it as 'UNKNOWN' and explain how it affects the score.
- Apply the [RISK_LEVEL] tolerance when evaluating impact and risk. A low risk tolerance should increase the score for ambiguous threats.
- If the item matches any pattern in [ESCALATION_RULES], bypass scoring and return an immediate escalation flag.

## SCORING INSTRUCTIONS
Evaluate the item across three independent dimensions on a scale of 1 (lowest) to 5 (highest):

1.  **Urgency**: Time-sensitivity. Does the item have a hard deadline, an SLA, or a rapidly decaying value? Consider [SLA_WINDOW] if provided.
2.  **Impact**: The blast radius if the item is not resolved or the action is taken incorrectly. Consider the number of users, systems, or revenue affected.
3.  **Risk**: The probability and severity of a negative outcome, including safety, compliance, or data integrity risks.

Calculate the **Final Priority Score** as a weighted average: `(Urgency * 0.4) + (Impact * 0.4) + (Risk * 0.2)`. Round to one decimal place.

## OUTPUT_SCHEMA
You must respond with a single valid JSON object conforming to this schema:
{
  "item_id": "string",
  "final_score": "number",
  "score_components": {
    "urgency": {
      "score": "number (1-5)",
      "rationale": "string"
    },
    "impact": {
      "score": "number (1-5)",
      "rationale": "string"
    },
    "risk": {
      "score": "number (1-5)",
      "rationale": "string"
    }
  },
  "missing_information": ["string"],
  "escalation_flag": "boolean",
  "escalation_reason": "string | null"
}

## EXAMPLES
[EXAMPLES]

To adapt this template, start by defining your [INPUT] and [CONTEXT] variables. [INPUT] should be the raw text of the work item, while [CONTEXT] can be a pre-assembled string of relevant customer history, system state, or policy documents retrieved via RAG. The [RISK_LEVEL] placeholder is a critical tuning knob; set it to a string like 'LOW', 'MEDIUM', or 'HIGH' based on the operational domain. For a healthcare or finance application, a 'LOW' risk tolerance would instruct the model to score ambiguous threats more aggressively. The [ESCALATION_RULES] placeholder should be a concise list of non-negotiable patterns, such as 'mention of data breach' or 'PII exposure', that force an immediate escalation_flag: true and bypass the scoring logic entirely. Finally, populate [EXAMPLES] with a few-shot set of scored items that demonstrate the desired calibration, especially for edge cases where urgency and risk conflict.

IMPLEMENTATION TABLE

Prompt Variables

Required and optional inputs for the Triage Priority Scoring prompt. Each placeholder must be populated before the prompt is sent. Validation notes describe how to check input quality before execution.

PlaceholderPurposeExampleValidation Notes

[WORK_ITEM]

The full text of the AI-generated work item, ticket, alert, or review task requiring priority scoring.

{"id": "INC-4821", "title": "Payment timeout spike in us-east-1", "body": "Monitoring detected p99 latency > 5s for /checkout endpoint starting 14:22 UTC. Affected 1,200 transactions."}

Must be non-empty string. Reject if null or whitespace-only. Validate that item contains enough semantic content for scoring (minimum 50 characters).

[QUEUE_CONTEXT]

Metadata about the review queue: current depth, average resolution time, SLA windows, and reviewer capacity.

{"queue_depth": 45, "avg_resolution_minutes": 22, "sla_p1_minutes": 15, "sla_p2_minutes": 60, "available_reviewers": 3}

Must be valid JSON object. Required fields: queue_depth (integer >= 0), sla_p1_minutes (integer > 0). Reject if SLA windows are missing or negative. Null allowed if queue is unbound.

[BUSINESS_IMPACT_CONTEXT]

Domain-specific information about what systems, customers, or revenue are affected and their criticality.

{"affected_service": "checkout-api", "tier": "critical-revenue", "peak_transaction_volume_per_minute": 850, "regulatory_impact": false}

Must be valid JSON object. Required fields: affected_service (string), tier (enum: critical-revenue, internal-tool, customer-facing-non-revenue, back-office). Reject if tier is unrecognized or missing.

[HISTORICAL_PRIORITY_EXAMPLES]

Few-shot examples of previously scored items with their assigned priorities and rationale. Used for calibration.

[{"item_summary": "DB replica lag > 30s", "priority": "P1", "rationale": "Revenue-critical read path degraded"}, {"item_summary": "Typo in admin panel label", "priority": "P4", "rationale": "Cosmetic, no user impact"}]

Must be valid JSON array with 3-10 objects. Each object requires item_summary (string), priority (enum: P1-P5), rationale (string). Reject if fewer than 3 examples provided. Validate priority values against allowed enum.

[SCORING_DIMENSIONS]

The dimensions to score on and their weights. Defines the output schema for the decomposition.

["urgency", "impact", "risk"]

Must be valid JSON array of strings. Minimum 2 dimensions. Reject if dimensions contain duplicates. Common dimensions: urgency, impact, risk, effort, customer_visibility. Weights are defined in the prompt system instruction, not here.

[OUTPUT_SCHEMA]

The expected JSON structure for the priority score output, including field types and required fields.

{"priority_score": "number", "priority_label": "string", "urgency_score": "number", "impact_score": "number", "risk_score": "number", "rationale_summary": "string"}

Must be valid JSON schema object. Required fields: priority_score (number 0.0-1.0), priority_label (enum: P1-P5), rationale_summary (string). Reject if schema allows priority_score outside 0-1 range. Validate that all scoring dimensions from [SCORING_DIMENSIONS] have corresponding _score fields.

[CONSTRAINTS]

Hard constraints on scoring behavior: maximum scores for certain conditions, mandatory escalation triggers, and forbidden overrides.

{"max_score_without_human_impact": 0.5, "mandatory_escalation_triggers": ["regulatory_impact", "data_loss_risk"], "forbidden_overrides": ["downgrade_p1_without_approval"]}

Must be valid JSON object. Validate that mandatory_escalation_triggers is array of strings matching known trigger types. Reject if forbidden_overrides contains empty entries. Null allowed if no constraints beyond defaults.

[CALIBRATION_REFERENCE]

A reference point for score calibration: expected distribution of priorities or a target threshold for P1 classification.

{"expected_p1_rate": 0.05, "expected_p2_rate": 0.15, "calibration_tolerance": 0.02}

Must be valid JSON object. Required fields: expected_p1_rate (number 0.0-1.0). Reject if rates sum to > 1.0. Null allowed if calibration is done post-hoc rather than in-prompt. Validate that tolerance is positive and <= 0.1.

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the Triage Priority Scoring prompt into a production review queue or escalation workflow.

This prompt is designed to be called as a deterministic scoring step within a larger triage pipeline, not as a standalone chat interaction. The primary integration point is immediately after a new work item is ingested and before it is routed to a human queue. The application should construct the [INPUT] by serializing the item's title, description, and any available metadata (e.g., affected system, customer tier, SLA deadline) into a structured text block. The [CONTEXT] placeholder must be populated with your organization's specific priority definitions, including what constitutes a P1/P2/P3/P4 incident, your business hours, and any explicit risk-weighting policies. Do not rely on the model's general knowledge of priority; inject your runbook definitions directly into the context to ensure calibration against your operational reality.

The model's response must be validated against a strict JSON schema before the score is trusted. The application layer should parse the output and confirm the presence of integer fields for urgency_score, impact_score, risk_score, and the final composite_priority_score. A retry loop with a maximum of two attempts is recommended: if the initial output fails schema validation, the retry prompt should include the raw response and the specific validation error (e.g., 'missing required field: risk_score'). For high-risk environments, implement a circuit breaker that routes the item to a 'Review Required' queue with a null score if validation fails after retries, ensuring no item is dropped. Log the raw prompt, the model's response, the validated score, and the chosen model version to your observability platform for later calibration analysis.

Model choice involves a trade-off between latency and reasoning depth. For high-volume, low-latency queues, a fast model like GPT-4o-mini or Claude Haiku is often sufficient if the priority definitions are unambiguous. For complex items where impact and risk are deeply intertwined, a stronger reasoning model like GPT-4o or Claude Sonnet may produce more calibrated scores, especially for the risk_factor_decomposition field. To prevent score drift, run a weekly eval job that samples 100 scored items and compares the model-assigned composite_priority_score against a human-assigned priority label. Monitor the distribution of scores; a sudden spike in P1 assignments often indicates a prompt regression or a change in the upstream data source, not a real-world crisis. The next step after scoring is to pass the validated JSON payload to your queue router, using the composite_priority_score to determine the target queue and SLA timer.

IMPLEMENTATION TABLE

Expected Output Contract

Defines the fields, types, and validation rules for the structured JSON object produced by the Triage Priority Scoring Prompt. Use this contract to parse and validate the model output before routing to a review queue.

Field or ElementType or FormatRequiredValidation Rule

priority_score

integer (1-100)

Must be an integer between 1 and 100 inclusive. Reject non-integer or out-of-range values.

urgency_score

integer (1-100)

Must be an integer between 1 and 100 inclusive. Must not exceed priority_score by more than 30 points without a non-null urgency_justification.

impact_score

integer (1-100)

Must be an integer between 1 and 100 inclusive. If impact_score > 80, the critical_impact_flag must be true.

risk_score

integer (1-100)

Must be an integer between 1 and 100 inclusive. If risk_score > 80, the escalation_required flag must be true.

critical_impact_flag

boolean

Must be true if impact_score > 80. Reject if true but impact_score <= 80.

escalation_required

boolean

Must be true if risk_score > 80. Reject if true but risk_score <= 80.

scoring_rationale

string (<= 300 chars)

Must be a non-empty string. Reject if length exceeds 300 characters. Must contain a reference to at least one factor from [INPUT].

urgency_justification

string or null

Must be null unless urgency_score exceeds impact_score by more than 30 points. If not null, must be a non-empty string explaining the discrepancy.

PRACTICAL GUARDRAILS

Common Failure Modes

Priority scoring prompts fail in predictable ways that can bury critical items or flood reviewers with false alarms. These are the most common failure modes and the guardrails that catch them before they reach production.

01

Score Inflation Under Uncertainty

What to watch: When the model lacks sufficient context to assess impact or urgency, it often defaults to a high score rather than flagging the gap. This buries truly critical items in a sea of false-high priorities. Guardrail: Require the prompt to output a confidence field per factor. If confidence is low, cap the maximum priority score and route to a calibration queue instead of the main triage pipeline.

02

Critical Item Misclassification as Routine

What to watch: Items containing subtle severity signals—such as compliance implications, data loss indicators, or security keywords buried in long descriptions—are scored as routine because the model fixates on surface-level urgency cues. Guardrail: Add a pre-scoring classification pass that checks for high-severity keywords and regulatory triggers. If triggered, bypass the standard scoring prompt and route directly to a human with the trigger evidence highlighted.

03

Factor Weighting Drift Across Batches

What to watch: The model inconsistently weights urgency, impact, and risk across different items in the same queue, producing scores that cannot be compared. One item gets a 9 for moderate impact while another gets a 6 for similar severity. Guardrail: Include explicit weighting instructions in the prompt and validate score consistency by running a small calibration set with known priorities before each batch. Flag batches where correlation with human-assigned priorities drops below a threshold.

04

Overfitting to Recent Examples in Context

What to watch: When few-shot examples are included in the prompt, the model anchors too heavily on the example scores and replicates their distribution even when new items differ significantly. Guardrail: Use examples only for format demonstration, not score calibration. Explicitly instruct the model that example scores are illustrative and should not influence scoring of new items. Validate with a holdout set that differs from the examples.

05

Silent Failure on Ambiguous or Contradictory Input

What to watch: When the input contains conflicting signals—such as a high-urgency label paired with a low-impact description—the model picks one signal and ignores the contradiction rather than surfacing the ambiguity. Guardrail: Add an explicit contradiction-detection step in the prompt. If urgency and impact signals conflict, the output must include an ambiguity_flag set to true and the item should be routed to a human clarification queue before scoring.

06

Score Compression at the Top of the Scale

What to watch: Most items cluster in the 7-9 range, making it impossible to distinguish between genuinely critical items and merely important ones. Reviewers learn to ignore the scores entirely. Guardrail: Enforce a distribution constraint in the prompt requiring that no more than a specified percentage of items receive the highest priority tier. Validate post-scoring and re-score with adjusted instructions if compression is detected.

IMPLEMENTATION TABLE

Evaluation Rubric

Criteria for testing whether the Triage Priority Scoring Prompt produces calibrated, safe, and actionable priority scores before deployment.

CriterionPass StandardFailure SignalTest Method

Score calibration against human-assigned priorities

Model-assigned priority score is within ±1 point of the median human-assigned score on a 1-10 scale for 90% of test cases

Systematic over-scoring of low-urgency items or under-scoring of critical items by more than 2 points

Run 50 labeled triage items through the prompt. Compare model scores to human ground-truth scores using mean absolute error and correlation coefficient

Critical item misclassification detection

Zero false negatives: all items with human-assigned priority 9-10 receive model scores of 8 or higher

Any item with human-assigned priority 9-10 receives a model score of 7 or below

Curate a golden set of 20 known critical items. Verify all pass the threshold. Flag any misses for prompt revision

Urgency factor decomposition accuracy

Urgency sub-score correctly reflects time-sensitivity for 95% of items with explicit deadline or SLA context

Urgency sub-score is high when no deadline exists, or low when SLA breach is imminent within 1 hour

Test 30 items with known SLA windows. Check that urgency sub-score correlates with time-remaining. Spot-check 5 edge cases manually

Impact factor decomposition accuracy

Impact sub-score correctly reflects affected user count, data scope, or revenue exposure for 90% of items

Impact sub-score is identical for a single-user cosmetic bug and a platform-wide outage

Create 15 paired examples differing only in impact scope. Verify impact sub-score direction and magnitude match expected ordering

Risk factor decomposition accuracy

Risk sub-score correctly identifies safety, compliance, or data-loss risk for 90% of items with known risk labels

Risk sub-score is zero or near-zero for items tagged with regulatory obligation or data-deletion action

Test 25 items with pre-labeled risk categories. Check that risk sub-score is non-zero when risk labels are present. Measure false-negative rate

Output schema compliance

100% of outputs parse as valid JSON matching the expected schema with all required fields present and correctly typed

Missing priority_score field, non-numeric sub-scores, or extra untyped fields that break downstream consumers

Run 100 diverse triage items through the prompt. Validate output with a JSON schema validator. Count parse failures and schema violations

Confidence annotation when evidence is thin

Model outputs a confidence indicator below 0.7 when fewer than 3 evidence fields are populated or source reliability is flagged

High confidence scores on items with empty evidence arrays or explicit uncertainty markers in the input

Feed 15 items with deliberately sparse evidence. Verify confidence scores drop below threshold. Check for overconfident outputs

Escalation flag for ambiguous or edge cases

Model sets an escalation flag to true when priority score is between 5-7 and confidence is below 0.6

No escalation flag on items where two of three sub-scores are null or where input contains contradictory severity signals

Test 10 borderline items with mixed signals. Verify escalation flag is true. Check for false negatives where escalation should have triggered

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Start with the base scoring prompt and a simple JSON schema for priority_score, urgency, impact, and risk. Use a single LLM call without retries. Accept raw string outputs and parse best-effort.

code
You are a triage assistant. Score the following item from 1 (low) to 5 (critical) for urgency, impact, and risk. Return JSON:
{
  "priority_score": <1-5>,
  "urgency": <1-5>,
  "impact": <1-5>,
  "risk": <1-5>,
  "rationale": "<one sentence>"
}

Item: [ITEM_DESCRIPTION]
Context: [QUEUE_CONTEXT]

Watch for

  • Missing schema validation causing parse failures downstream
  • Overly broad instructions producing scores without decomposition reasoning
  • No calibration against human-assigned priorities
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.