Unanswered Question Severity Scoring Prompt

This prompt is a triage engine for production AI systems that manage multi-turn conversations. Its job is to take a list of previously detected, unanswered user questions and rank them by severity so that an agent planning loop or a human review queue can address the most critical item first. The ideal user is an AI engineer or product developer building a copilot, support agent, or task-oriented assistant where dropped questions are a known failure mode. You should use this prompt when you already have a structured list of pending questions—typically the output of a detection step like the Pending Question Detection Prompt Template—and you need to decide which one to action next.

The prompt evaluates each question across four concrete dimensions: blocking status (does this question prevent the user from completing their primary goal?), time sensitivity (is there an explicit or implied deadline?), user sentiment signals (does the user's language indicate frustration or urgency?), and downstream dependency impact (will answering this question unblock other pending tasks?). It produces a ranked list with severity scores and a recommended next action, making it suitable for injection into an agent's planning context or for sorting a human review queue. Do not use this prompt as a question detector. It assumes the input list is already accurate and will not catch false negatives from an upstream detection failure. Pair it with a re-surfacing or repair prompt, such as the Conversation Repair Prompt for Forgotten Questions, to close the loop after prioritization.

Before deploying, you must define what 'high severity' means for your specific product. A blocking question in a real-time support chat has different urgency than one in an asynchronous email assistant. Calibrate the scoring dimensions with a golden dataset of conversations where domain experts have ranked the questions. Run the prompt against this dataset and measure ranking alignment using a metric like Normalized Discounted Cumulative Gain (NDCG). If the prompt's rankings consistently diverge from expert judgment on a particular dimension, add a [CONSTRAINTS] section with domain-specific rules. For high-stakes domains like healthcare or finance, always route high-severity items to a human review queue and log the prompt's output as audit evidence for why that item was prioritized.

This template is the core instruction you send to the model. It expects a list of unanswered questions and supporting conversation context, then returns a severity-ranked list with scores and justifications. Replace every square-bracket placeholder before calling the model. The prompt is designed to be self-contained: it defines the scoring dimensions, output schema, and constraints in one block so you can drop it into an existing agent planning or human-review pipeline without additional system instructions.

textCopy
You are a conversation triage analyst. Your task is to score each unanswered question in the provided list for severity and produce a ranked output.

## INPUT
- Unanswered Questions: [UNANSWERED_QUESTIONS_LIST]
- Conversation Context: [CONVERSATION_CONTEXT]
- User Sentiment Signals: [USER_SENTIMENT_SIGNALS]
- Downstream Dependencies: [DOWNSTREAM_DEPENDENCIES]

## SCORING DIMENSIONS
For each question, assign a score from 1 (lowest) to 5 (highest) on each dimension:

1. **Blocking Status**: Does this question block the user from completing their primary goal? (5 = completely blocked, 1 = purely informational)
2. **Time Sensitivity**: How urgently does this need an answer? Consider explicit deadlines, implied urgency from context, and conversation pacing. (5 = immediate deadline or active waiting, 1 = no time pressure)
3. **User Sentiment Impact**: How much frustration, anxiety, or escalation risk is associated with this question remaining unanswered? (5 = visible frustration or explicit escalation language, 1 = neutral or casual curiosity)
4. **Downstream Dependency Impact**: How many other tasks, decisions, or workflows depend on this answer? (5 = multiple dependent workflows blocked, 1 = isolated question with no dependencies)

## OUTPUT SCHEMA
Return a JSON object with this exact structure:
{
  "ranked_questions": [
    {
      "rank": 1,
      "question_id": "string",
      "question_text": "string",
      "severity_scores": {
        "blocking_status": 1-5,
        "time_sensitivity": 1-5,
        "user_sentiment_impact": 1-5,
        "downstream_dependency_impact": 1-5
      },
      "composite_severity": 1-5,
      "justification": "string explaining the composite score with evidence from the conversation",
      "recommended_action": "answer_immediately | escalate_to_human | schedule_followup | defer"
    }
  ],
  "triage_summary": "string summarizing the overall severity landscape",
  "escalation_candidates": ["question_id"]
}

## COMPOSITE SEVERITY CALCULATION
composite_severity = max(blocking_status, time_sensitivity, user_sentiment_impact, downstream_dependency_impact)
Rationale: The most critical dimension should drive priority. Do not average.

## CONSTRAINTS
- Score every question in the input list. Do not skip any.
- If a dimension cannot be determined from context, score it as 1 and note "insufficient context" in the justification.
- Do not invent facts, deadlines, or user emotions not present in the input.
- Rank by composite_severity descending. Break ties by blocking_status, then time_sensitivity.
- recommended_action must be one of the four enumerated values.
- Only include question_ids in escalation_candidates if composite_severity >= 4 AND user_sentiment_impact >= 4.

Adaptation guidance: Replace [UNANSWERED_QUESTIONS_LIST] with a structured array of question objects, each containing at minimum an id and text field. [CONVERSATION_CONTEXT] should include the last N turns or a session summary. [USER_SENTIMENT_SIGNALS] can be extracted from a prior sentiment detection pass or left as an empty string if unavailable. [DOWNSTREAM_DEPENDENCIES] should list known dependent workflows or tasks; if your system doesn't track dependencies, replace with "No dependency data available." The composite severity formula uses max rather than average to prevent a question with one critical dimension from being buried. If your triage logic requires a weighted formula instead, modify the calculation rule and document the change in your prompt version history.

This prompt is designed to be called by a state management middleware layer, not directly by the end user. The middleware should maintain a structured list of unresolved questions (detected by a companion prompt like Pending Question Detection) and pass them into this scoring prompt as a batch. The output is a ranked list that downstream systems consume: an agent planner can inject the top-N blocked items into its next planning step, a support dashboard can highlight high-severity items for human agents, or a notification system can escalate time-sensitive questions that have exceeded their SLA. The prompt expects a JSON array of question objects as [UNRESOLVED_QUESTIONS] and a conversation summary or recent turn context as [CONVERSATION_CONTEXT] to ground severity judgments in actual user sentiment and workflow state.

Wire the prompt into an application pipeline with these concrete steps: (1) Input assembly: Collect unresolved questions from your state tracker and serialize them with fields id, question_text, turn_number, age_in_turns, and any existing tags. Include the last N turns of conversation context so the model can detect sentiment signals and blocking dependencies. (2) Model selection: Use a model with strong reasoning and structured output support (GPT-4o, Claude 3.5 Sonnet, or equivalent). This task requires cross-item comparison and multi-factor judgment, not simple classification. (3) Schema enforcement: Require the output to match the [OUTPUT_SCHEMA] defined in the prompt template—each scored item must include question_id, severity_score (1-5), blocking_status (boolean), time_sensitivity (low/medium/high/critical), sentiment_signal (neutral/concerned/frustrated/urgent), dependency_impact (none/low/medium/high), and a one-sentence rationale. Use structured output mode or a JSON schema validator post-generation. (4) Validation layer: After receiving the model output, validate that every input question ID appears in the output, severity scores are within range, and the ranked order is consistent with individual scores. Reject and retry if IDs are missing or scores are malformed. (5) Retry logic: On validation failure, re-invoke the prompt with the same input plus an [ERROR_CONTEXT] field describing the validation failure. Limit to 2 retries before falling back to a default FIFO ordering with a warning log. (6) Logging: Log the input question count, output ranking, validation status, retry count, and model latency. This data feeds prompt regression tests and cost monitoring.

For high-stakes workflows—such as healthcare triage, financial support, or legal intake—add a human review gate before the ranked list drives automated actions. Route items scored 4 or above to a review queue where a human confirms or adjusts the severity before the item blocks a workflow or triggers an escalation. For lower-stakes deployments, you can use the ranked list directly but should still monitor for severity inflation (the model scoring everything as high) or severity blindness (missing genuinely blocking items). Run periodic eval passes using a golden dataset of conversations with known dropped questions and expert-labeled severity scores. Compare model rankings against human rankings using NDCG or Kendall's tau. If the model consistently underperforms on time-sensitivity detection, consider augmenting the prompt with explicit SLA rules or adding a pre-processing step that computes age-in-minutes from timestamps rather than relying on the model to infer urgency from turn counts alone.