Correction vs. New Intent Discrimination Prompt

This prompt is for dialogue state managers that must decide whether a user's latest message is a correction to the assistant's previous turn or an entirely new, unrelated request. The distinction is critical: misclassifying a correction as a new intent causes the system to ignore user feedback and repeat errors. Misclassifying a new intent as a correction pollutes the active dialogue state with irrelevant repair logic. Use this prompt inside a stateful conversation loop, immediately after receiving user input and before updating the dialogue state or calling downstream tools. It assumes you have access to the current user message, the assistant's last response, and a structured summary of the active task or topic.

Required inputs are non-negotiable for reliable discrimination. You must provide the [USER_MESSAGE] (the raw latest turn), the [ASSISTANT_LAST_RESPONSE] (the full text of the immediately preceding assistant turn), and the [ACTIVE_DIALOGUE_STATE] (a structured object containing the current intent, filled slots, and pending actions). Without the assistant's last response, the model cannot anchor what the user might be correcting. Without the active dialogue state, it cannot determine if the new message is a continuation. Do not use this prompt for single-turn stateless interactions, for classifying the first user message in a session, or when the assistant's last turn was a simple acknowledgment with no substantive content to correct. In those cases, route directly to intent classification.

Production placement matters. Wire this prompt as a synchronous gate after user input ingestion and before any tool calls, database writes, or state mutations. If the classifier returns correction, your system must update the dialogue state to reverse or amend the prior turn's effects before proceeding. If it returns new_intent, archive or summarize the prior task and initialize a fresh state. Avoid using this prompt on every turn unconditionally—if the user's message is clearly a simple confirmation ('yes', 'ok', 'thanks'), skip classification to save latency and cost. Implement a lightweight pre-filter: if the user message contains explicit correction markers ('no,', 'wrong,', 'actually,', 'I meant') or topic-shift signals ('new topic:', 'instead,', 'unrelated:'), route to this prompt. Otherwise, assume continuation and bypass.

The primary failure mode is false negatives on corrections. When a user corrects the assistant without using explicit negation language—for example, by restating their request with slightly different parameters—the model may classify it as a new intent. This causes the system to abandon the original task and start over, frustrating the user who expected a repair. Mitigate this by including few-shot examples in [EXAMPLES] that show implicit corrections (e.g., 'Can you do that for March instead?' following a February report). The secondary failure mode is false positives on topic shifts, where a genuinely new request is treated as a correction because it shares surface-level vocabulary with the prior turn. Mitigate this by requiring the model to output explicit [EVIDENCE] spans from both the user message and the assistant's last response that support its classification. Log this evidence for offline review and eval dataset construction.

This prompt template is the core of the Correction vs. New Intent Discrimination workflow. It is designed to be dropped into a stateful dialogue manager where the model has access to the recent conversation history. The prompt forces the model to ground its classification in specific evidence from the turn history, preventing lazy or assumption-driven labels. The output is a structured JSON object that your application can parse to decide whether to update an existing task, reverse a prior action, or start a fresh workflow.

textCopy
SYSTEM:
You are a dialogue state classifier. Your job is to analyze the user's latest message against the provided conversation history and determine if it is a correction to a prior assistant action or a new, unrelated intent.

INPUT:
[CONVERSATION_HISTORY]
[LATEST_USER_MESSAGE]

CLASSIFICATION_RULES:
1. A "correction" means the user is rejecting, modifying, or undoing a specific prior assistant output or action. The user's message must reference a prior turn explicitly or implicitly.
2. A "new_intent" means the user is starting a new topic, asking a new question, or making a request that does not depend on fixing a prior assistant mistake.
3. If the user's message could be interpreted either way, default to "new_intent" unless there is strong evidence of a correction.

OUTPUT_SCHEMA:
{
  "classification": "correction" | "new_intent",
  "confidence": 0.0-1.0,
  "evidence": [
    {
      "turn_id": "string",
      "quote": "string",
      "reasoning": "string"
    }
  ],
  "corrected_turn_id": "string | null",
  "new_topic_summary": "string | null"
}

CONSTRAINTS:
- Always cite specific turns from [CONVERSATION_HISTORY] as evidence.
- If classification is "correction", `corrected_turn_id` must be populated with the ID of the turn being corrected.
- If classification is "new_intent", `new_topic_summary` must be a concise phrase describing the new request.
- Do not invent turns not present in the history.
- If the history is empty, classification must be "new_intent".

To adapt this template, replace [CONVERSATION_HISTORY] with a serialized list of prior turns, each with a unique turn_id, the user's message, and the assistant's response. The [LATEST_USER_MESSAGE] is the current user input you are classifying. For high-stakes applications, such as financial transactions or healthcare orders, add a [RISK_LEVEL] placeholder that triggers a human review step when confidence is below a threshold. The output schema is designed for direct parsing; validate that corrected_turn_id references a real turn before acting on it. Next, wire this prompt into a harness that logs every classification, compares it against a golden dataset of boundary cases, and escalates low-confidence predictions for manual review.

This prompt is a classification step, not a user-facing response. It must be called before the dialogue manager updates its state. The harness should intercept the user's latest turn and the last N turns of conversation history, inject them into the prompt's [CONVERSATION_HISTORY] and [LATEST_USER_MESSAGE] placeholders, and parse the structured output to decide whether to update the existing intent stack or push a new one. The model choice should favor low latency and high instruction-following reliability; a fast model like claude-3-haiku or gpt-4o-mini is appropriate because this is a binary classification with evidence extraction, not a generation task.

The output must be validated against a strict schema before any state change occurs. The expected JSON object contains a classification field (enum: correction, new_intent), an evidence array of strings quoting the relevant prior turns, and a confidence float between 0.0 and 1.0. Implement a post-processing validator that rejects any response where classification is missing, evidence is empty for a correction label, or confidence is below a configurable threshold (start at 0.7). On validation failure, log the raw response and fall back to a safe default: treat the turn as a new_intent to avoid corrupting existing state, but flag it for human review if the session is high-stakes. Retry once with a simplified prompt that asks only for the classification and the single strongest piece of evidence.

Logging is critical for tuning the threshold and catching drift. Record the user input, the truncated history, the model's raw classification, the validated output, and the final state transition decision. This trace allows you to build an evaluation dataset of boundary cases—such as a user saying 'actually, also check the billing issue' which blends correction and new intent—and measure whether your threshold and prompt are calibrated correctly. Do not use this prompt in isolation; pair it with a downstream slot-filling or intent-routing prompt that consumes the corrected or new intent, and ensure that a correction classification triggers a state reversal on the specific slot or claim the user rejected.