Inferensys

Prompt

Slot-Filling Coreference Prompt for Task-Oriented Dialogue

A practical prompt playbook for resolving user references to previously mentioned slot values and producing an updated, validated dialogue state in production AI workflows.
Developer doing prompt engineering on laptop, prompt variations visible on screen, casual coding session.
PROMPT PLAYBOOK

When to Use This Prompt

Determines when a slot-filling coreference prompt is the right tool versus when a simpler or more complex approach is required.

This prompt is for task-oriented assistant developers who need to resolve user references to previously mentioned slot values and produce an updated dialogue state. Use it when your system tracks structured slots—such as departure city, destination, date, or product SKU—and users refer back to those values with phrases like 'change that to Friday' or 'make it the earlier one.' The prompt resolves the reference, fills the correct slot, and validates type consistency to prevent cross-slot contamination. It is designed for systems where the set of slots is known in advance, the dialogue state is explicitly represented as a structured object, and the primary challenge is mapping anaphoric or elliptical user utterances back to the correct slot-value pairs.

Do not use this prompt for open-ended chat without a defined slot schema, or when the user's reference is entirely new information rather than a reference to an existing slot value. If your system does not maintain a structured dialogue state, you should first implement a state tracking mechanism before attempting coreference resolution. Similarly, if the user introduces a new entity that has never been mentioned, this prompt is not the right tool—use a slot extraction or entity recognition prompt instead. Avoid this prompt when the dialogue history is too long for the model's context window without summarization, as the resolver may hallucinate referents from truncated history. For high-stakes domains such as healthcare scheduling or financial transactions, always pair this prompt with a human-approval step before committing the resolved state to a system of record.

Before deploying this prompt, ensure you have a well-defined slot schema with type constraints, a reliable mechanism for passing the last N turns of dialogue history, and a validation layer that checks the resolved output against your schema. The prompt works best when combined with a regression test suite containing examples of cross-slot contamination, number disagreement, and temporal reference ambiguity. If your use case involves resolving references to entities in retrieved documents rather than dialogue history, use the Reference Resolution with RAG Context Prompt Template instead. For voice or ASR transcripts with disfluent references, prefer the Voice Transcript Reference Resolution Prompt, which includes robustness handling for false starts and recognition errors.

PRACTICAL GUARDRAILS

Use Case Fit

Where the Slot-Filling Coreference Prompt works well and where it introduces risk. Use these cards to decide if this prompt fits your production architecture.

01

Good Fit: Structured Task Dialogs

Use when: your assistant maintains a known slot schema (e.g., origin, destination, date) and users refer back to previously mentioned values. Guardrail: Define the slot schema explicitly in the prompt and validate output against it before merging into dialogue state.

02

Bad Fit: Open-Ended Chat

Avoid when: the conversation has no fixed slot schema or users jump between unrelated topics. Coreference resolution without a target schema produces hallucinated slots. Guardrail: Gate this prompt behind a task-oriented intent classifier; fall back to a general reference resolver for non-task turns.

03

Required Inputs

Must provide: the current user utterance, the prior dialogue turn (or summary), the defined slot schema with types, and the current slot values. Guardrail: If any required input is missing or stale, return an empty patch rather than guessing. Log missing-input events for monitoring.

04

Operational Risk: Cross-Slot Contamination

What to watch: the model copies a value from one slot into another (e.g., a date into a location field) because the reference was ambiguous. Guardrail: Run a post-resolution validator that checks each resolved value against its slot type and rejects type-mismatched assignments.

05

Operational Risk: Silent Null Resolution

What to watch: the user's reference has no clear antecedent, but the model fills a slot anyway rather than leaving it unresolved. Guardrail: Require the prompt to output a confidence score per slot. Escalate slots below threshold to a clarification prompt instead of merging them into state.

06

Scale Risk: Multi-Entity Confusion

What to watch: when multiple entities of the same type exist in the dialogue (e.g., two dates), the model resolves to the wrong one. Guardrail: Include unique entity IDs in the dialogue state and require the prompt to reference them explicitly. Test with regression cases that swap similar entities.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A production-ready prompt template for resolving coreferences in task-oriented dialogue and updating a structured slot schema without cross-slot contamination.

This prompt template is designed to be wired directly into your dialogue management pipeline. Its job is to take the latest user utterance, the current dialogue state, and your application's slot schema, then return a precise, validated update. The model is instructed to resolve any anaphoric or implicit references (e.g., 'change it to Tuesday') against the dialogue history before writing to the correct slot. This prevents the most common production failure mode in slot-filling: updating the wrong field or corrupting the state with an unresolved value.

text
SYSTEM:
You are a state-updating module for a task-oriented dialogue system. Your only job is to resolve references in the latest user utterance and update the dialogue state.

You will receive:
1. A SLOT SCHEMA defining valid slots, their types, and constraints.
2. The CURRENT DIALOGUE STATE with previously filled slots.
3. The LAST ASSISTANT UTTERANCE for context.
4. The LATEST USER UTTERANCE to process.

Your task:
1. Identify if the user's utterance contains a reference to a slot value (explicitly or via anaphora/ellipsis).
2. Resolve the reference using the dialogue history. If the reference is ambiguous, do not guess; set the value to null and set a clarification flag.
3. Determine which single slot in the SLOT SCHEMA should be updated.
4. Validate that the resolved value matches the slot's type and constraints.
5. Output a strict JSON patch object. Do not output any other slot values.

--- SLOT SCHEMA ---
[SLOT_SCHEMA]

--- CURRENT DIALOGUE STATE ---
[CURRENT_STATE]

--- LAST ASSISTANT UTTERANCE ---
[ASSISTANT_UTTERANCE]

--- LATEST USER UTTERANCE ---
[USER_UTTERANCE]

--- OUTPUT INSTRUCTIONS ---
Return a single JSON object with these exact keys:
{
  "slot_to_update": "string (the exact slot name from the schema)",
  "resolved_value": "<value> | null",
  "requires_clarification": boolean,
  "clarification_question": "string | null",
  "reference_resolution_notes": "string (brief explanation of how the reference was resolved)"
}

Do not wrap the JSON in markdown. Do not include any text outside the JSON object.

To adapt this template, replace the square-bracket placeholders with your application's live data. [SLOT_SCHEMA] should be a JSON object defining your slots, their types (e.g., date, string, enum), and any constraints (e.g., "date": "future_only"). [CURRENT_STATE] is the current key-value map of filled slots. [ASSISTANT_UTTERANCE] and [USER_UTTERANCE] are the raw strings from the last two turns. The output contract is intentionally narrow: a single slot update. This prevents the model from hallucinating changes to other fields. If you need to update multiple slots from a single utterance, you should either chain multiple calls to this prompt or use a more complex agentic loop, but be aware that multi-slot updates significantly increase the risk of state corruption.

Before deploying, you must build a thin validation layer around this prompt's output. First, confirm that slot_to_update is a valid key in your [SLOT_SCHEMA]. Second, validate that resolved_value conforms to the type and constraints defined for that slot. If validation fails, do not merge the patch into your dialogue state. Instead, log the failure and either retry with a more explicit error message or escalate to a human agent. For high-stakes domains like healthcare scheduling or financial transactions, always set a requires_clarification threshold and route any ambiguous resolution to a human review queue before state mutation occurs.

IMPLEMENTATION TABLE

Prompt Variables

Inputs the prompt needs to work reliably. Each variable must be populated by your application before sending the request.

PlaceholderPurposeExampleValidation Notes

[CURRENT_USER_UTTERANCE]

The latest user turn containing a reference to resolve

Book me a table at the Italian place I mentioned earlier

Must be a non-empty string. Check for null or whitespace before sending.

[DIALOGUE_HISTORY]

Prior conversation turns as structured JSON with speaker roles, timestamps, and turn IDs

[{"turn_id": 1, "speaker": "user", "text": "I need a restaurant in Soho."}, {"turn_id": 2, "speaker": "assistant", "text": "Any cuisine preference?"}]

Must be a valid JSON array. Validate schema: each object requires turn_id, speaker, text. Max 20 turns to control context budget.

[ACTIVE_SLOTS]

Current dialogue state with slot names, values, and last-updated turn IDs

{"cuisine": {"value": "Italian", "turn_id": 3}, "location": {"value": null, "turn_id": null}}

Must be a valid JSON object. Each slot requires value and turn_id fields. Null values indicate unfilled slots.

[SLOT_SCHEMA]

Schema defining allowed slots, their types, and constraints

[{"slot_name": "cuisine", "type": "enum", "allowed_values": ["Italian", "Mexican", "Japanese", "Indian"]}, {"slot_name": "location", "type": "string"}]

Must be a valid JSON array. Each entry requires slot_name and type. Enum types require allowed_values array.

[RESOLUTION_RULES]

Rules governing how references map to slots and when clarification is required

{"max_candidates_for_auto_resolve": 1, "require_confidence_threshold": 0.85, "disallowed_cross_slot_types": ["cuisine->location", "date->time"]}

Must be a valid JSON object. Confidence threshold must be between 0.0 and 1.0. Disallowed cross-slot types must reference valid slot names from SLOT_SCHEMA.

[OUTPUT_SCHEMA]

Expected JSON structure for the resolved output

{"resolved_slots": {}, "unresolved_references": [], "clarification_question": null, "confidence_scores": {}}

Must be a valid JSON schema object. Validate that resolved_slots keys match SLOT_SCHEMA slot names. Unresolved_references must be an array.

[CONFIDENCE_THRESHOLD]

Minimum confidence score for automatic resolution without clarification

0.85

Must be a float between 0.0 and 1.0. Values below 0.7 produce excessive clarification; values above 0.95 risk silent errors. Default 0.85.

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the slot-filling coreference prompt into a production pipeline with validation, retry, and human escalation.

This prompt is designed to sit inside a stateful dialogue manager, not as a standalone call. The typical integration point is after each user turn, before the downstream NLU or action dispatcher runs. The application should maintain a structured dialogue state object (slots with types, values, and last-updated timestamps) and pass it alongside the current user utterance and the last N turns of conversation history. The prompt returns a patch object—only the slots that need updating—which the application merges into the existing state. This design keeps the prompt focused and avoids re-transmitting the full state on every call, which reduces token cost and prevents the model from accidentally overwriting slots it should not touch.

Validation is the critical gate here. Before merging the model's output, run a strict schema check: every returned slot key must exist in the predefined slot schema, every value must match the declared type (string, date, enum, integer, boolean), and no slot should receive a value that contradicts a previously confirmed value unless the user explicitly corrected it. Implement a cross-slot contamination check that flags cases where the model places a value into the wrong slot—for example, putting a date into a currency slot or a person's name into a location slot. If validation fails, do not merge the patch. Instead, log the failure with the raw model output and the validation error, then retry with the same prompt but append the validation error message as a [CONSTRAINT] block. Limit retries to two attempts. If both fail, escalate to a human reviewer with the original utterance, the dialogue state, and both failed outputs. For high-stakes domains like healthcare or finance, consider requiring human approval on any slot update where the model's confidence is below a threshold or where the slot value changes a previously confirmed field.

Model choice matters for this task. Use a model with strong instruction-following and structured output capabilities—GPT-4o, Claude 3.5 Sonnet, or Gemini 2.0 Flash are good starting points. Avoid smaller or older models that struggle with multi-entity tracking across turns. Set temperature low (0.0–0.2) to reduce variance in slot assignment. If your application has strict latency requirements, consider caching the system prompt prefix and using a model router that sends simple, single-slot updates to a faster model and complex multi-entity resolutions to a more capable model. Log every resolution attempt with the input utterance, the prior state, the model's raw output, the validation result, and whether it was merged or escalated. This audit trail is essential for debugging coreference failures and for demonstrating compliance in regulated deployments. The most common production failure mode is the model silently resolving a pronoun to the wrong entity of the same type—your eval suite should include test cases with multiple entities sharing a type (e.g., two dates, two locations) to catch this before it reaches users.

IMPLEMENTATION TABLE

Expected Output Contract

The model must produce a JSON object representing the updated dialogue state. Each slot must be validated for type consistency, cross-slot contamination, and resolution confidence before downstream consumption.

Field or ElementType or FormatRequiredValidation Rule

resolved_slots

object

Must be a valid JSON object. Each key must match a slot name from [SLOT_SCHEMA].

resolved_slots.[slot_name].value

string | number | boolean | null

Type must match the expected type defined in [SLOT_SCHEMA]. Null is allowed only if the slot is optional or the reference is explicitly unresolvable.

resolved_slots.[slot_name].source_turn

integer | null

Must be a valid turn index from [DIALOGUE_HISTORY] where the value was grounded. Null if the value was provided in the current turn or is a system default.

resolved_slots.[slot_name].confidence

number

Float between 0.0 and 1.0. If below [CONFIDENCE_THRESHOLD], the slot must be null and an entry added to clarification_needed.

resolved_slots.[slot_name].resolution_method

string

Must be one of: 'explicit_match', 'coreference_resolved', 'bridging_inference', 'user_provided_current_turn', 'system_default'. No other values allowed.

clarification_needed

array

Array of objects, each with 'slot_name' (string) and 'question' (string). Must be empty if all required slots are resolved above the confidence threshold.

state_change_log

array

Array of objects, each with 'slot_name' (string), 'previous_value' (any), and 'new_value' (any). Must be empty if no slot values changed from the previous state in [CURRENT_DIALOGUE_STATE].

validation_warnings

array

Array of strings describing any type mismatches, cross-slot contamination risks, or low-confidence resolutions detected. Must be empty if no issues found. Do not halt output for warnings.

PRACTICAL GUARDRAILS

Common Failure Modes

Slot-filling coreference prompts fail in predictable ways when deployed in production task-oriented dialogue systems. These are the most common breakages and the guardrails that prevent them.

01

Cross-Slot Contamination

What to watch: The model resolves a reference correctly but places the value into the wrong slot (e.g., 'it' refers to the departure city but gets written into the arrival city slot). This is the most common production failure in multi-slot forms. Guardrail: Run a slot-type consistency validator after every resolution. Check that the resolved value matches the expected type, format, and domain of the target slot. Reject updates where the value violates slot constraints.

02

Silent Null Resolution

What to watch: The user says 'change it to Tuesday' but the model cannot identify what 'it' refers to. Instead of flagging ambiguity, the model guesses or leaves the slot unchanged without signaling uncertainty. The dialogue continues with a stale or incorrect state. Guardrail: Require the prompt to output an explicit reference_confidence score. If below a configurable threshold, trigger a targeted clarification question rather than proceeding with a low-confidence resolution.

03

Stale Reference Drift

What to watch: A user resolves 'the hotel' in turn 3, discusses restaurants for five turns, then says 'book it.' The model resolves 'it' to the hotel from turn 3, but the user now means the restaurant from turn 7. Recency and topic shift are ignored. Guardrail: Include a last_mentioned_turn field for every tracked entity. Weight resolution candidates by recency and topic continuity. Flag references to entities not mentioned in the last N turns for re-confirmation.

04

Over-Resolution of Explicit Inputs

What to watch: The user provides an explicit value ('departing from Chicago') but the coreference resolver treats it as a reference and maps it to a prior entity, overwriting the explicit input with stale state. Guardrail: Add a pre-check: if the current turn contains an explicit slot value, skip coreference resolution for that slot entirely. Only invoke resolution for slots that are empty or contain pronouns, demonstratives, or definite descriptions.

05

Entity Identity Swap

What to watch: Two entities of the same type are active (e.g., two different flights, two contacts). The model confuses them and resolves a reference to the wrong entity ID, causing downstream actions on the wrong object. Guardrail: Maintain a structured entity grid with unique IDs and distinguishing attributes. Include entity disambiguation in the prompt: when multiple candidates share a type, require the model to justify its selection using a distinguishing attribute before resolving.

06

Clarification Loop Exhaustion

What to watch: The model correctly identifies ambiguity and asks a clarification question, but the user's answer is still ambiguous, triggering another question. This repeats until the context window fills with clarification turns and the user abandons the task. Guardrail: Set a maximum clarification budget per slot (e.g., 2 attempts). On the third failure, escalate to a human agent or fall back to presenting the top candidates for explicit user selection rather than another open-ended question.

IMPLEMENTATION TABLE

Evaluation Rubric

How to test output quality before shipping. Run this rubric against a golden dataset of 50+ dialogue turns with known correct slot updates.

CriterionPass StandardFailure SignalTest Method

Slot Value Accuracy

Resolved slot value exactly matches the golden label for the target slot

Resolved value differs from golden label; value placed in wrong slot; value is null when golden label is non-null

Exact string match against golden dataset after normalization (lowercase, trim). Report precision/recall per slot type.

Cross-Slot Contamination

No entity from the dialogue appears in a slot it does not belong to

A date value appears in a name slot; a location appears in a quantity slot; entity type mismatch detected

Schema validator checks each resolved slot value against its expected type. Flag any type mismatch as contamination failure.

Coreference Chain Integrity

All pronouns and anaphoric references in the turn are resolved to the correct antecedent entity ID

A pronoun is resolved to the wrong entity; a reference is left unresolved when a valid antecedent exists; entity ID mismatch

Compare resolved entity IDs against golden antecedent mapping. Require exact ID match for each reference span in the turn.

Null Handling for Unresolvable References

Slots with no new information in the turn remain null; unresolvable references produce null with a clarification flag

Slot is overwritten with a hallucinated value; unresolvable reference is guessed instead of flagged; null slot is incorrectly populated

Check that slots without golden updates remain null. Verify clarification flag is true when no antecedent exceeds confidence threshold.

Slot Type Consistency

Every resolved slot value conforms to its declared type (date, string, integer, enum, boolean)

Date slot contains non-date string; enum slot contains value outside allowed set; integer slot contains float or text

Run type validator against output schema. Parse dates with strict format. Check enum membership. Reject any type violation.

Confidence Score Calibration

Confidence score above 0.8 correlates with correct resolution; score below 0.5 correlates with errors or clarification requests

High-confidence predictions are wrong; low-confidence predictions are correct; scores are uniformly high regardless of difficulty

Bin predictions by confidence decile. Compute accuracy per bin. Expect monotonic relationship. Flag inversion where low-confidence bin outperforms high-confidence bin.

Turn History Non-Regression

Resolving the current turn does not corrupt slot values set in prior turns that are not referenced now

Previously correct slot value is overwritten with null; prior slot value is replaced with a different entity; state drift detected

Run full dialogue sequence through resolver. Compare final state after all turns against golden final state. Flag any slot where final value differs from golden.

Clarification Trigger Appropriateness

Clarification is requested only when no candidate exceeds the confidence threshold and the slot is required for downstream action

Clarification requested for easily resolvable reference; no clarification requested when ambiguity is genuine and slot is critical; excessive clarification loops

Count clarification requests per dialogue. Compare against golden clarification flags. Measure false-positive and false-negative clarification rate.

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Start with the base prompt and a simple JSON schema for slots. Use a small, hand-labeled dialogue history of 5-10 turns. Skip the validator initially; just inspect outputs manually for correct slot filling and coreference resolution.

code
[SYSTEM]
You are a slot-filling assistant. Given [DIALOGUE_HISTORY] and [CURRENT_TURN], resolve any references to previously mentioned values and output a JSON object with the updated slot values for [TARGET_SLOTS].

[OUTPUT_SCHEMA]
{
  "slots": {
    "[SLOT_NAME]": "[RESOLVED_VALUE | null]"
  },
  "resolved_from_turn": [TURN_NUMBER | null]
}

Watch for

  • The model hallucinating slot values not present in the history
  • Cross-slot contamination where a value for one slot bleeds into another
  • Pronouns resolving to the wrong entity when multiple candidates share the same type
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.