Conversation Phase Segmentation Prompt

Full conversation transcript to segment into phases

USER: Hi, I need help with my account. AGENT: Sure, can you provide your email? USER: user@example.com

Must be non-empty string. Validate minimum 2 turns present. Check for turn delimiter consistency (USER:/AGENT:). Reject if only one speaker.

List of valid phase labels with definitions

["greeting", "authentication", "problem_description", "troubleshooting", "resolution", "closing"]

Must be a JSON array of strings. Validate no duplicate labels. Each label must have a corresponding definition in [PHASE_DEFINITIONS]. Reject if taxonomy is empty.

Mapping of phase labels to their semantic descriptions

{"greeting": "Initial pleasantries and rapport building", "authentication": "Identity verification steps"}

Must be a JSON object with keys matching [PHASE_TAXONOMY] exactly. Validate each definition is a non-empty string. Schema check required before prompt assembly.

String pattern separating individual turns in the transcript

"\n\n" or "---"

Must be a non-empty string. Validate that splitting [TRANSCRIPT] by this delimiter produces the expected turn count. Reject if delimiter not found in transcript.

Prefixes identifying speaker roles in each turn

["USER:", "AGENT:"]

Must be a JSON array of strings. Validate each turn in [TRANSCRIPT] starts with one of these prefixes. Reject if unknown speaker prefix detected.

Expected JSON structure for phase boundaries

{"phases": [{"phase_label": "string", "start_turn_index": 0, "end_turn_index": 3, "transition_justification": "string"}]}

Must be a valid JSON Schema object. Validate that required fields include phase_label, start_turn_index, end_turn_index, and transition_justification. Reject if schema allows overlapping turn ranges.

Upper bound on number of phases to prevent over-segmentation

8

Must be a positive integer. Validate that value is less than or equal to total turn count in [TRANSCRIPT]. Reject if set to 0 or exceeds reasonable segmentation granularity for transcript length.

Optional human-annotated phase boundaries for evaluation

null or {"phases": [...]}

If provided, must match [OUTPUT_SCHEMA] structure. Validate no overlapping turn ranges. Use for boundary accuracy scoring against model output. Null allowed when running without evaluation.

Array of objects

Must be a non-empty array. Each element must conform to the segment object schema defined in subsequent rows.

Object with start_turn and end_turn integers

start_turn must be >= 1. end_turn must be >= start_turn. Ranges must not overlap between segments. All turn indices must be covered from the first to the last turn of the input transcript.

String from a predefined taxonomy

Must match one of the labels in the provided [PHASE_TAXONOMY] exactly. Case-sensitive check required. Reject unknown labels.

String

Must be non-empty and contain a direct reference to at least one turn index or user utterance that triggered the phase change. Null or empty string is a failure.

Float between 0.0 and 1.0

Must be a valid float. Values outside the 0.0-1.0 range are invalid. If the model cannot provide a score, the parser should default to null and flag for human review.

Array of strings

If present, each string must be non-empty. An empty array is allowed to represent no dominant topic. Null is allowed if the field is omitted entirely.

Array of strings

If present, each string must describe an open question or incomplete task from the segment. Null is allowed. An empty array means no unresolved items were detected.

String

If present, must match the version identifier of the [PHASE_TAXONOMY] provided in the prompt. A mismatch triggers a warning and potential re-run with a corrected prompt.

Predicted phase boundary turn indices match gold annotations within ±1 turn tolerance for 90% of boundaries

Systematic offset >2 turns from gold boundaries or missed boundaries on clear topic shifts

Compute Jaccard similarity between predicted boundary indices and gold boundary indices across test set; flag conversations with similarity <0.85

Predicted phase labels match gold labels for 85% of segments when boundary alignment is within tolerance

Consistent label confusion between similar phases (e.g., troubleshooting vs diagnosis) or generic label overuse

Build confusion matrix from aligned segments; calculate Cohen's kappa against gold labels; investigate label pairs with kappa <0.7

Justification text identifies the specific user utterance or assistant action that triggered the phase change in 80% of transitions

Vague justifications (e.g., 'topic changed') without citing evidence or justifications that reference future turns not yet spoken

Human reviewer rates 100 random transition justifications on 1-5 scale for evidence grounding; pass if mean score ≥4.0

Fewer than 10% of gold single-phase segments are split into multiple predicted phases

Predicted phase count exceeds gold phase count by >20% on same conversation; rapid oscillation between phases within one topic

Count predicted phases per conversation; flag conversations where predicted_count > 1.2 * gold_count; review for flickering on clarification turns

Fewer than 10% of gold multi-phase segments are merged into a single predicted phase

Predicted phase count is less than 80% of gold phase count; gradual topic drift goes undetected across >5 turns

Count predicted phases per conversation; flag conversations where predicted_count < 0.8 * gold_count; review for missed drift on gradual shifts

Phase segmentation of two conversations following the same workflow produces boundary indices within ±2 turns of each other for 90% of equivalent transitions

Same workflow type produces wildly different phase counts or boundary placements across similar transcripts

Pair conversations by workflow template; compute variance of phase count and mean boundary position; flag pairs with count difference >2 or mean boundary offset >3 turns

Social turns, greetings, and off-topic digressions do not trigger phase boundaries in task-oriented segments

Greeting or sign-off turns classified as phase transitions; small talk within a task phase causes boundary insertion

Inject small talk turns into gold-annotated transcripts; measure false boundary rate on injected turns; pass if <5% of small talk turns trigger boundaries

User correction turns (e.g., 'no, I meant...') within a phase do not trigger new phase boundaries unless they represent a genuine topic reset

Every user correction creates a new phase; correction cascades produce multiple spurious boundaries

Identify correction turns in gold set; measure boundary insertion rate on correction turns; pass if <15% of corrections trigger boundaries when gold shows no phase change