Inferensys

Prompt

Support Ticket Unresolved Item Audit Prompt

A practical prompt playbook for using the Support Ticket Unresolved Item Audit Prompt in production AI workflows to identify dropped questions and unaddressed requests in closed support tickets.
Accountant using AI for financial close automation, accounting software on screen, home office evening work session.
PROMPT PLAYBOOK

When to Use This Prompt

Define the job, the reader, and the operational constraints for the Support Ticket Unresolved Item Audit Prompt.

This prompt is designed for support operations and quality assurance (QA) teams who need to systematically audit closed support tickets. Its core job is to analyze a full ticket transcript and identify any customer questions, explicit requests, or agent commitments that were left unaddressed before the ticket was closed. The primary user is a QA analyst, team lead, or operations manager responsible for ensuring service quality, preventing customer churn from ignored issues, and gathering data for agent coaching. The required context is a complete, unredacted ticket transcript including all public replies, internal notes, and system events, along with the official closure reason.

Use this prompt when you have a high volume of closed tickets and need a structured, repeatable audit process that scales beyond manual review. It is appropriate for tickets of medium-to-high complexity where multiple issues may be interleaved, and where a premature or incomplete resolution is a significant business risk. The prompt is designed to be wired into a batch processing pipeline where its output—a structured JSON audit report—can be ingested by a dashboard, a QA scoring system, or a re-open queue. It is not a replacement for a real-time agent assist tool; it is a post-hoc analysis instrument.

Do not use this prompt for real-time conversation guidance, for tickets still in progress, or for legal/compliance discovery where a human attorney's review is mandatory. It is not designed to judge agent politeness, tone, or adherence to scripts unless those factors directly caused an item to be dropped. The prompt's value is in its strict focus on factual resolution status: was a question asked, and was an answer provided? If your workflow requires sentiment analysis, script adherence scoring, or fraud detection, those should be separate evaluation steps. After running this audit, the next step is to route high-severity unresolved items to a human reviewer for final disposition and, if necessary, ticket re-opening.

PRACTICAL GUARDRAILS

Use Case Fit

Where the Support Ticket Unresolved Item Audit Prompt delivers value and where it introduces risk. Use these cards to decide if this prompt fits your workflow before you integrate it.

01

Good Fit: Post-Close QA Review

Use when: Support tickets are already closed and you need a systematic audit to detect unresolved questions or commitments before the customer notices. Guardrail: Run this prompt as a batch process on closed tickets, not in real-time during agent workflows, to avoid disrupting active conversations.

02

Bad Fit: Real-Time Agent Copilot

Avoid when: You need live detection of unanswered questions during an active conversation. This prompt is designed for post-hoc audit, not turn-by-turn tracking. Guardrail: Use the Pending Question Detection Prompt Template for real-time use cases instead.

03

Required Inputs

What you need: A complete, timestamped ticket transcript with agent and customer turns clearly delineated. The prompt cannot audit what it cannot read. Guardrail: Validate transcript completeness before invoking the prompt. Missing turns or merged agent/customer roles will produce false negatives on unresolved items.

04

Operational Risk: False Positives on Rhetorical Questions

What to watch: The model may flag rhetorical questions, social niceties, or questions the customer answered themselves as unresolved. Guardrail: Add explicit instructions in the prompt to distinguish genuine information-seeking questions from polite phrasing. Include few-shot examples of rhetorical questions that should not be flagged.

05

Operational Risk: Severity Inflation

What to watch: Without calibration, the model may classify minor unanswered questions as high-severity, overwhelming QA teams with false critical alerts. Guardrail: Define severity criteria with concrete examples in the prompt. Require the model to cite specific evidence from the transcript for any severity classification above 'low'.

06

Human Review Requirement

What to watch: The audit report is a screening tool, not a final determination. Model errors on intent classification or severity can lead to unnecessary ticket reopens or missed genuine issues. Guardrail: Route all audit findings through a human QA reviewer before any ticket is reopened. Use the report as a prioritization aid, not an automated decision engine.

PROMPT PLAYBOOK

Copy-Ready Prompt Template

A reusable prompt template with square-bracket placeholders for auditing closed support tickets to find unresolved items.

This prompt template is the core of the unresolved item audit workflow. It is designed to be copied directly into your AI harness, with placeholders that you replace with real ticket data, severity definitions, and output constraints before execution. The template assumes you have already extracted the full conversation transcript and ticket metadata from your support platform. It does not include retrieval steps; those belong in the implementation harness.

text
You are a support quality auditor. Your task is to analyze a closed support ticket transcript and identify every question, request, or commitment that was not fully addressed before the ticket was closed.

## INPUT
Ticket ID: [TICKET_ID]
Closure Reason: [CLOSURE_REASON]
Transcript:
[TRANSCRIPT]

## DEFINITIONS
- **Unresolved Question**: A direct or implied question from the customer that received no answer, a partial answer, or a deflection.
- **Unresolved Request**: A customer request for action, information, or escalation that was acknowledged but not completed.
- **Unresolved Commitment**: A statement by the support agent promising follow-up, investigation, or future action that has no evidence of completion in the transcript.

## SEVERITY CLASSIFICATION
Use these levels for each finding:
- **BLOCKER**: The unresolved item prevents the customer from using the product or service. Ticket should not have been closed.
- **HIGH**: The unresolved item materially impacts the customer's outcome or satisfaction. Re-open recommended.
- **MEDIUM**: The unresolved item is a quality gap but does not change the core resolution. Note for coaching.
- **LOW**: Minor follow-up or clarification that could be handled in a new interaction.

## CONSTRAINTS
- Only flag items where the transcript provides clear evidence the item was raised and not resolved.
- Do not flag rhetorical questions, social niceties, or items the customer explicitly withdrew.
- If the agent provided a partial answer, classify it as unresolved and note what was missing.
- If the customer's question was ambiguous and the agent did not seek clarification, flag it as unresolved with a note about the ambiguity.
- Do not invent items not present in the transcript.

## OUTPUT_SCHEMA
Return a JSON object with this structure:
{
  "ticket_id": "string",
  "audit_timestamp": "ISO 8601",
  "total_unresolved_items": integer,
  "reopen_recommended": boolean,
  "reopen_reason": "string or null",
  "findings": [
    {
      "finding_id": "string",
      "type": "unresolved_question | unresolved_request | unresolved_commitment",
      "severity": "BLOCKER | HIGH | MEDIUM | LOW",
      "customer_turn_reference": "string (turn number or timestamp from transcript)",
      "agent_turn_reference": "string or null (where the item should have been addressed)",
      "description": "string (what was asked or promised)",
      "evidence_excerpt": "string (verbatim quote from transcript)",
      "gap_analysis": "string (why this was not resolved)",
      "recommended_action": "string"
    }
  ]
}

## EXAMPLES
[EXAMPLES]

## RISK_LEVEL
[RISK_LEVEL]

To adapt this template, replace each square-bracket placeholder with real data. [TRANSCRIPT] should contain the full conversation with turn markers or timestamps. [EXAMPLES] should include one or two few-shot examples showing correct severity classification, especially for borderline cases like partial answers. [RISK_LEVEL] should be set to HIGH for regulated industries or customer-facing audit workflows, which triggers additional constraints in the harness such as requiring human review before the audit report is finalized. If your support platform uses custom closure reasons, map them into [CLOSURE_REASON] and consider adding a [CLOSURE_REASON_DEFINITIONS] section if the model needs to interpret them. The output schema is designed for programmatic consumption; do not modify the field names without updating downstream validators.

IMPLEMENTATION TABLE

Prompt Variables

Required inputs for the Support Ticket Unresolved Item Audit Prompt. Each placeholder must be populated before the prompt is assembled and sent to the model. Missing or malformed inputs are the most common cause of false negatives in audit runs.

PlaceholderPurposeExampleValidation Notes

[TICKET_TRANSCRIPT]

Full conversation transcript between customer and support agent, including timestamps and speaker labels

CUSTOMER (2025-01-15 14:02): My billing shows two charges... AGENT (2025-01-15 14:05): Let me look into that...

Must contain at least one customer turn and one agent turn. Reject if empty or only system-generated messages. Timestamps required for severity scoring of aged unresolved items.

[TICKET_METADATA]

Structured ticket context including ID, priority, status, category, and closure reason

{"ticket_id": "TKT-48291", "status": "closed", "closure_reason": "resolved", "priority": "high", "category": "billing"}

Must be valid JSON with required fields: ticket_id, status, closure_reason. Reject if status is not 'closed'. Closure reason must be non-null for closed tickets.

[CLOSURE_TIMESTAMP]

ISO 8601 timestamp of when the ticket was closed

2025-01-16T09:30:00Z

Must parse as valid ISO 8601 datetime. Used to calculate age of unresolved items and flag premature closure. Reject if timestamp is before the last transcript turn.

[AGENT_NAME]

Name or identifier of the support agent who handled the ticket

Must match the agent label used in the transcript speaker prefixes. Used for ownership attribution in the audit report. Null allowed if agent identity is anonymized.

[AUDIT_POLICY]

Rules defining what constitutes an unresolved item, severity thresholds, and re-open criteria for this team

{"unresolved_definitions": ["direct question without answer", "promised follow-up not delivered"], "severity_thresholds": {"critical": "billing or account access", "high": "promised callback missed"}, "reopen_criteria": ["any critical unresolved item", "3+ high items"]}

Must be valid JSON with at least one entry in unresolved_definitions and reopen_criteria. Reject if policy is empty or missing required keys. Policy drives classification consistency across audit runs.

[PREVIOUS_AUDIT_RESULT]

Output from the last audit of this ticket, if any, for detecting regressions or repeated patterns

{"audit_id": "AUD-2025-01-10-003", "unresolved_count": 2, "reopen_recommended": false}

Null allowed for first audit. If present, must contain audit_id and unresolved_count fields. Used to detect whether previously flagged items were addressed before re-closure.

[OUTPUT_SCHEMA_VERSION]

Schema version identifier for the expected output format, enabling safe migration when the audit report structure changes

2.1.0

Must match a known schema version in the audit system registry. Reject if version is unsupported. Prevents silent output format breakage when prompt is updated.

PROMPT PLAYBOOK

Implementation Harness Notes

How to wire the Support Ticket Unresolved Item Audit Prompt into a production quality review pipeline.

This prompt is designed as a batch audit step, not a real-time chat interaction. The primary integration point is a post-resolution quality assurance (QA) pipeline that runs after a support ticket is closed. The application should fetch the full ticket transcript, including all agent and customer messages, and inject it into the [TICKET_TRANSCRIPT] placeholder. The model choice should favor strong reasoning and long-context handling; GPT-4o or Claude 3.5 Sonnet are appropriate defaults. Because this is a high-stakes audit that can trigger ticket re-opening, the system must never automatically act on the model's output. The generated audit report must be routed to a human QA reviewer queue for confirmation before any ticket status change occurs.

The implementation should wrap the LLM call in a validation layer that enforces the expected output schema. Parse the JSON response and verify that every identified unresolved item includes a turn_reference (a quote or message index from the transcript) and a severity value from the allowed enum: critical, high, medium, or low. If the model output fails JSON parsing or schema validation, implement a single retry with the validation error message appended to the prompt as a correction hint. If the retry also fails, log the raw output and flag the ticket for manual audit without an AI-generated report. Logging must capture the ticket ID, prompt version, model used, raw response, validation status, and the final reviewer disposition for every audit run. This creates an evidence trail for QA process improvement and model performance monitoring.

Avoid wiring this prompt directly into a customer-facing chatbot or an automated ticket-closure workflow. The model can misclassify rhetorical questions as unresolved or miss implicit requests that a human reviewer would catch. The harness should treat the AI output as a draft audit that accelerates human review, not a replacement for it. For high-volume support operations, consider batching audits during off-peak hours and caching transcripts to avoid repeated fetches. If your ticketing system supports webhooks, trigger the audit pipeline on a ticket.closed event and enqueue the result for QA review within your existing support tooling. The next step after implementing the harness is to build a golden dataset of 50–100 manually audited tickets to measure precision and recall before scaling the pipeline.

PRACTICAL GUARDRAILS

Common Failure Modes

What breaks first when auditing support tickets for unresolved items and how to guard against it.

01

Implicit Questions Missed

What to watch: The model only flags explicit question-mark sentences and misses implied requests like 'I wish I knew why...' or 'Still confused about...'. Guardrail: Include few-shot examples of implicit questions in the prompt and add a secondary check for sentiment-laden or unresolved statements.

02

Rhetorical Questions Flagged

What to watch: The model treats every question as unresolved, including rhetorical ones ('Isn't that weird?') or self-answered questions. Guardrail: Add a classification step that distinguishes rhetorical from genuine information-seeking questions before flagging them as unresolved.

03

Partial Answers Scored as Resolved

What to watch: The agent provided a vague acknowledgment or a partial answer, but the model marks the item as resolved. Guardrail: Require the model to extract the specific answer provided and compare it against the question's intent. Flag mismatches where the answer lacks required detail.

04

Severity Inflation on Low-Risk Items

What to watch: Every unresolved item is classified as 'Critical' or 'High' severity, overwhelming the review queue. Guardrail: Define explicit severity criteria in the prompt based on blocking status, SLA risk, and customer sentiment signals rather than defaulting to high severity.

05

Context Window Truncation

What to watch: Long ticket threads exceed the context window, causing the model to miss questions in earlier turns. Guardrail: Implement a pre-processing step that extracts all user questions first, then run the audit on the extracted list rather than the full transcript.

06

Re-Open Criteria Too Aggressive

What to watch: The model recommends re-opening tickets for minor unresolved items that could be handled in a follow-up. Guardrail: Add a gating rule that requires at least one blocking or SLA-impacting unresolved item before recommending re-open, with a separate track for non-blocking follow-ups.

IMPLEMENTATION TABLE

Evaluation Rubric

Use this rubric to test the Support Ticket Unresolved Item Audit Prompt before shipping. Each criterion targets a known failure mode in production audit workflows. Run these checks against a golden dataset of closed tickets with known unresolved items.

CriterionPass StandardFailure SignalTest Method

Recall of explicit questions

All direct questions (ending in '?') are identified with turn reference

Missed question in transcript; output count lower than ground truth

Run against 50 transcripts with hand-labeled questions; require recall >= 0.95

Implicit question detection

At least 80% of implied requests ('I need...', 'Can you...') flagged as unresolved when no response exists

Systematic miss of polite-form requests or statements of need without question marks

Test set of 30 implicit-only questions; measure recall against human annotator agreement

Rhetorical question exclusion

Zero rhetorical questions ('you know?', 'right?') flagged as unresolved

Flagged rhetorical question appears in output with severity > low

Curated list of 20 rhetorical examples; require precision = 1.0 on exclusion

Severity classification accuracy

Blocking items scored high; cosmetic items scored low; agreement with QA lead on 90% of cases

Blocking issue scored low or cosmetic issue scored high in >10% of test cases

Pairwise comparison against senior QA agent ratings on 40 mixed-severity items

Re-open criteria validity

Every recommended re-open includes specific evidence from transcript and a verifiable condition

Re-open recommendation without citation or with condition already satisfied in transcript

Manual review of 25 re-open recommendations; require 100% evidence grounding

Hallucinated unresolved items

Zero items reported that are not traceable to a specific turn in the transcript

Output contains an item with fabricated turn number or non-existent user statement

Diff output against transcript; flag any item without exact string match or valid paraphrase

Output schema compliance

Valid JSON matching [OUTPUT_SCHEMA] on first generation; all required fields present

Parse error, missing required field, or extra field not in schema

Automated schema validation on 100 outputs; require 100% parse success rate

Empty transcript handling

Returns empty items array with null-safe structure when no unresolved items exist

Returns error, hallucinated items, or non-empty array for clean transcript

Test set of 20 clean transcripts with zero unresolved items; require empty output

ADAPTATION OPTIONS

Adapt This Prompt

How to adapt

Start with the base prompt and a single ticket transcript. Remove the structured output schema initially and ask the model to return findings as a bulleted list. Use a frontier model (GPT-4o, Claude 3.5 Sonnet) with default temperature. Manually review 10-20 tickets to calibrate severity thresholds before adding automation.

Prompt modification

Remove the [OUTPUT_SCHEMA] block and replace with: Return findings as a bulleted list with ticket ID, unresolved item, severity (Low/Medium/High/Critical), and the turn where it was first raised.

Watch for

  • Over-flagging rhetorical questions or small talk as unresolved items
  • Missing implicit questions where the customer didn't use a question mark
  • Inconsistent severity classification without clear rubrics
  • No distinction between partially answered and completely dropped questions
Prasad Kumkar

About the author

Prasad Kumkar

CEO & MD, Inference Systems

Prasad Kumkar is the CEO & MD of Inference Systems and writes about AI systems architecture, LLM infrastructure, model serving, evaluation, and production deployment. Over 5+ years, he has worked across computer vision models, L5 autonomous vehicle systems, and LLM research, with a focus on taking complex AI ideas into real-world engineering systems.

His work and writing cover AI systems, large language models, AI agents, multimodal systems, autonomous systems, inference optimization, RAG, evaluation, and production AI engineering.