Unresolved Question Audit Trail Prompt

This prompt is designed for support operations managers and AI engineers who need to systematically audit long-running chat sessions for dropped user questions. The core job-to-be-done is not just answering a user, but proving that every explicit and implicit question was addressed. The ideal user is someone responsible for quality assurance, compliance, or agent handoff accuracy, who needs a structured, machine-readable audit record rather than a conversational summary. You should use this prompt when you have a complete, multi-turn conversation transcript and need to generate a traceable list of unresolved questions with turn references and resolution statuses before closing a ticket, escalating to a human, or archiving a session.

To use this prompt effectively, you must provide the full conversation history in a structured format, typically as a list of turns with speaker roles and timestamps. The prompt works best when the conversation is bounded (a single session) and the user's questions are distinguishable from rhetorical statements or social pleasantries. It requires a clear definition of what constitutes an 'unresolved' state in your operational context—whether it means no direct answer was given, the answer was a deflection, or the assistant promised a follow-up that never occurred. The prompt is designed to be wired into a post-session processing pipeline, not a real-time chat flow, because it requires the complete context to make accurate judgments about question resolution.

Do not use this prompt for real-time turn-by-turn processing, as it is a batch analysis tool. It is also not suitable for conversations where questions are intentionally left open-ended as a teaching or discovery technique, unless you explicitly define those as resolved. Avoid using it on sessions with heavy multimodal content (images, audio) unless you have transcribed and linearized that content into the text history. The primary failure mode is false positives on rhetorical questions, so you must pair this prompt with an evaluation harness that checks for over-flagging. The next step after reading this section is to prepare your conversation transcript in the required input schema and define your resolution status taxonomy before running the prompt.

The following prompt template is designed to be copied directly into your prompt management system, IDE, or orchestration layer. It uses square-bracket placeholders for all dynamic inputs, making it straightforward to wire into an application harness. The template instructs the model to act as a conversation auditor, systematically reviewing each user turn, identifying questions, and determining whether they received a satisfactory answer. It is structured to produce a JSON output that can be ingested by downstream monitoring dashboards, support QA tools, or compliance review systems.

textCopy
You are a conversation quality auditor. Your task is to scan the full conversation history and produce a structured audit trail of every question asked by the user that received no answer or an incomplete answer.

## INPUT

[CONVERSATION_HISTORY]

## INSTRUCTIONS

1. Scan every user turn in the conversation history chronologically.
2. For each user turn, identify all explicit and implicit questions. An explicit question ends with a question mark or uses a clear interrogative structure. An implicit question is a statement that clearly expects a response or resolution (e.g., "I need help with my billing," "The error is still happening").
3. For each identified question, determine whether the assistant provided a complete, substantive answer in any subsequent turn. A complete answer directly addresses the core request with specific information. An incomplete answer acknowledges the question but defers, asks for clarification without resolving, or provides only partial information.
4. Exclude rhetorical questions where the user clearly does not expect an answer. Exclude social niceties ("How are you?") unless the context suggests a genuine expectation of response.
5. For each unresolved or incompletely resolved question, produce a record in the output array.

## OUTPUT SCHEMA

Produce a valid JSON object with the following structure:

{
  "audit_summary": {
    "total_user_turns_scanned": <integer>,
    "total_questions_identified": <integer>,
    "total_unresolved_questions": <integer>,
    "audit_timestamp": "<ISO 8601 timestamp of audit generation>"
  },
  "unresolved_questions": [
    {
      "question_id": "<unique identifier, e.g., Q-001>",
      "user_turn_number": <integer, the turn number in the conversation where the question was asked>,
      "question_text": "<the exact or closely paraphrased question text>",
      "question_type": "explicit | implicit",
      "resolution_status": "unanswered | incomplete",
      "resolution_detail": "<explanation of why the question is considered unresolved, referencing the assistant's response or lack thereof>",
      "assistant_response_turn_number": <integer or null if no response attempted>,
      "suggested_action": "<one of: 're-queue for agent', 'auto-answer with knowledge base', 'escalate to supervisor', 'clarify with user'>"
    }
  ]
}

## CONSTRAINTS

- Do not fabricate questions that were not asked.
- If a question was answered across multiple assistant turns, consider the full response sequence when determining completeness.
- If the conversation is empty or contains no user turns, return an empty `unresolved_questions` array with appropriate summary counts.
- If you are uncertain whether a question was resolved, mark it as `incomplete` and note the uncertainty in `resolution_detail`.
- Do not include questions that were resolved in a later turn, even if the user re-asked them.

## RISK LEVEL

[RISK_LEVEL]

## EXAMPLES

[EXAMPLES]

To adapt this template, replace each square-bracket placeholder with the appropriate content for your application. [CONVERSATION_HISTORY] should contain the full transcript, ideally with clear speaker labels and turn numbers. [RISK_LEVEL] can be set to low, medium, or high to adjust the model's sensitivity—at high, the model should err toward flagging borderline cases as unresolved and include a note recommending human review. [EXAMPLES] should contain one or two few-shot demonstrations of a conversation excerpt and the correct audit output, which is critical for calibrating the model's judgment on implicit questions and rhetorical question exclusion. If your application requires a different output schema, replace the JSON structure in the OUTPUT SCHEMA section while preserving the square-bracket convention for any new dynamic fields. Always validate the model's output against the schema before ingesting it into downstream systems, and log any parse failures for retry or human review.

This prompt is designed to be called at session boundaries—typically when a support ticket is closed, a chat is escalated, or a compliance review is triggered. It should run as a post-processing step, not inline during the conversation, because it requires the full, unpruned transcript as input. The primary integration point is a webhook or queue worker that fires on session.end or ticket.close events, retrieves the complete message history from your conversation store, and passes it to the model with this prompt template. The output is a structured JSON audit record that your system should persist alongside the session for downstream review, analytics, or regulatory evidence.

Wire the prompt into an application by building a thin service with three stages: fetch, audit, and store. First, fetch the full conversation from your database or log system, ensuring you include turn metadata (speaker role, timestamp, message ID) for every message. Second, call the LLM with the prompt template, passing the conversation as [CONVERSATION_HISTORY] and your output schema as [OUTPUT_SCHEMA]. Use a model with strong JSON mode or structured output support (e.g., GPT-4o with response_format, Claude with tool use for the schema). Set temperature=0 to maximize consistency. Third, validate the returned JSON against your expected schema before storing it. If validation fails, retry once with the error message appended to the prompt as a correction hint. If it fails again, flag the session for human review rather than silently dropping the audit record.

Critical harness checks: (1) Validate that every turn_reference in the output points to a real message ID in the input conversation—hallucinated turn references are a common failure mode. (2) Compare the count of unresolved questions against a simple heuristic baseline (e.g., count of user messages ending in ? that received no assistant reply within two turns) to catch gross over- or under-detection. (3) Log every audit run with the session ID, model version, prompt version, and validation status so you can trace regressions when the prompt or model changes. (4) For high-stakes compliance use cases, route sessions where unresolved_questions is non-empty to a human reviewer before the audit record is finalized. Do not treat the model's output as ground truth without verification when the downstream consequence is regulatory exposure or customer harm.

When to avoid this prompt: Do not run it on every turn—it is too expensive and slow for real-time use. Do not use it on sessions shorter than three turns, where the signal is too sparse to justify the audit overhead. Do not use it as a substitute for real-time question tracking in your application layer; if you need live detection of unanswered questions to trigger alerts or agent prompts, implement a lightweight classifier or rule-based detector that runs per turn, and reserve this full audit prompt for post-session review. The prompt is an audit tool, not a real-time intervention mechanism.