Unresolved Question Carry-Forward Prompt

This prompt is designed for multi-turn RAG assistants and copilots that must maintain a running ledger of questions the system could not answer earlier. The core job-to-be-done is transforming 'I don't know yet' into 'I can answer that now' without requiring the user to repeat themselves. When a user asks a question and the retrieved context lacks sufficient evidence, the system records it as unresolved. On subsequent turns, when new documents are retrieved or the user provides additional information, the system re-evaluates pending questions and attempts to answer them. This is essential for customer-facing Q&A systems, research assistants, support copilots, and any multi-turn RAG application where answer continuity across turns is a product requirement.

The ideal user is an AI engineer or product developer building a conversational RAG system where users expect the assistant to remember what it couldn't answer and proactively close those gaps. Required context includes: a persistent session store that can hold the unresolved question ledger across turns, a retrieval system that may return different or expanded evidence on subsequent turns, and a clear policy for when a question should be marked as permanently unresolvable versus temporarily pending. The prompt expects [CURRENT_QUESTION], [RETRIEVED_CONTEXT], [UNRESOLVED_QUESTIONS_LEDGER], and [CONVERSATION_HISTORY] as inputs, and produces a structured output containing both a resolution attempt for each pending question where new evidence is now available and an updated ledger reflecting any remaining gaps.

Do not use this prompt when the assistant operates in a single-turn mode with no memory, when the retrieval corpus is static and will never yield new evidence on subsequent turns, or when the product requirement is to simply apologize and move on rather than track unresolved questions. This prompt is also inappropriate for systems where users expect immediate answers or escalation to a human rather than deferred resolution. If your RAG pipeline cannot guarantee that subsequent retrieval calls will return different or expanded context, the carry-forward mechanism will produce repeated 'still cannot answer' outputs that frustrate users rather than build trust. Before implementing this prompt, ensure your retrieval system supports re-querying with expanded or reformulated queries when initial evidence is insufficient, and that your session management layer can reliably persist and update the unresolved question ledger across turns.

This prompt is designed to be wired into your RAG pipeline after retrieval but before answer generation. It instructs the model to maintain a running list of questions the system could not answer in prior turns, compare them against newly retrieved evidence, and either resolve them with grounded answers or carry them forward with updated status. The template uses square-bracket placeholders that your application must populate at runtime from conversation state, retrieval results, and session metadata.

textCopy
You are an assistant that tracks unresolved questions across conversation turns.

## PRIOR UNRESOLVED QUESTIONS
[UNRESOLVED_QUESTIONS]

## CURRENT USER QUESTION
[CURRENT_QUESTION]

## RETRIEVED CONTEXT
[RETRIEVED_CONTEXT]

## INSTRUCTIONS
1. Review the PRIOR UNRESOLVED QUESTIONS list. For each question, check whether the RETRIEVED CONTEXT now contains sufficient evidence to answer it.
2. For any previously unresolved question that can now be answered:
   - Provide a grounded answer with inline citations to the RETRIEVED CONTEXT.
   - Mark the question as RESOLVED.
3. For any previously unresolved question that still cannot be answered:
   - Keep it in the UNRESOLVED list.
   - Update its status with a brief note on what evidence is still missing.
4. Answer the CURRENT USER QUESTION using only the RETRIEVED CONTEXT.
   - If the context is insufficient, add the question to the UNRESOLVED list with a clear description of what information is needed.
   - If the context is sufficient, provide a grounded answer with citations.
5. Do not fabricate answers. If evidence is missing, say so explicitly.

## OUTPUT FORMAT
Return a JSON object with this exact structure:
{
  "current_answer": {
    "text": "string (the answer to CURRENT_QUESTION, or null if unanswered)",
    "citations": ["source_id_1", "source_id_2"],
    "status": "ANSWERED | UNANSWERED"
  },
  "resolved_questions": [
    {
      "question": "string",
      "answer": "string",
      "citations": ["source_id"]
    }
  ],
  "updated_unresolved_questions": [
    {
      "question": "string",
      "missing_evidence": "string (what information is still needed)",
      "first_asked_turn": number,
      "attempts": number
    }
  ]
}

## CONSTRAINTS
- Never invent sources or citations.
- If the RETRIEVED_CONTEXT is empty, mark all questions as UNANSWERED.
- Do not repeat resolved questions in the unresolved list.
- Preserve the original question text when carrying forward.

Adapt this template by adjusting the output schema to match your application's data model. The [UNRESOLVED_QUESTIONS] placeholder should be populated from your session state store, not from model memory. The [RETRIEVED_CONTEXT] placeholder expects pre-ranked passages with source identifiers. If your retrieval system does not return source IDs, add a preprocessing step to assign them before populating this prompt. For high-stakes domains, route outputs with status: UNANSWERED to a human review queue rather than surfacing them directly to users.

The Unresolved Question Carry-Forward Prompt is not a standalone call; it is a stateful component that must be integrated into a conversation loop. The application layer is responsible for maintaining a persistent list of unresolved questions across turns, injecting that list into the prompt's [UNRESOLVED_QUESTIONS] placeholder, and updating the list based on the model's structured output. This means your harness must parse the model's response, extract the resolved and new_unresolved fields, and merge them into the canonical question list before the next turn begins. Do not rely on the model to remember unresolved questions implicitly through conversation history alone—explicit state management prevents questions from being silently dropped when context windows shift or when conversation history compression occurs.

The prompt expects a JSON output with a strict schema: an array of resolved question objects (each with question_id, question_text, and answer_summary), an array of new_unresolved question objects (with question_id, question_text, and reason_unresolved), and a current_answer string. Your implementation harness must validate this schema after every model call. Use a JSON schema validator to check required fields, types, and non-null constraints. If validation fails, implement a retry loop with a maximum of two correction attempts, feeding the validation error message back to the model in a repair prompt. If the output still fails validation after retries, log the failure, preserve the existing unresolved question list unchanged, and surface a fallback answer to the user indicating a processing error. For model choice, prefer models with strong JSON mode or structured output support (such as GPT-4o with response_format set to json_schema or Claude with tool-use-based structured output). Avoid models with inconsistent JSON adherence for this workflow unless you add a robust repair layer.

Logging and observability are critical for debugging question carry-forward failures. At minimum, log the following on every turn: the incoming unresolved question count, the model's raw output, the parsed resolved and new_unresolved counts, any validation errors, and the updated question list after merging. This trace data lets you detect regressions such as questions that persist across too many turns without resolution, questions that are incorrectly marked as resolved, or new questions that duplicate existing entries. Implement a deduplication check in the application layer: before adding a new_unresolved question to the canonical list, compare its text against existing unresolved questions using a simple embedding similarity threshold or exact-match check. This prevents the list from bloating with near-duplicate entries when the model rephrases the same unresolved concern across turns. For high-stakes deployments, add a human-review queue trigger when the unresolved question count exceeds a configurable threshold (e.g., five open questions) or when a single question remains unresolved for more than three turns—this prevents the system from silently accumulating debt it cannot resolve.