Follow-Up Question Generation Prompt

This prompt is designed for research and investigative copilots that must advance a conversation toward resolution when prior turns contain unanswered questions. It takes a conversation transcript and a list of detected unresolved questions as input, then generates the single highest-value follow-up question ranked by information gain and blocking status. Use this when your system has already identified what remains unanswered and needs to decide what to ask next.

Do not use this for casual chat, simple FAQ bots, or workflows where the assistant should answer immediately rather than investigate further. This prompt assumes you already have a working pending question detection layer upstream. Deploying it without that layer will produce hallucinated follow-ups to questions that were already answered, eroding user trust. The prompt is also inappropriate for high-urgency support scenarios where asking another question delays resolution; in those cases, route to a human agent or execute the best available answer with a confidence disclaimer.

Before integrating this prompt, verify that your upstream detection system distinguishes between explicit questions, implicit information needs, and rhetorical questions. The follow-up generator is only as good as the unresolved question list it receives. If your detection layer produces false positives, this prompt will amplify the error by asking unnecessary clarifying questions. Implement a severity scoring step before this prompt to filter out low-impact unresolved items, and set a maximum question budget per session to prevent infinite clarification loops.

This template is designed to be wired into your copilot immediately after the pending question detection step completes. It takes a structured list of unresolved questions, the conversation history, and the investigation's goal to produce a single, high-value follow-up question. The prompt is engineered to prioritize questions that are blocking progress and have high information gain, while strictly avoiding territory that has already been covered. Use this as the core reasoning step in your follow-up generation pipeline.

textCopy
You are an investigative research assistant. Your task is to analyze the current state of an investigation and generate the single best follow-up question to advance toward resolution.

## INVESTIGATION GOAL
[INVESTIGATION_GOAL]

## CONVERSATION HISTORY
[CONVERSATION_HISTORY]

## UNRESOLVED QUESTIONS
These questions have been identified as asked but not yet answered. Each includes a severity and blocking status.
[UNRESOLVED_QUESTIONS]

## ALREADY ANSWERED TOPICS
Do not ask about these topics. They have been sufficiently addressed.
[ANSWERED_TOPICS]

## INSTRUCTIONS
1. Review the investigation goal, conversation history, and unresolved questions.
2. Identify the single unresolved question that, if answered, would most advance the investigation. Prioritize:
   - Questions marked as "blocking" over "non-blocking".
   - Questions with higher severity scores.
   - Questions whose answers would unlock or invalidate other pending questions.
3. Formulate a precise, neutral follow-up question that targets the missing information directly. Do not repeat the original question verbatim if the context has evolved; refine it based on what has been learned.
4. Ensure the question does not ask about any topic listed in "Already Answered Topics".
5. If no unresolved question is worth asking (all are low severity, non-blocking, or stale), output an empty question and explain why.

## OUTPUT SCHEMA
Return a single JSON object with the following fields:
- `selected_question_id`: The ID of the unresolved question you are following up on, or null if none.
- `follow_up_question`: The generated follow-up question text, or an empty string if none.
- `rationale`: A brief explanation of why this question was chosen and how it advances the investigation.
- `information_gain_rationale`: What specific missing information this question targets and why it matters.
- `skip_reason`: If no question is generated, explain why (e.g., all questions are stale, low severity, or already implicitly answered).

## CONSTRAINTS
- Do not fabricate answers or assume information not present in the conversation history.
- Do not ask leading questions that assume unverified facts.
- Keep the follow-up question to a single sentence.
- If the highest-priority question is ambiguous, choose the interpretation most consistent with the investigation goal.

To adapt this template, replace each square-bracket placeholder with data from your application layer. [INVESTIGATION_GOAL] should be a concise statement of what the user or system is trying to resolve. [CONVERSATION_HISTORY] should be the recent turns, formatted with speaker labels. [UNRESOLVED_QUESTIONS] must be the structured output from your pending question detection step, including IDs, text, severity scores, and blocking flags. [ANSWERED_TOPICS] is a critical guardrail: populate it with a summary of topics already resolved to prevent the model from circling back to dead ends. After generating the question, validate the output against the schema, check that the selected_question_id references a real entry in your unresolved list, and confirm the question does not overlap with any answered topic. For high-stakes investigations, route the generated question through a human review step before presenting it to the user or executing it via tool use.

The Follow-Up Question Generation Prompt is designed to be called by an application orchestrator, not exposed directly to end users. The orchestrator is responsible for maintaining the conversation state, tracking which questions have been asked and answered, and deciding when to invoke this prompt. The prompt expects a structured input containing the current conversation transcript, a list of previously identified unresolved questions, and any domain constraints or research goals. The application should call this prompt after each assistant response or when the system detects that the conversation has stalled on an unresolved thread.

To wire this into a production system, implement a state management layer that persists a pending_questions array across turns. Each entry should include a unique question ID, the original turn reference, the question text, a status enum (unresolved, partially_answered, addressed, retired), and a timestamp. Before calling the prompt, the orchestrator should filter out questions with status addressed or retired and serialize the remaining items into the [UNRESOLVED_QUESTIONS] placeholder. After receiving the model's output, validate the generated follow-up question against a checklist: (1) the question is not already answered in the transcript, (2) it does not duplicate a previously asked follow-up, (3) it is ranked by information gain with a clear rationale, and (4) it respects any [CONSTRAINTS] such as domain boundaries or prohibited question types. If validation fails, retry with the validation errors injected into the prompt as additional context, up to a maximum of two retries before falling back to a human reviewer or a simpler clarification question.

For model selection, use a capable reasoning model (such as GPT-4o, Claude 3.5 Sonnet, or Gemini 1.5 Pro) because the task requires cross-turn inference, information gain estimation, and avoidance of already-covered territory. Lighter models often fail on the 'already-answered' check and generate repetitive questions. Log every generated follow-up question along with its rationale, the input state snapshot, and the validation result. This trace data is essential for debugging dropped-question failures and for tuning the prompt's sensitivity to implicit answers. In high-stakes research or investigative domains, route generated questions through a human approval queue before they are shown to the user, especially when the question implies a sensitive line of inquiry or when the system confidence score falls below a configurable threshold. Avoid wiring this prompt directly into a user-facing message without state checks—sending a follow-up question that was already answered is the most common production failure mode and erodes user trust quickly.