Session Context Scoring Prompt for Query Fusion

This prompt is for retrieval pipeline architects who need to fuse context from a multi-turn conversation into a single, high-quality search query. The core job-to-be-done is to score the relevance of each prior user and assistant turn against the user's current question, then use those scores to weight the contribution of each turn when generating a fused, context-aware query. You should use this prompt when your RAG system suffers from 'context dilution'—where naively stuffing the entire conversation history into a query rewriter introduces noise, exceeds context windows, or gives equal weight to irrelevant small talk and critical prior instructions. The ideal user is an engineer who already has a working conversational retrieval system and needs to add a selective attention mechanism before the final query rewrite step.

This prompt is not a standalone query rewriter. It is a scoring and weighting component that should sit upstream of a query generation prompt. Do not use this prompt if your application consists of single-turn queries, if you have no mechanism to pass weighted context to a downstream rewriter, or if your conversation history is consistently short enough to include in its entirety without degradation. The prompt requires a structured conversation history as input, with each turn clearly labeled by role and index. It also requires a defined output schema that maps turn indices to relevance scores and weights. Without this structured input, the model cannot produce calibrated scores, and without a downstream consumer for the weights, the output has no operational value.

Before implementing this prompt, ensure you have a calibration harness in place. Session context scoring is prone to recency bias, where the model over-weights the most recent turns regardless of their actual relevance to the current query. Your eval suite should include test cases where the critical context is deliberately placed several turns back, behind filler exchanges. Measure whether the scoring prompt correctly assigns higher weights to semantically relevant turns over temporally proximate but irrelevant ones. If your harness cannot detect this failure mode, you risk building a system that appears functional but silently ignores important user instructions issued earlier in the session.

This prompt template is the core instruction set for the Session Context Scoring Prompt for Query Fusion. It takes the current user query, the full conversation history, and a set of scoring criteria, then outputs a relevance-weighted fusion of prior context into a single, self-contained retrieval query. The template uses square-bracket placeholders for all dynamic inputs, making it safe to copy directly into your prompt management system or codebase. Before using it in production, you must define your scoring dimensions, set a minimum relevance threshold to exclude noise, and decide on a weighting strategy for the final fusion.

textCopy
You are a retrieval query rewriter for a conversational AI system. Your task is to fuse relevant context from the conversation history into the current user query to produce a single, self-contained retrieval query.

**Current Query:**
[CURRENT_QUERY]

**Conversation History:**
[CONVERSATION_HISTORY]

**Scoring Dimensions:**
[SCORING_DIMENSIONS]

**Instructions:**
1. For each turn in the conversation history, assign a relevance score from 0.0 to 1.0 against the current query for each scoring dimension listed above.
2. Calculate a composite relevance score for each turn by [WEIGHTING_STRATEGY].
3. Exclude any turn with a composite score below [MINIMUM_RELEVANCE_THRESHOLD].
4. For turns that meet the threshold, extract the key entities, constraints, and unresolved references that are still relevant to the current query.
5. Fuse the extracted context into the current query to produce a single, self-contained retrieval query. Do not introduce information not present in the history or current query.
6. Output the result as a JSON object with the following structure:

**Output Schema:**
{
  "fused_query": "string",
  "turn_scores": [
    {
      "turn_index": integer,
      "turn_text": "string",
      "dimension_scores": {
        "[DIMENSION_NAME]": float
      },
      "composite_score": float,
      "included": boolean,
      "extracted_context": "string or null"
    }
  ],
  "excluded_turns": integer,
  "fusion_notes": "string"
}

**Constraints:**
- Do not hallucinate entities or constraints not present in the history.
- If no turns meet the relevance threshold, return the current query unchanged as the fused query.
- If the current query is already self-contained, do not force irrelevant context into the rewrite.

To adapt this template, start by defining your scoring dimensions. Common dimensions include topical overlap, entity co-reference, temporal proximity, and constraint persistence. For a customer support system, you might score on issue continuity and product mention overlap. For a research assistant, you might score on conceptual relevance and evidence chain continuity. Next, choose a weighting strategy: simple averaging works for balanced dimensions, while weighted sums let you prioritize recency or entity persistence. Set the minimum relevance threshold based on your precision-recall trade-off; a threshold of 0.3 is a reasonable starting point, but calibrate it against your eval dataset. Finally, wire the output into your retrieval pipeline by passing fused_query to your search backend and logging turn_scores for observability. If you are operating in a high-stakes domain where incorrect context fusion could produce misleading answers, add a human review step for queries where the maximum composite score falls in an ambiguous range, or where the fusion notes flag uncertainty.

Integrating the Session Context Scoring Prompt into a query fusion pipeline requires treating it as a deterministic pre-retrieval scoring step, not a standalone chat interaction. The prompt receives the current user query and a structured array of prior conversation turns, each with a unique turn ID, timestamp, and text. It returns a JSON object mapping each turn ID to a relevance score between 0.0 and 1.0, along with a brief justification. This output feeds directly into a weighted query fusion module that constructs the final retrieval query by blending the current query with high-scoring prior turns. The harness must enforce strict input and output schemas to prevent malformed scores from corrupting downstream retrieval.

Build the harness with these concrete checks: (1) Validate that every input turn ID appears in the output mapping and that no hallucinated IDs are introduced. (2) Clamp all scores to the [0.0, 1.0] range and reject any response where scores fall outside this interval. (3) Implement a recency bias detector that flags if the model consistently assigns the highest scores to the most recent turns regardless of semantic relevance—this indicates calibration drift. (4) Log the raw prompt, model response, and parsed scores for every call to enable offline calibration analysis. (5) Set a minimum score threshold (e.g., 0.3) below which turns are excluded from fusion to prevent noise injection. For model choice, use a fast, instruction-tuned model such as GPT-4o-mini or Claude 3.5 Haiku; latency matters here because this step sits on the critical path before retrieval. If the model returns invalid JSON after one retry, fall back to a recency-only weighting scheme rather than blocking the user.

Avoid wiring this prompt directly into a user-facing chat loop without caching. Session context scoring should be recomputed only when a new user turn arrives, not on every retrieval attempt. Cache the score map per turn and invalidate it only when the conversation history changes. For high-traffic systems, consider batching scoring requests for multiple concurrent sessions. Finally, run periodic offline evaluations using a golden dataset of session-turn pairs with human-annotated relevance scores to detect scoring drift. If the model's agreement with human annotators drops below a defined threshold, recalibrate the prompt or switch to a simpler recency-decay baseline until the issue is resolved.