Evidence Ranking with User Intent Alignment Prompt

Standard retrieval ranking optimizes for surface-level relevance to a query string. This prompt is for teams building conversational AI, search personalization, and task-oriented assistants where the user's underlying goal matters more than keyword overlap. It instructs the model to infer user intent, task context, and information need depth from the query and conversation history, then re-rank evidence passages by how well they serve what the user is actually trying to accomplish. Use this when your RAG pipeline retrieves relevant passages but the top-K results are not the most useful ones for the user's real task.

The ideal user is an engineering lead or RAG pipeline builder who can already retrieve relevant documents but sees a gap between retrieval relevance and answer usefulness. For example, a user asking 'How do I handle timeout errors in the payment API?' might retrieve passages about API timeouts in general, but the user's actual intent is troubleshooting a specific integration failure during payment processing. This prompt re-ranks evidence to prioritize passages about payment-specific timeout handling, error recovery patterns, and idempotency—even if those passages scored lower on pure keyword overlap. Required context includes the original query, conversation history (if multi-turn), and the retrieved evidence set with passage IDs. The prompt works best when retrieval recall is high but precision for the user's task is low.

Do not use this prompt for simple factoid lookup where intent is unambiguous—a query like 'What is the capital of France?' does not benefit from intent inference. Avoid it for latency-critical paths where the extra inference step is too expensive; intent-aligned re-ranking adds one model call per ranking operation. It is also not a substitute for poor retrieval: if your retriever is not surfacing relevant passages at all, fix retrieval recall before adding intent-aware ranking. For high-stakes domains such as medical or legal, pair this prompt with a grounding verification step to ensure re-ranked evidence still meets domain-specific authority and accuracy standards before it reaches answer generation.

This template is designed to be dropped into a RAG pipeline or search ranking system where surface-level relevance scoring is insufficient. It forces the model to first infer what the user is actually trying to accomplish—their task, depth of need, and decision context—before scoring each evidence passage. The output is a ranked list where position reflects intent alignment, not just keyword overlap. Use this when your retrieval set contains passages that are topically relevant but misaligned with the user's real goal, such as returning introductory material when the user needs a technical specification, or returning recent news when the user needs historical context.

textCopy
You are an evidence ranking specialist. Your job is to re-rank a set of retrieved evidence passages so the most useful passages appear first, based on the user's inferred intent, task context, and information need depth.

## INPUT

**User Query:** [USER_QUERY]

**User Context (if available):** [USER_CONTEXT]

**Retrieved Evidence Passages:**
[EVIDENCE_PASSAGES]

## INSTRUCTIONS

### Step 1: Infer User Intent
Analyze the user query and any provided context. Determine:
- **Task Type:** What is the user trying to do? (e.g., make a decision, learn a concept, troubleshoot an issue, compare options, find a specific fact, get a procedural guide)
- **Depth of Need:** Does the user need a quick answer, a detailed explanation, or exhaustive coverage?
- **Domain Familiarity:** Is the user likely a novice, practitioner, or expert in this domain?
- **Temporal Sensitivity:** Does the query require current information, historical context, or is time irrelevant?
- **Actionability:** Does the user need information they can act on immediately, or background understanding?

### Step 2: Score Each Passage
For each evidence passage, assign a score from 0 to 10 based on how well it serves the inferred intent:
- **Task Alignment (0-4 points):** Does this passage help the user accomplish their inferred task?
- **Depth Match (0-3 points):** Does the passage's level of detail match the user's inferred depth of need?
- **Audience Fit (0-2 points):** Is the passage written at an appropriate level for the user's inferred domain familiarity?
- **Temporal Fit (0-1 point):** Does the passage's recency or timeframe match the query's temporal sensitivity?

### Step 3: Rank and Explain
Produce a ranked list of all passages, ordered from highest to lowest intent-alignment score. For each passage, include:
- The passage ID
- The total score
- A one-sentence explanation of why this passage does or does not align with the inferred user intent

## OUTPUT FORMAT

First, state your inferred user intent in 2-3 sentences.

Then, output the ranked list in this exact format:

**Ranked Evidence:**
1. [Passage ID] | Score: [X.X/10] | [One-sentence rationale]
2. [Passage ID] | Score: [X.X/10] | [One-sentence rationale]
...

## CONSTRAINTS
- Do not skip any passage. Every retrieved passage must appear in the ranked list.
- If two passages serve the intent equally well, prefer the one with higher specificity.
- If a passage is actively misleading or contradicts the inferred intent, score it below 3 and explain why.
- Do not invent facts not present in the passages.

Adaptation notes: Replace [USER_CONTEXT] with session history, user profile signals, or an empty placeholder if unavailable. The [EVIDENCE_PASSAGES] block should include passage IDs and full text, formatted consistently (e.g., [P1] Title: ... Content: ...). If your system already has a user intent classifier upstream, you can replace Step 1 with the classifier's output and instruct the model to use that intent directly rather than re-inferring it. For production systems with strict latency budgets, consider splitting this into two calls: one for intent inference and one for scoring, caching the intent result across related queries.

Validation and risk controls: After the model returns a ranking, validate that every input passage ID appears exactly once in the output. If passage counts don't match, reject the output and retry with a stronger constraint instruction. For high-stakes domains like medical or legal search, route outputs scoring below 5/10 to a human reviewer before surfacing them to users. Log the inferred intent alongside the ranking for offline evaluation—this lets you measure whether intent misinference is the root cause of poor rankings over time. Run periodic eval checks comparing intent-aligned rankings against pure relevance baselines using user satisfaction metrics or click-through data.