Passage Re-Ranking Prompt with CoT Reasoning

This prompt is for AI architects and retrieval engineers who need auditable, step-by-step reasoning behind passage re-ranking decisions. Use it when a black-box relevance score is not enough and downstream systems or human reviewers need to understand why specific passages were prioritized. The prompt forces the model to assess each passage against query intent, entity mentions, and factual coverage before assigning a rank. It is designed for RAG pipelines where transparency reduces hallucination risk and supports debugging.

Do not use this prompt when latency budgets are under 200ms or when the retrieval set exceeds 20 passages, as the chain-of-thought overhead becomes prohibitive. This prompt assumes you already have a candidate passage set from a first-stage retriever. It does not perform retrieval itself. The ideal user is someone who can inspect the reasoning traces, calibrate the eval criteria, and decide when the transparency gain justifies the token and latency cost.

Before adopting this prompt, verify that your downstream answer synthesis step actually consumes the reasoning output. If the synthesis prompt only needs the final ranked list, consider a lighter-weight scoring prompt instead. Reserve this CoT re-ranker for debugging phases, high-stakes domains, or compliance workflows where the reasoning trail is itself a required artifact.

This template provides a complete re-ranking prompt that forces the model to reason about each passage's relevance before producing a final ranking. The chain-of-thought (CoT) structure makes the ranking logic auditable, which is essential for debugging retrieval quality, calibrating confidence thresholds, and generating explainability traces for downstream users or evaluators. Use this prompt when you need transparent evidence selection, not just a sorted list.

textCopy
You are a precise evidence-ranking specialist. Your task is to re-rank a set of retrieved passages by their relevance to a user query. You must reason step-by-step about each passage before assigning a final rank.

## INPUT

**User Query:**
[QUERY]

**Retrieved Passages:**
[PASSAGES]

## RE-RANKING INSTRUCTIONS

For each passage, perform the following reasoning steps before producing the final ranked list:

1. **Query Intent Analysis:** Identify the core entities, relationships, temporal constraints, and information needs expressed in the user query.
2. **Passage-by-Passage Assessment:** For each passage, evaluate:
   - **Entity Match:** Does the passage mention the key entities from the query?
   - **Intent Alignment:** Does the passage address the user's actual information need, or only surface-level keyword overlap?
   - **Factual Coverage:** What specific facts, claims, or data points in the passage directly bear on the query?
   - **Information Density:** Does the passage contain substantive, non-redundant information, or is it mostly filler?
   - **Credibility Signals:** Based on [CREDIBILITY_CRITERIA], does the passage appear authoritative, recent, and well-sourced?
3. **Comparative Ranking:** After assessing all passages individually, compare them head-to-head. Resolve ties by prioritizing passages that provide unique, complementary information. Flag any passages that contradict each other.
4. **Exclusion Justification:** For any passage you exclude from the final ranking, state a clear reason (e.g., irrelevant, redundant, outdated, low credibility).

## OUTPUT FORMAT

Return a JSON object with the following structure:

{
  "query_intent": "A concise summary of the user's core information need.",
  "passage_assessments": [
    {
      "passage_id": "[ID]",
      "entity_match_score": 0.0-1.0,
      "intent_alignment_score": 0.0-1.0,
      "factual_coverage_notes": "Specific facts relevant to the query.",
      "information_density": "high|medium|low",
      "credibility_notes": "Brief assessment based on provided criteria.",
      "relevance_rationale": "Concise explanation of why this passage is or is not relevant."
    }
  ],
  "ranked_passages": [
    {
      "rank": 1,
      "passage_id": "[ID]",
      "relevance_score": 0.0-1.0,
      "ranking_rationale": "Why this passage earned this rank relative to others."
    }
  ],
  "excluded_passages": [
    {
      "passage_id": "[ID]",
      "exclusion_reason": "Clear reason for exclusion."
    }
  ],
  "contradictions_flagged": [
    {
      "passage_id_a": "[ID]",
      "passage_id_b": "[ID]",
      "conflict_description": "What specific claims conflict."
    }
  ]
}

## CONSTRAINTS

- Do not fabricate facts not present in the passages.
- If no passage is sufficiently relevant, return an empty ranked_passages array and explain why in the query_intent field.
- Preserve all original passage IDs exactly as provided.
- [ADDITIONAL_CONSTRAINTS]

To adapt this template, replace [QUERY] with the user's question or search string. [PASSAGES] should be a JSON array of objects, each containing at minimum an id and text field. [CREDIBILITY_CRITERIA] is a critical placeholder: populate it with your domain-specific rules for source trustworthiness, such as publication date thresholds, author authority indicators, or document type preferences. The [ADDITIONAL_CONSTRAINTS] field lets you inject task-specific rules like token budgets, required source diversity, or domain terminology preferences without modifying the core reasoning structure.

Before deploying this prompt, validate that your passage objects include stable, unique IDs that survive your retrieval pipeline. The output JSON schema is designed for direct consumption by a downstream synthesis prompt or an evaluation harness. If you are operating in a regulated domain, route outputs where contradictions_flagged is non-empty or relevance_score falls below a calibrated threshold to a human reviewer. Test the prompt against a golden dataset of query-passage pairs with known relevance judgments to calibrate your scoring expectations before production use.