Passage Ranking Prompt for RAG Retrieval Sets

This prompt is designed for search and retrieval engineers who need to transform a noisy set of retrieved passages into a clean, ranked list before answer synthesis. It sits between retrieval and generation in a RAG pipeline. Use it when your vector or keyword search returns multiple chunks with overlapping content, varying credibility, and uneven information density. The prompt forces the model to rank passages by relevance to the user query, source credibility, and factual density while handling ties, near-duplicates, and context window constraints.

The ideal user is an AI engineer or backend developer who already has a working retrieval step but observes that the top-k results are not optimally ordered for downstream synthesis. You should have access to passage text, source metadata (title, date, author), and the original user query. The prompt works best when you can provide at least 5–20 candidate passages and need to select a smaller subset for a constrained context window. It is particularly valuable when your retrieval mix includes sources of uneven authority—such as internal documentation, user forums, and vendor docs—where credibility weighting matters as much as semantic similarity.

Do not use this prompt when you have a single retrieved passage, when retrieval already returns perfectly ordered results, or when downstream latency budgets cannot absorb an additional model call. It is also the wrong tool if your passages are already homogeneous in credibility and density, or if you need real-time ranking over thousands of candidates—this prompt is designed for re-ranking a manageable candidate set, not for full-scale retrieval. If your primary need is deduplication without relevance ordering, use the Multi-Passage Deduplication and Selection Prompt instead. If you need to decide whether the evidence is sufficient to answer at all, pair this prompt with the Evidence Sufficiency Check Prompt before proceeding to synthesis.

Before deploying, ensure you have a validation layer that checks the output schema—ranked passage IDs, scores, and exclusion reasons—and a retry path for malformed responses. For high-stakes domains like healthcare or legal, always log the ranking decisions for auditability and consider human review of the ranked set before it feeds into answer generation. Start by testing this prompt against a golden dataset of query-passage pairs with known relevance judgments to calibrate whether the model's ranking aligns with your domain-specific definition of 'best evidence.'

This prompt template is designed to be pasted directly into your system or user message for a passage ranking step in a RAG pipeline. It takes a user query and a set of retrieved passages as input, and it produces an ordered list of passages ranked by relevance, credibility, and information density. The output is structured JSON, making it suitable for direct consumption by a downstream synthesis prompt or for logging and evaluation. The square-bracket placeholders must be replaced with your specific query, passages, and constraints before execution.

textCopy
You are a precise evidence ranking engine for a retrieval-augmented generation (RAG) system. Your only job is to rank the provided passages by their relevance, credibility, and information density for answering the user's query. Do not answer the query. Do not synthesize information. Only rank and explain.

## User Query
[USER_QUERY]

## Retrieved Passages
[RETRIEVED_PASSAGES]

## Ranking Criteria
1. **Relevance**: How directly does the passage address the core intent and entities of the user query?
2. **Credibility**: Consider the source authority, publication date, and factual consistency. A passage from a dated or unknown source should be ranked lower.
3. **Information Density**: Prioritize passages with high factual content and low redundancy. A short, fact-packed passage should outrank a long, verbose one.

## Constraints
- [CONSTRAINTS]
- If a passage is a near-duplicate of a higher-ranked passage, exclude it and note the reason.
- If no passage is sufficiently relevant, return an empty list.
- You must operate within a total output token budget of [MAX_OUTPUT_TOKENS].

## Output Schema
Return a valid JSON object with the following structure. Do not include any text outside the JSON object.
{
  "ranked_passages": [
    {
      "passage_id": "string",
      "rank": integer,
      "relevance_score": float (0.0 to 1.0),
      "credibility_score": float (0.0 to 1.0),
      "information_density_score": float (0.0 to 1.0),
      "composite_score": float (0.0 to 1.0),
      "justification": "A concise, evidence-based reason for this ranking.",
      "exclusion_reason": null
    }
  ],
  "excluded_passages": [
    {
      "passage_id": "string",
      "exclusion_reason": "One of: 'near_duplicate', 'below_relevance_threshold', 'token_budget_exceeded'"
    }
  ]
}

## Examples
[FEW_SHOT_EXAMPLES]

To adapt this template, start by replacing [USER_QUERY] with the exact user question and [RETRIEVED_PASSAGES] with a structured list of your retrieval results, ensuring each passage has a unique passage_id. The [CONSTRAINTS] placeholder is critical for enforcing domain-specific rules, such as 'Prefer peer-reviewed sources from the last 5 years' or 'Exclude passages from competitor domains.' Use the [FEW_SHOT_EXAMPLES] placeholder to provide 1-3 examples of ideal input-output pairs, which dramatically improves ranking consistency and score calibration. For high-stakes applications, the composite_score should be used as a threshold to filter passages before they reach the answer synthesis step, and all outputs should be logged for offline evaluation of ranking quality.

The passage ranking prompt is not a standalone artifact—it is a component inside a retrieval pipeline. Wire it after your initial retrieval step (vector, keyword, or hybrid) returns a candidate set of passages. The prompt receives the user query and the raw candidate passages, then returns a ranked, scored, and deduplicated list. Your application should treat this prompt as a re-ranking and filtering stage before answer synthesis. Do not send every retrieved chunk to the ranking prompt; pre-filter to a reasonable candidate count (20–50 passages) to stay within context window limits and control latency. If your initial retrieval returns hundreds of passages, apply a lightweight first-pass filter (BM25 score threshold, metadata filter, or embedding similarity cutoff) before invoking this prompt.

Integration pattern: Call the ranking prompt as a synchronous step between retrieval and synthesis. The output JSON should be validated against a strict schema before downstream use. Expect fields: ranked_passages (ordered list with passage_id, rank, relevance_score, credibility_score, information_density_score, justification), excluded_passages (with passage_id and exclusion_reason), and tie_groups (groups of passages with statistically indistinguishable scores). Validate that every passage_id in the input appears exactly once across ranked_passages and excluded_passages. If validation fails, retry once with the same input and a stronger instruction to produce complete output. If the second attempt fails, log the failure, fall back to a simpler ranking heuristic (e.g., cosine similarity sort), and alert the operations channel. Never pass unvalidated ranking output to the synthesis step.

Model choice and latency: Use a model with strong instruction-following and JSON output capabilities (GPT-4o, Claude 3.5 Sonnet, or equivalent). For high-throughput pipelines, consider a smaller fine-tuned model if you have ranking-labeled data. Set response_format to json_object or equivalent structured output mode. Implement a timeout of 5–10 seconds; if the model does not return within the window, fall back to the heuristic ranking and log the timeout for capacity planning. Human review placement: For high-stakes domains (healthcare, legal, finance), insert a human review step when the ranking prompt flags unresolved ties in critical passages, when credibility scores fall below a configured threshold, or when the prompt's confidence calibration score drops below 0.7. Route these cases to a review queue rather than proceeding automatically to synthesis. Observability: Log the full prompt input, output, validation result, retry count, and fallback decision to your tracing system. This trace data is essential for debugging ranking regressions, calibrating confidence thresholds, and comparing prompt versions over time.