Cross-Encoder Reranking Prompt Template

This prompt is designed for the specific job of fine-grained passage ordering when your vector or keyword retrieval system has returned a candidate set that is too large or too coarsely scored for direct context assembly. The ideal user is a search engineer or RAG pipeline architect who needs to transform a set of, for example, 20-50 retrieved passages into a strictly relevance-ordered list of the top 5-10, complete with explicit justifications for each ranking decision. The required context is a well-defined user query and a set of candidate passages, each with a unique identifier. The core value is not just the final order, but the pairwise reasoning trace, which is invaluable for debugging retrieval failures and calibrating upstream embedding models.

You should use this template when the cost of including irrelevant or suboptimal context in your generation step is high—for instance, in a legal research tool where a missed precedent is a critical failure, or in a technical support copilot where the correct code snippet must be found among many similar-looking results. The pairwise comparison structure forces the model to make relative judgments ('Passage A is more relevant than Passage B because...'), which is a closer match to the cross-encoder architecture than pointwise scoring. The output is explicitly designed to be consumed by a downstream context window budget allocator or to provide a ranked list for an answer generation step. However, you must not use this prompt when your task is a simple binary filter (relevant/irrelevant), as the pairwise overhead is wasteful. Similarly, avoid it when your latency budget is under a few hundred milliseconds, as the O(n²) comparison depth makes it unsuitable for real-time search over large sets.

Before implementing, verify that your passage count and length fit within a single model call with adequate comparison depth. If you have 100 passages, a full pairwise comparison is impractical; you should instead use a pointwise scoring prompt or a multi-stage filtering pipeline first. The next step after reading this section is to examine the prompt template itself, paying close attention to the [PASSAGES] and [QUERY] placeholders, and then review the implementation harness to understand how to validate the output schema and integrate it into a production RAG pipeline.

The following prompt template is designed to replicate the behavior of a cross-encoder reranker using a general-purpose LLM. Instead of outputting a single relevance score, the model is instructed to compare every passage against every other passage, building a ranked list from the ground up. This pairwise comparison method is more compute-intensive but often yields more accurate and explainable rankings than pointwise scoring, especially when the differences between top passages are subtle. The template is self-contained and can be copied directly into a notebook or application harness. All dynamic inputs are represented as square-bracket placeholders.

textCopy
You are a precise relevance ranking engine. Your task is to rank a set of passages by their relevance to a given query using pairwise comparison.

## QUERY
[QUERY]

## PASSAGES TO RANK
[PASSAGES]

## INSTRUCTIONS
1. Compare every passage to every other passage in a pairwise fashion.
2. For each pair, determine which passage is more relevant to the query. Base your decision on:
   - Direct topical alignment with the query's core intent.
   - Specificity of the information provided.
   - Factual density and absence of fluff.
3. After all comparisons, produce a fully ordered list from most relevant (rank 1) to least relevant.
4. For each passage in the final ranking, provide a concise justification explaining its position relative to the passages ranked immediately above and below it.

## OUTPUT FORMAT
Return a single JSON object with the following structure:
{
  "ranked_passages": [
    {
      "rank": 1,
      "passage_id": "[ID_FROM_INPUT]",
      "justification": "Why this passage is ranked here."
    },
    ...
  ]
}

## CONSTRAINTS
- Do not skip any passage. Every input passage must appear in the output.
- Do not assign ties. Every rank must be unique.
- Base your ranking strictly on the provided text, not on external world knowledge.

To adapt this template, replace [QUERY] with the user's search string or natural language question. The [PASSAGES] placeholder should be replaced with a pre-formatted list of candidate passages, each assigned a unique identifier. A robust format for this input is a JSON array of objects, each with an id and text field. When integrating this into a RAG pipeline, this prompt should be placed after a fast, high-recall retrieval step (like vector search) and before the final context assembly for answer generation. For production use, always validate the output JSON against the expected schema and implement a retry mechanism with a stricter format reminder if the model deviates.

The cross-encoder reranking prompt template is designed to sit between your initial retrieval stage and your answer generation stage. In a typical RAG pipeline, you first retrieve a candidate set of passages—often 20 to 100 results from a vector or hybrid search—and then use this prompt to produce a relevance-ordered list with pairwise comparison justifications. The prompt expects a [QUERY] and a [PASSAGES] list as input, and it returns a ranked output with scores and reasoning. Wire this into your application as a synchronous post-retrieval step: after retrieval but before context assembly for the generator. The model choice matters here—use a model with strong instruction-following and reasoning capabilities (such as GPT-4, Claude 3.5 Sonnet, or a fine-tuned Llama 3 variant) because pairwise comparison requires careful attention to semantic nuance. Lighter models often produce inconsistent rankings or skip justifications under load.

Validation is the most critical part of the harness. Before passing the reranked output downstream, validate that every passage in the output exists in the input set (no hallucinated passages), that the ranking is complete (all input passages appear in the output unless explicitly excluded with a reason), and that each justification references specific content from both the query and the passage. Implement a structural validator that checks the output schema—expect a JSON array of objects with passage_id, rank, relevance_score, and justification fields. If validation fails, retry once with the same prompt and add a [CONSTRAINTS] note emphasizing the validation error. After two failures, log the raw output, fall back to the original retrieval order, and flag the query for offline review. For high-stakes domains like legal or medical search, route validation failures to a human review queue rather than silently degrading.

Logging and observability are essential for production. Capture the query, the input passage count, the output ranking, per-passage scores, and the latency of the reranking call. Track reranking-specific metrics: ranking inversion rate (how often the reranker disagrees with the original retrieval order), justification quality flags (empty or generic justifications), and truncation events (when the prompt exceeds the context window and passages are dropped). If you're using this prompt in a system with user-facing citations, verify that the top-ranked passages after reranking actually contain the information needed to answer the query—this is your grounding check. Build an eval harness that compares reranked orderings against human-judged relevance labels on a golden dataset of query-passage pairs, and monitor nDCG or precision@k drift across prompt versions. When you update the prompt template, run this eval suite before deployment and set a minimum nDCG threshold as a release gate. Finally, consider cost: cross-encoder reranking with an LLM is more expensive per query than embedding-based reranking, so apply this prompt only to the top-N candidates (typically 20–30) rather than the full retrieval set, and cache reranking results for repeated queries where the passage set hasn't changed.