Retrieval Set Re-Ranking Prompt with Weighted Criteria

This prompt is designed for the specific job of re-ranking an initial retrieval set using multiple weighted evidence quality criteria. It is not a first-pass retrieval prompt. Use it when your vector store, keyword index, or hybrid retriever has already returned a candidate set of 10 to 50 passages and you need to order them by a composite score that balances relevance, recency, authority, specificity, and query alignment. The ideal user is a search engineer or RAG pipeline architect who has access to passage metadata—such as publication dates, source identifiers, or domain classifications—and needs an auditable, debuggable ranking before answer generation or evidence selection.

The prompt produces a final ranked list with per-passage score breakdowns showing exactly how each criterion contributed to the overall ranking. This makes it suitable for production systems where you must log why one passage was chosen over another, debug ranking anomalies, or demonstrate to compliance reviewers that evidence selection follows a defined policy. You should wire this prompt after your initial retrieval step but before any answer synthesis or citation selection. The weighted criteria are configurable, so you can adjust the importance of recency for time-sensitive domains like news or finance, or increase the authority weight for regulated industries like healthcare or legal tech.

Do not use this prompt for single-factor relevance scoring, for filtering without ranking, or when you lack the metadata required for multi-factor assessment. If you only need to filter passages above a relevance threshold, a simpler threshold-based filtering prompt will be faster and cheaper. If your retrieval set is smaller than 5 passages, the overhead of multi-factor scoring may not be justified. This prompt also assumes you already have a defined set of weighted criteria; if you are still calibrating which factors matter for your domain, start with the Multi-Factor Evidence Weighting Prompt Template to establish your weights before deploying this re-ranking prompt in production.

This prompt template is designed to be copied directly into your application code, test suite, or prompt management platform. It accepts an initial retrieval set and a set of weighted criteria, then returns a re-ranked list where each passage includes a composite score and a breakdown showing how each criterion contributed. Replace every square-bracket placeholder with your actual data before sending the prompt to the model. The template is model-agnostic but assumes the target model can follow structured output instructions and produce valid JSON.

textCopy
You are an evidence ranking specialist. Your task is to re-rank a set of retrieved passages using multiple weighted criteria. You will receive a query, a list of passages, and a set of scoring criteria with weights. For each passage, you must assign a score for every criterion, compute a weighted composite score, and return the passages in descending order of composite score.

## INPUT

**Query:** [USER_QUERY]

**Retrieved Passages:**
[PASSAGE_LIST]

**Scoring Criteria (with weights):**
[CRITERIA_LIST]

## INSTRUCTIONS

1. For each passage, evaluate it against every criterion listed above. Assign a score from [SCORE_RANGE] for each criterion, where [SCORE_RANGE_DESCRIPTION].
2. Compute the weighted composite score for each passage by multiplying each criterion score by its weight and summing the results.
3. Include a brief justification (1-2 sentences) for each criterion score, citing specific content from the passage.
4. Rank all passages in descending order of composite score.
5. If two passages have identical composite scores, break the tie using [TIEBREAK_RULE].
6. Flag any passage that falls below the minimum threshold of [MINIMUM_THRESHOLD] with a `discard: true` field and a reason.

## OUTPUT FORMAT

Return a JSON object with the following structure:

{
  "ranked_passages": [
    {
      "passage_id": "string",
      "composite_score": number,
      "rank": integer,
      "criteria_scores": {
        "[CRITERION_NAME]": {
          "score": number,
          "justification": "string"
        }
      },
      "discard": boolean,
      "discard_reason": "string or null"
    }
  ],
  "ranking_metadata": {
    "total_passages": integer,
    "discarded_count": integer,
    "score_range_used": "string",
    "tiebreaks_applied": integer
  }
}

## CONSTRAINTS

[CONSTRAINTS]

## EXAMPLES

[FEW_SHOT_EXAMPLES]

Adaptation guidance: Replace [USER_QUERY] with the original search query. [PASSAGE_LIST] should be a JSON array of objects, each containing at minimum an id and text field. [CRITERIA_LIST] must be a JSON array of objects with name and weight keys, where weights sum to 1.0. [SCORE_RANGE] and [SCORE_RANGE_DESCRIPTION] define your scoring scale—common choices are 0-10 or 1-5 with explicit anchors. [TIEBREAK_RULE] should specify a deterministic secondary sort, such as "prefer the passage with the higher relevance criterion score." [MINIMUM_THRESHOLD] is the composite score below which passages are discarded. [CONSTRAINTS] is where you add domain-specific rules, such as "penalize passages older than 90 days" or "require at least one citation from a peer-reviewed source." [FEW_SHOT_EXAMPLES] should include 2-3 worked examples showing the expected scoring behavior, especially for edge cases like near-miss passages or conflicting evidence.

Before deploying this prompt, validate that your model reliably produces the output schema. Run at least 20 test cases through the prompt and check for schema compliance, score range adherence, and tiebreak consistency. For high-stakes ranking pipelines, add a post-processing validation step that rejects malformed JSON, scores outside the defined range, or missing justifications. If the ranking output feeds directly into answer generation or user-facing results, implement a human review gate for the first several hundred production runs until score calibration is confirmed against human judgments.

This prompt is designed to sit between your initial retrieval step and the final answer generation step in a RAG pipeline. It expects a pre-fetched set of candidate passages and a user query, and it returns a ranked list with per-criterion score breakdowns. The harness should treat this prompt as a scoring microservice: it receives a batch of passages, applies weighted criteria, and returns a structured ranking that downstream components can consume without re-parsing.

To integrate this into production, wrap the prompt call in a function that validates the output schema before passing results forward. The expected output is a JSON array of ranked passages, each containing a passage_id, rank, total_score, and a criteria_scores object with keys matching your configured weights (e.g., relevance, recency, authority, specificity). Implement a schema validator that checks: (1) all input passage IDs appear exactly once in the output, (2) ranks are sequential integers starting at 1, (3) total scores are numeric and within expected ranges, and (4) criteria score objects contain only the expected keys. If validation fails, retry once with the same input and a stronger constraint instruction appended to the prompt. If the second attempt also fails, log the malformed output and fall back to the original retrieval order with a warning flag.

For observability, log the input passage count, the output rank distribution, and the per-criterion score ranges on each call. This lets you detect criteria weight dominance—where one criterion (often relevance) overwhelms the others and makes the weighting effectively single-factor. Set up a periodic eval that runs a fixed query set through the re-ranker and checks rank stability: if re-ranking the same passages produces materially different orders across calls, your temperature may be too high or your prompt instructions too loose. For high-stakes domains, route a sample of re-ranked outputs to a human review queue where reviewers can flag misranked passages and feed corrections back into your eval dataset.

When wiring this into a full RAG pipeline, place the re-ranker after retrieval and before context window packing. If your initial retrieval returns more passages than fit in the model's context, use the re-ranked scores to select the top-N passages for answer generation. Avoid the temptation to re-rank the entire corpus—this prompt is designed for re-ranking a candidate set, not for primary retrieval. For latency-sensitive applications, consider batching multiple queries into a single re-ranking call if your prompt template supports it, but validate that batch size does not degrade ranking quality through attention dilution.