Maximal Marginal Relevance Rerank Prompt

This prompt is for RAG system builders who need to assemble a context window from a large set of retrieved passages but are hitting a wall with redundancy. When a vector store returns ten chunks that all say the same thing in slightly different words, the language model loses the signal it needs to answer complex, multi-aspect questions. The job-to-be-done is to rerank those passages so the final context set maximizes both relevance to the query and novelty relative to the passages already selected. The ideal user is an AI engineer or search engineer who already has a working retrieval pipeline and needs a controllable diversity mechanism before answer generation.

Use this prompt when your retrieval results exhibit high semantic overlap, when users complain about shallow answers that miss nuance, or when you need to cover multiple facets of a query within a fixed token budget. It is particularly effective for exploratory questions ('What are all the causes of X?'), comparative questions ('How does A differ from B?'), and regulatory or due-diligence research where missing a distinct piece of evidence is worse than including a slightly less relevant one. The prompt expects a list of passages with relevance scores and a query, and it returns a reranked list with both relevance and novelty scores so you can tune the diversity trade-off yourself.

Do not use this prompt when your retrieval pipeline already returns highly diverse results, when latency constraints prohibit an extra LLM call before answer generation, or when the downstream task requires strict rank preservation from the initial retriever. It is also the wrong tool when you need to deduplicate near-identical passages—use a dedicated deduplication prompt for that. If your application is in a regulated domain such as healthcare or legal, this prompt should feed into a human review step where the reranked passage set is inspected for evidence loss before it reaches the answer generation stage. The next step after reading this section is to review the prompt template, wire it into your context assembly harness, and define the eval criteria that will tell you whether the diversity gain is worth the added latency.

The following prompt template applies Maximal Marginal Relevance (MMR) to a set of retrieved passages. It instructs the model to iteratively select passages that are both relevant to the query and novel compared to already-selected passages. The template uses square-bracket placeholders for all dynamic inputs, making it straightforward to integrate into a RAG pipeline where retrieval returns a candidate set that needs diversity filtering before answer generation.

textCopy
You are a context reranking engine. Your task is to apply Maximal Marginal Relevance (MMR) to a set of retrieved passages, selecting a diverse subset that balances relevance to the query with novelty relative to already-selected passages.

## INPUTS

**Query:** [QUERY]

**Candidate Passages:**
[PASSAGES]

**Diversity Parameter (λ):** [LAMBDA]

**Target Selection Count (k):** [K]

## INSTRUCTIONS

1. For each candidate passage, assess its relevance to the query on a scale of 0.0 to 1.0, where 1.0 is perfectly relevant.
2. Initialize an empty set of selected passages.
3. Iteratively select the next passage that maximizes the MMR score:
   - MMR = λ × relevance(passage, query) - (1 - λ) × max_similarity(passage, already_selected)
   - Where max_similarity is the highest cosine similarity between the candidate passage and any already-selected passage.
4. Stop when you have selected [K] passages or no remaining candidates meet a minimum relevance threshold of [MIN_RELEVANCE_THRESHOLD].
5. For each selected passage, compute and report:
   - The relevance score to the query
   - The novelty score (1 - max_similarity to already-selected passages at time of selection)
   - The final MMR score at selection time

## OUTPUT SCHEMA

Return a JSON object with this exact structure:
{
  "query": "[QUERY]",
  "lambda": [LAMBDA],
  "total_candidates": <integer>,
  "selected_count": <integer>,
  "ranked_passages": [
    {
      "rank": <integer starting at 1>,
      "passage_id": "<original passage identifier>",
      "passage_text": "<full passage text>",
      "relevance_score": <float 0.0-1.0>,
      "novelty_score": <float 0.0-1.0>,
      "mmr_score": <float>,
      "selection_rationale": "<brief explanation of why this passage was selected at this rank>"
    }
  ],
  "excluded_passages": [
    {
      "passage_id": "<original passage identifier>",
      "relevance_score": <float>,
      "exclusion_reason": "<redundancy | below_threshold | not_reached>"
    }
  ],
  "diversity_assessment": "<summary of how well the selected set covers distinct aspects of the query>"
}

## CONSTRAINTS

- Preserve the original passage text exactly. Do not summarize or paraphrase.
- If two passages have identical relevance scores, prefer the one with higher novelty.
- If [LAMBDA] is close to 1.0, prioritize relevance over diversity. If close to 0.0, prioritize diversity.
- Report scores to 3 decimal places.
- If fewer than [K] passages meet the minimum relevance threshold, select only those that do and note the shortfall.

To adapt this template, replace the placeholders with your pipeline's actual values. [QUERY] should be the original user question or search query. [PASSAGES] should be a JSON array of objects, each with an id and text field. [LAMBDA] controls the relevance-diversity trade-off—start with 0.7 for balanced results and tune based on your use case. [K] sets the target number of passages to retain, typically matching your context window budget. [MIN_RELEVANCE_THRESHOLD] prevents low-quality passages from being selected purely for novelty; 0.3 is a reasonable starting point. Before deploying, validate that the output JSON matches the schema exactly, and run a few test queries to confirm that the diversity assessment accurately reflects the selected set's coverage.

The MMR rerank prompt is not a standalone step—it sits between retrieval and answer generation in your RAG pipeline. The typical call sequence is: (1) retrieve an initial candidate set from your vector store or hybrid search, (2) pass those candidates plus the original query into this prompt, (3) receive a reranked list with relevance and novelty scores, (4) apply a threshold or top-k cut to select the final context, and (5) feed that filtered context into your answer-generation prompt. The prompt expects a structured input containing the query string and an array of passage objects, each with at minimum an id and text field. You can extend the passage schema with metadata like source, date, or authority if your downstream answer prompt needs citation provenance.

Validation and retry logic. Parse the model's JSON output immediately after generation. Validate that every passage ID in the output exists in the input set, that relevance_score and novelty_score are numbers between 0 and 1, and that the rank field is a sequential integer starting at 1 with no gaps. If validation fails, retry once with the same prompt plus a brief error note appended to the system message—something like: 'Previous output failed validation: [specific error]. Return valid JSON only.' If the second attempt also fails, log the raw output, fall back to the original retrieval order, and fire an alert. For high-stakes domains, add a human review step before the reranked context reaches answer generation, especially when the MMR lambda parameter is tuned aggressively toward novelty and risks dropping highly relevant but redundant passages.

Model choice and latency budget. This prompt works well with models that have strong instruction-following and JSON output capabilities—GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro are solid defaults. For lower latency or cost, smaller models like Claude 3 Haiku or GPT-4o-mini can handle the task if you constrain the input passage count to 20 or fewer and keep the output schema tight. If you're reranking 50+ passages, consider batching: split the candidate set into overlapping windows of 15-20 passages, run MMR on each window independently, then merge results with a final deduplication pass. Always log the lambda value, input passage count, output passage count, and per-call latency so you can tune the trade-off between rerank quality and pipeline speed.

Threshold tuning and eval. The relevance_score and novelty_score fields are your primary tuning levers. Start by logging score distributions across a few hundred queries to understand your retrieval baseline. Set an initial relevance floor (e.g., 0.4) and a novelty floor (e.g., 0.3), then run a side-by-side eval comparing answer quality with and without MMR reranking. Use an LLM judge or human review to score answer completeness and faithfulness. If you see answer quality drop, your novelty threshold may be too aggressive—back it off. If you see no improvement, your retrieval set may already be diverse enough, and MMR reranking may be adding latency without value. Store the reranked passage list alongside each query in your trace store so you can replay and debug specific cases later.

What to avoid. Don't use this prompt as a replacement for retrieval quality improvements—if your initial retrieval is returning mostly irrelevant passages, MMR reranking won't fix it. Don't set the lambda parameter to an extreme (0.0 or 1.0) without testing; pure relevance ranking duplicates your retriever's job, and pure novelty ranking can surface irrelevant but unique passages. Don't skip the output validation step—model JSON errors in production are common, and a single malformed rerank output can break your entire answer-generation pipeline. Finally, don't treat the MMR scores as absolute truth; they're model estimates and should be calibrated against your own eval data before you rely on them for automated threshold decisions.