Maximum Marginal Relevance (MMR)

MMR operates on a linear combination of two competing scores. The algorithm calculates a marginal relevance score for each candidate document. This score is the document's similarity to the query minus the maximum similarity to any document already in the selected set. The lambda parameter controls the weight of this trade-off. A lambda of 1.0 maximizes pure relevance, while a lambda of 0.0 maximizes pure diversity, effectively penalizing any document that resembles an already-chosen one.

MMR builds the final ranked list one document at a time using a greedy selection process. It begins by selecting the single most relevant document for the query. In each subsequent iteration, it re-scores all remaining candidates against the updated set of chosen documents. The algorithm selects the candidate with the highest MMR score and adds it to the output set. This process repeats until the desired number of results is reached, ensuring each new addition contributes maximum novel information.

The λ (lambda) parameter is a user-defined constant between 0 and 1 that explicitly controls the diversity-relevance balance. When λ is high (e.g., 0.8), the algorithm strongly favors query relevance, producing a result set that may contain redundant information. When λ is low (e.g., 0.2), the algorithm aggressively penalizes similarity to prior selections, producing a highly diverse set that may drift from the core query intent. Tuning this parameter is critical for domain-specific performance.

The novelty component of MMR is calculated by taking the maximum similarity between a candidate document and all documents already in the selected set. This is a conservative penalty: a candidate is judged by its closest neighbor in the output set. This prevents the algorithm from selecting a document that is highly relevant but nearly identical to a previously chosen result. The approach effectively clusters similar documents and selects only the single best representative from each cluster.

In Retrieval-Augmented Generation, MMR is a standard post-retrieval step applied to the initial set of candidates from a vector store. After an Approximate Nearest Neighbor (ANN) search returns the top-K most similar chunks, MMR re-ranks them to maximize the information density of the final context window. This prevents the language model from receiving five nearly identical chunks, which wastes the limited context window and provides no additional factual grounding for generation.

The MMR score for a candidate document D_i is defined as: MMR = λ * Sim(D_i, Q) - (1 - λ) * max_{D_j ∈ S} Sim(D_i, D_j). Here, Q is the query, S is the set of already-selected documents, and Sim is a similarity function like cosine similarity. The first term rewards relevance to the query. The second term penalizes similarity to the most similar document already chosen. This formula is computed iteratively until the desired number of documents is selected.

An algorithm that merges multiple ranked result lists into a single consensus ranking without requiring relevance scores. It assigns a reciprocal score based on each document's rank position, naturally penalizing low-ranked items. Unlike MMR, which operates on a single result set to balance relevance and novelty, RRF is a fusion method designed to combine outputs from different retrieval sources such as vector search and BM25.

A two-stage retrieval refinement where a computationally expensive cross-encoder model processes the query and each candidate document jointly to produce a precise relevance score. This is applied only to the top results from a faster initial retrieval. While MMR re-ranks by penalizing inter-document similarity, a cross-encoder re-ranks by deeply analyzing query-document interaction, often serving as the final scoring step before MMR is applied for diversification.

A retrieval approach that combines dense vector similarity search with sparse keyword-based retrieval such as BM25. This leverages the strengths of both semantic understanding and exact term matching. MMR is frequently applied as a post-processing step on the combined candidate set from a hybrid search pipeline to ensure the final results presented to the user or language model are both relevant and non-redundant.

The identification and removal of duplicate or near-duplicate content from an indexing pipeline. While MMR reduces semantic redundancy in a ranked list by penalizing similar documents, de-duplication is a hard pre-processing or post-processing filter that removes exact or near-exact copies. MMR handles the gray area of documents that are topically similar but not identical, preventing information overlap in the final context window.

A class of algorithms that trade a small amount of accuracy for significant gains in speed when finding the most similar vectors in a high-dimensional space. ANN is the initial retrieval engine that generates the candidate set. MMR is a subsequent re-ranking operation applied to this candidate set. The efficiency of the ANN algorithm directly determines the size and quality of the pool that MMR can diversify.

A measure of the ratio of unique, substantive facts to total text length within a content chunk. High density improves retrieval precision by reducing noise. MMR complements this by ensuring that even high-density chunks are not selected if they are semantically redundant with already-chosen documents. Together, they maximize the total unique information delivered within a limited token budget for a language model.