Maximum Marginal Relevance (MMR)

MMR balances two competing objectives using a mixing parameter λ (lambda):

• λ = 1: Pure relevance ranking. The algorithm selects items solely based on similarity to the query, ignoring redundancy entirely.
• λ = 0: Pure diversity ranking. The algorithm maximizes novelty, selecting items that are maximally dissimilar to each other.
• Typical values range from 0.5 to 0.7, providing a tunable trade-off between relevance and diversity in the final result set.

MMR builds its output set iteratively, one document at a time, rather than evaluating all possible combinations globally.

• The first selection is always the document with the highest pure relevance score to the query.
• Each subsequent selection re-scores all remaining candidates by subtracting a penalty proportional to their maximum similarity to any already-chosen document.
• This greedy approach is computationally efficient, with a time complexity of O(n²) for selecting k items from a candidate set of size n, making it practical for real-time search and summarization.

The novelty component of MMR explicitly penalizes content that is too similar to what has already been selected.

• Similarity is typically computed using cosine similarity between TF-IDF or embedding vectors.
• The penalty is calculated as the maximum similarity between a candidate and any previously selected item, not the average similarity.
• This max-penalty approach aggressively suppresses near-duplicate content, ensuring each selected passage contributes unique information to the final summary or result list.

MMR is a critical post-retrieval step in modern RAG pipelines to prevent context window pollution.

• After a vector database returns a set of top-k relevant chunks, MMR re-ranks them to maximize information diversity within the limited token budget sent to the LLM.
• This prevents the model from receiving five nearly identical passages about the same sub-topic, which wastes context and provides no additional grounding value.
• Frameworks like LangChain and LlamaIndex implement MMR as a standard search_type parameter for vector store retrievers.

The MMR score for a candidate document Dᵢ given a query Q and the set of already-selected documents S is defined as:

MMR(Dᵢ) = λ · Sim(Dᵢ, Q) — (1 — λ) · max(Dⱼ ∈ S) Sim(Dᵢ, Dⱼ)

• Sim(Dᵢ, Q): The relevance score between the candidate and the query.
• max(Dⱼ ∈ S) Sim(Dᵢ, Dⱼ): The highest similarity between the candidate and any already-selected document.
• The subtraction of the penalty term directly reduces the score of candidates that are redundant with the existing set.

Introduced by Carbonell and Goldstein in 1998, MMR was originally designed for text summarization and information retrieval.

• Query-focused summarization: Producing concise, non-redundant summaries that address a specific user query.
• Search result diversification: Ensuring the top results for an ambiguous query cover all possible interpretations.
• Recommender systems: Presenting users with a diverse set of recommendations rather than highly similar items.
• The algorithm remains widely used because it requires no training and is fully transparent in its selection logic.

A parameter in decoding or retrieval algorithms that penalizes repetition and encourages the selection of semantically varied tokens or passages. In MMR, the diversity constraint is explicitly controlled by the λ (lambda) hyperparameter, which trades off relevance against novelty. A higher diversity weight produces more comprehensive, non-redundant summaries by actively suppressing content that is too similar to what has already been selected.

A scoring mechanism that reduces the weight of a token or passage if it is highly similar to content already present in the generated summary or retrieved set. MMR implements this through cosine similarity between candidate passages and already-selected content. The penalty is subtracted from the relevance score, ensuring that each new sentence adds incremental information value rather than restating known facts.

An NLP technique that identifies and directly copies the most salient sentences from a source document without altering the original wording. MMR is a foundational algorithm for extractive summarization, iteratively selecting sentences that maximize query relevance while minimizing similarity to the existing summary set. This contrasts with abstractive methods that generate novel phrasing.

A summarization task that generates a concise answer specifically tailored to a user's natural language query rather than providing a general overview. MMR was originally designed for this exact use case, where the relevance term measures similarity between candidate passages and the query, while the redundancy term ensures the summary covers diverse aspects of the answer rather than repeating the same point.

The mathematical backbone of MMR's redundancy calculation. It measures the cosine of the angle between two vectors in a high-dimensional embedding space, producing a score from -1 to 1 where 1 indicates identical direction. In MMR, cosine similarity between the candidate passage and each already-selected passage is computed, and the maximum similarity value is used as the penalty term to ensure diversity.

A content evaluation metric that quantifies the unique, substantive value a piece of content provides beyond what is already known. MMR operationalizes this concept algorithmically: each selected passage should maximize the marginal information gain—the relevance to the query minus the redundancy with prior selections. This ensures every sentence in a summary contributes novel information.