Maximum Marginal Relevance (MMR)

MMR operates on a linear combination of two competing objectives. The first term measures query relevance—how similar a candidate document or sentence is to the original query. The second term measures maximum similarity to any item already selected for the output set. A tunable λ (lambda) parameter controls the balance. When λ=1, the algorithm behaves as a pure relevance ranker. When λ=0, it maximizes diversity by selecting items most dissimilar to everything already chosen. This explicit trade-off is what distinguishes MMR from simple deduplication heuristics.

MMR is a greedy algorithm, meaning it makes the locally optimal choice at each step without backtracking. The process begins by selecting the single most relevant item to the query. For each subsequent selection, the algorithm re-scores all remaining candidates by computing:

• Sim1: Cosine similarity between the candidate and the query.
• Sim2: The maximum cosine similarity between the candidate and any item already in the selected set. The candidate with the highest MMR score is added to the set, and the process repeats. This greedy approach is computationally efficient—O(n²) in the number of candidates—but does not guarantee a globally optimal diverse set.

The standard MMR formulation uses cosine similarity as its similarity function, operating on TF-IDF vector representations of text. Cosine similarity measures the cosine of the angle between two vectors, producing a score from -1 to 1, where 1 indicates identical orientation. In practice, MMR is embedding-agnostic. Modern implementations often substitute:

• Dense embeddings from models like BERT or Sentence-BERT for semantic similarity.
• Sparse lexical overlap metrics like BM25 for keyword-level relevance. The choice of similarity function fundamentally alters the behavior of the algorithm, with dense embeddings capturing paraphrastic relationships that TF-IDF would miss.

In extractive summarization, MMR is applied to sentence selection. The algorithm treats the entire document or document cluster as the candidate set and the user query (or the document centroid) as the query vector. Key behaviors include:

• Redundancy elimination: Prevents selecting multiple sentences that convey the same information, a common failure mode of pure relevance ranking.
• Information coverage: By penalizing similarity to already-selected sentences, MMR implicitly maximizes the breadth of concepts covered in the summary.
• Query-focused summarization: When a specific query is provided, MMR balances answering the question directly with providing necessary context that a purely extractive approach might omit.

The λ parameter (typically in the range [0,1]) is the primary control knob for MMR behavior. Empirical guidance for common use cases:

• λ = 0.7 to 0.9: Standard information retrieval and search result diversification. Strong preference for relevance with moderate redundancy penalty.
• λ = 0.3 to 0.5: High-diversity summarization where covering distinct subtopics is prioritized over strict query alignment.
• λ = 1.0: Degrades to pure relevance ranking. Useful as a baseline ablation.
• λ = 0.0: Maximum diversity. The algorithm selects items maximally dissimilar to the current set, often producing an unfocused but highly varied output. Tuning λ requires domain-specific evaluation, typically measuring ROUGE or BERTScore against reference summaries.

MMR has several known limitations that motivate modern alternatives:

• Greedy suboptimality: The incremental selection can lock in early choices that prevent a globally better diverse set.
• Fixed similarity function: TF-IDF cosine similarity fails to capture semantic equivalence, leading to redundant selections of paraphrased content.
• No global coherence modeling: MMR selects sentences independently without considering how they flow together as a narrative. Modern approaches like Determinantal Point Processes (DPPs) provide a probabilistic framework for diverse subset selection with global normalization. Submodular optimization with monotone submodular functions like facility location offers theoretical guarantees on solution quality that MMR lacks. In neural systems, cross-encoder rerankers with diversity-aware training objectives have largely replaced MMR for production summarization.