Maximum Marginal Relevance (MMR)

MMR operates via a linear combination of two competing scores controlled by the hyperparameter λ (lambda), typically set between 0 and 1.

• When λ = 1: The algorithm defaults to standard, greedy relevance ranking. It selects items solely based on their similarity to the query (maximizing Sim_1).
• When λ = 0: The algorithm maximizes novelty, selecting items that are maximally dissimilar to everything already chosen, regardless of the query.
• Standard Calibration: A λ of 0.5 to 0.7 is common in summarization to penalize redundancy without drifting off-topic.

The core formula is: MMR = argmax [ λ * Sim_1(D_i, Q) - (1 - λ) * max Sim_2(D_i, D_j) ]

MMR is a greedy algorithm, meaning it builds the final result set iteratively rather than finding the mathematically optimal global set.

• Process: It selects the single best item according to the MMR score, adds it to the summary set, and then recalculates the redundancy penalty for all remaining candidates against the newly expanded set.
• Computational Efficiency: This greedy approach avoids the combinatorial explosion of evaluating all possible subsets, making it scalable for real-time search and summarization.
• Order Dependency: Because it is greedy, the first item selected (which is always the most query-relevant) heavily influences the trajectory of the remaining selections.

The novelty component of MMR does not penalize a candidate for its average similarity to the existing set; it penalizes it based on the maximum similarity to any single item already selected.

• Max Function: The term max Sim_2(D_i, D_j) identifies the single most similar document already in the summary.
• Diversity Guarantee: By subtracting this maximum similarity, MMR explicitly prevents the selection of near-duplicate content. If a candidate is highly similar to any existing summary member, its score is heavily penalized.
• Contrast with Centroid: This differs from centroid-based methods that push the summary toward the mean vector of the document cluster.

MMR abstracts the notion of 'similarity' into two distinct functions, Sim_1 (relevance) and Sim_2 (novelty), which do not need to be the same metric.

• Asymmetric Metrics: You can use a dense semantic embedding (e.g., cosine similarity of BERT vectors) for Sim_1 to capture topical relevance, while using a sparse lexical metric (e.g., Jaccard similarity or TF-IDF) for Sim_2 to penalize exact phrase overlap.
• Cross-Modal Application: In video summarization, Sim_1 might measure visual similarity to a text query, while Sim_2 measures temporal frame redundancy.
• Practical Impact: This flexibility allows engineers to tune the algorithm to specific failure modes, such as penalizing verbatim repetition while rewarding semantic paraphrasing.

Unlike modern attention-based neural summarizers, MMR is a query-focused extractive model that operates without a learned context window limit.

• Extractive Nature: MMR selects whole sentences or passages verbatim from the source text; it does not generate new tokens or paraphrases.
• No Token Limit: The algorithm itself has no hard limit on input length, making it suitable for summarizing very long documents where transformer-based models might exceed their maximum context length.
• Comparison to RNNs: It avoids the recency bias found in recurrent neural networks, treating the first and last sentences of a document with equal statistical weight in the similarity matrix.

MMR was introduced in the seminal 1998 paper 'The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries' by Jaime Carbonell and Jade Goldstein.

• Original Domain: It was initially designed for information retrieval to re-rank search engine results, ensuring the top 10 results were not merely 10 copies of the same document.
• Adaptation: It was rapidly adopted by the NLP community for multi-document summarization, where redundancy is the primary failure mode.
• Enduring Legacy: Despite the rise of neural abstractive summarization, MMR remains a strong, unsupervised baseline due to its transparency and lack of training data requirements.

A bag-of-words retrieval function that ranks documents by term frequency saturation and inverse document frequency. Unlike MMR, BM25 scores each document independently against the query without considering redundancy among results. It serves as the relevance baseline that MMR then re-ranks by penalizing documents too similar to already-selected passages. The k1 and b parameters control term frequency saturation and document length normalization respectively.

A foundational numerical statistic measuring word importance as the product of how often a term appears in a document and how rare it is across the corpus. MMR extends TF-IDF's core insight—that not all matching terms are equally valuable—by adding a diversity constraint. While TF-IDF treats each document independently, MMR explicitly models inter-document similarity to suppress near-duplicate content in result sets.

A graph-based ranking algorithm inspired by PageRank that identifies key sentences or keywords by modeling text as a co-occurrence network. TextRank scores sentences by their centrality in the similarity graph, naturally surfacing diverse yet representative content. MMR and TextRank share the goal of reducing redundancy in extractive summarization, but MMR explicitly balances novelty against a query, while TextRank relies purely on graph structure.

A self-supervised learning paradigm where models learn representations by pulling positive pairs (similar data points) together and pushing negative pairs apart in embedding space. The InfoNCE loss function used in contrastive learning formalizes a trade-off analogous to MMR's relevance-novelty balance. Models like SimCSE apply this to sentence embeddings, creating vector spaces where MMR-style diversity selection can operate efficiently on dense representations rather than sparse lexical features.

A generative probabilistic model that discovers abstract topics within document collections by assuming each document is a mixture of topics and each topic is a distribution over words. LDA provides an alternative approach to diversity: instead of MMR's greedy pairwise comparison, LDA ensures coverage by selecting documents from different latent topic distributions. This topic-level diversity is computationally lighter but less precise than MMR's instance-level redundancy penalty.

A model compression technique where a smaller student model learns to mimic the output distribution of a larger teacher model. In retrieval contexts, MMR's diversity logic can be distilled into a learned scoring function: the teacher system applies explicit MMR re-ranking, and the student learns to predict the diversified ranking directly. This preserves MMR's redundancy suppression behavior while reducing inference latency for production search systems.