Chunk Maximal Marginal Relevance

A post-retrieval stage where a more computationally intensive model re-scores initial search results to prioritize the most relevant chunks before LLM synthesis. MMR is a classic re-ranking algorithm that operates on the candidate set returned by vector similarity search. While modern re-rankers often use cross-encoders like Cohere Rerank or BGE-Reranker, MMR remains valuable for its diversity-aware scoring that prevents the top-k results from being near-duplicate passages. The re-ranking pipeline typically: - Retrieves an initial candidate set (e.g., top-100 by cosine similarity) - Applies MMR or a neural reranker to select the final top-k - Passes the deduplicated, high-quality chunks to the LLM context window

The process of identifying and removing redundant or highly similar text segments from a vector index to improve retrieval diversity and storage efficiency. MMR directly addresses deduplication at query time by penalizing chunks with high cosine similarity to already-selected chunks. This is distinct from index-time deduplication, which uses techniques like: - MinHash for fuzzy near-duplicate detection - Exact substring matching for boilerplate removal - Embedding clustering to collapse semantic duplicates MMR complements these by handling query-dependent redundancy — two chunks may be semantically distinct in isolation but redundant for a specific query.

A search approach that combines dense vector similarity with sparse keyword matching (BM25) to improve recall for both semantic and exact-match queries. MMR can be applied to the fused result set from hybrid retrieval to ensure diversity across both retrieval modalities. The typical flow: - Dense retrieval returns top-N by embedding similarity - Sparse retrieval returns top-N by BM25 term frequency - Results are merged via Reciprocal Rank Fusion (RRF) or score normalization - MMR re-ranks the merged candidates to maximize relevance while minimizing redundancy This ensures the final context includes both semantically related and lexically precise chunks without duplication.

A measure of the ratio of unique factual content to total token length within a chunk, used to prioritize high-value segments for indexing. MMR implicitly favors high-density chunks because the relevance term in the MMR equation scores factual alignment with the query, while the redundancy penalty eliminates low-density filler that overlaps with better chunks. Key density metrics include: - Entity count per token: number of named entities divided by chunk length - Novelty score: percentage of information not present in preceding chunks - Token efficiency: ratio of content-bearing tokens to stop words and boilerplate High-density chunks improve MMR performance by providing more discriminative signals for both relevance and diversity scoring.

A quality metric measuring whether a text segment contains a logically complete and self-contained idea without requiring external context to be understood. MMR's effectiveness depends heavily on chunk coherence: - High coherence: MMR can accurately assess both relevance and inter-chunk redundancy because each chunk represents a distinct, complete thought - Low coherence: fragmented chunks cause MMR to either over-penalize (treating complementary fragments as redundant) or under-penalize (failing to detect topical overlap across broken sentences) Best practices for MMR-friendly chunking include: - Respecting paragraph and section boundaries - Ensuring each chunk contains a subject-predicate-object structure - Avoiding mid-sentence splits that create dangling anaphora

The maximum span of tokens a large language model can process in a single forward pass, defining the upper limit for chunk selection. MMR operates within this constraint by selecting a diverse subset of chunks that maximizes information coverage within the token budget. The relationship is critical: - Small context windows (4K-8K tokens): MMR's diversity penalty is essential to avoid wasting limited space on redundant passages - Large context windows (128K-1M tokens): MMR still provides value by preventing attention dilution, where the model's focus degrades across too many similar chunks The lambda parameter in MMR should be tuned based on context window size — smaller windows demand stronger diversity enforcement.