Late Chunking

A content segmentation strategy that splits text based on meaning and topic boundaries using embedding similarity rather than fixed character or token counts. Unlike late chunking, which operates on pre-computed token embeddings, semantic chunking analyzes cosine similarity between adjacent sentences to detect natural topic shifts. This method produces chunks with high chunk coherence, ensuring each segment contains a logically complete idea. It is particularly effective for long-form, multi-topic documents where fixed splitting would fracture conceptual units.

A method that prepends document-level context—such as a summary, title, or section heading—to each chunk before embedding. This technique directly addresses the core problem late chunking solves: preserving fine-grained context that would otherwise be lost during segmentation. By enriching each chunk with its broader document context, contextual chunking improves retrieval relevance and reduces ambiguity. It is a complementary approach to late chunking, often used when the encoder does not support token-level embedding preservation.

A configurable buffer of tokens shared between adjacent text chunks to preserve cross-boundary context and prevent information fragmentation during retrieval. Late chunking inherently mitigates the need for aggressive overlap because the encoder processes the full document before segmentation, capturing long-range dependencies. In contrast, fixed-length chunking relies heavily on overlap to avoid splitting entities or concepts across chunk boundaries. Typical overlap values range from 10% to 20% of the chunk size.

A fine-grained segmentation method that decomposes text into atomic, self-contained factual statements to maximize retrieval precision and minimize noise. Each proposition represents a single, indivisible fact. Late chunking operates at a higher level of abstraction, preserving the token-level context of the entire document, while propositional chunking focuses on creating the smallest possible retrievable units. This approach is ideal for fact-dense content like legal documents or scientific papers where precise fact retrieval is critical.

A post-retrieval stage where a more computationally intensive model re-scores initial search results to prioritize the most relevant chunks before LLM synthesis. Late chunking improves the quality of initial retrieval by preserving token-level context, but re-ranking adds a second layer of precision. Common re-ranking models include cross-encoders and ColBERT, which evaluate query-chunk pairs jointly rather than relying solely on vector similarity. This two-stage pipeline significantly improves answer quality in RAG systems.

A search approach that combines dense vector similarity with sparse keyword matching (such as BM25) to improve recall for both semantic and exact-match queries. Late chunking enhances the dense vector component by generating more contextually rich embeddings. Hybrid retrieval compensates for cases where dense embeddings miss rare terms, proper nouns, or exact identifiers. The fusion of results typically uses reciprocal rank fusion (RRF) or weighted score combination to produce a unified ranked list.