Contextual Retrieval

Each text chunk is prepended with a document-level summary before embedding. For example, a chunk from a financial report might be prefixed with: "This chunk is from ACME Corp's Q2 2024 SEC 10-Q filing, discussing revenue recognition policies." This ensures the vector representation captures the chunk's provenance and purpose, not just its isolated text.

• Prevents semantic drift from orphaned chunks
• Enables retrieval of domain-specific content without domain-specific queries
• Compatible with any embedding model without fine-tuning

The enriched chunk—original text plus its contextual prefix—is passed through an embedding model like text-embedding-3-large or Cohere Embed v3. The resulting vector encodes both the chunk's semantic content and its document-level identity. This contrasts with naive chunking, where a sentence like "The company increased spending by 15%" loses all reference to which company or time period it describes.

• Preserves entity resolution across chunk boundaries
• Eliminates ambiguity from pronouns and implicit references
• Works with both dense and sparse retrieval methods

Contextual Retrieval extends beyond vector search by also indexing the enriched text in a BM25 inverted index. This means exact keyword matches on the contextual prefix—such as "SEC 10-Q" or "Q2 2024"—directly contribute to retrieval scores. The combination of dense vector search and context-aware sparse retrieval creates a hybrid retrieval pipeline with superior recall.

• Boosts precision for metadata-heavy queries
• Enables filtering by document type without separate metadata stores
• Reduces reliance on complex metadata filtering infrastructure

The contextual prefixes are generated by prompting a language model with the entire document and requesting a concise summary for each chunk. To reduce cost and latency, the full document is loaded once and cached in the prompt context, then each chunk is processed sequentially. This prompt caching technique minimizes redundant token processing while ensuring every chunk receives accurate document-level context.

• Single-pass document processing for efficiency
• Consistent context quality across all chunks
• Scales to documents with thousands of chunks

After initial retrieval, a cross-encoder reranker evaluates the full enriched chunk against the query. Because the chunk now contains its document context, the reranker can assess relevance with complete information. A chunk discussing "liability limitations" from a vendor contract is correctly distinguished from one discussing "liability coverage" from an insurance policy—a distinction lost in naive chunking.

• Reduces false positives from ambiguous terminology
• Improves precision in multi-document corpora
• Works with Cohere Rerank, BGE-Reranker, and other cross-encoders

Contextual Retrieval introduces a one-time preprocessing cost for generating contextual prefixes via an LLM. However, this cost is amortized across all future queries against the indexed content. For enterprise knowledge bases where content changes infrequently but is queried thousands of times, the retrieval quality gains far outweigh the initial embedding cost.

• Typical cost: $1.20 per million tokens for prefix generation
• No additional query-time latency or cost
• Re-indexing required only when source documents change

A segmentation strategy that splits documents based on semantic boundaries—paragraphs, sections, or topic shifts—rather than arbitrary character counts. This preserves contextual integrity so that each chunk represents a coherent idea.

• Prevents mid-thought truncation that degrades embedding quality
• Often uses sentence-boundary detection or embedding similarity thresholds
• Pairs directly with Contextual Retrieval: well-formed chunks receive more meaningful document-level prefixes

A retrieval architecture that indexes small, precise child chunks for search but returns the full parent document they originated from. This ensures the language model receives complete context rather than isolated fragments.

• Commonly implemented in LangChain and LlamaIndex
• Solves the context-window tradeoff: search with precision, generate with completeness
• Contextual Retrieval complements this by enriching child chunks with parent-level metadata before embedding

Combines dense vector similarity with sparse keyword-based retrieval (BM25) to leverage both semantic understanding and exact term matching. This dual approach improves recall for queries containing rare entities, codes, or domain-specific jargon.

• Dense retrieval captures meaning; sparse retrieval captures exact matches
• Results are merged using algorithms like Reciprocal Rank Fusion (RRF)
• Contextual Retrieval's enriched chunks benefit both retrieval paths simultaneously

A query expansion technique where a language model generates a hypothetical ideal document in response to a query. The embedding of that synthetic document is then used to search the vector store instead of the raw query.

• Bridges the semantic gap between short queries and detailed documents
• Particularly effective when Contextual Retrieval prefixes provide rich document context for the hypothetical document to align with

A two-stage retrieval refinement where a computationally expensive cross-encoder model processes the query and each candidate document jointly. Applied only to the top-k results from a faster initial retrieval, it produces highly precise relevance scores.

• Stage 1: Fast bi-encoder or ANN search over contextualized chunks
• Stage 2: Cross-encoder re-ranks candidates with full attention between query and document
• The enriched context from Contextual Retrieval gives the cross-encoder more signal for accurate scoring

A fine-grained chunking method that decomposes text into atomic, self-contained propositions—each expressing a single fact. This maximizes retrieval precision for fact-checking and grounding tasks.

• Each proposition is independently retrievable and verifiable
• Reduces noise by isolating individual claims
• When combined with Contextual Retrieval prefixes, each atomic fact retains its document origin for accurate attribution and citation