Chunk Summarization

This technique creates two distinct vector representations for a single chunk of text. The full chunk is embedded for comprehensive context, while a concise summary is embedded separately. During retrieval, the query is matched against the summary embeddings first. This ensures the system finds the right document based on its core meaning, not just keyword overlap, before returning the full, detailed chunk to the LLM for synthesis.

Long or dense chunks often contain multiple topics, causing their vector to represent a diffuse average of their content. This leads to poor retrieval. A chunk summary acts as a high-signal anchor by distilling the chunk's single most important concept into a short text. Embedding this summary creates a sharply focused vector that is far more likely to match a specific user query, directly combating the 'curse of dimensionality' in vector search.

The core mechanism relies on the semantic similarity between a user's question and the generated summary. A well-crafted summary anticipates the questions the chunk answers. For example, a chunk detailing a complex API endpoint might have the summary: 'How to authenticate user requests using OAuth 2.0 bearer tokens.' This summary embeds closer to a user's natural language query than the raw, code-heavy chunk itself, dramatically improving top-k recall accuracy.

The most effective chunk summaries are generated by an LLM, not extractive algorithms. A prompt instructs the model to create a short, standalone statement of the chunk's core proposition. This is a pre-processing step in the ingestion pipeline. The cost of this LLM call is a trade-off for significantly higher retrieval quality. The summary must be a self-contained factual statement that makes sense without the surrounding document context.

There are two primary styles:

• Propositional Summary: A single, atomic factual statement. Example: 'The capital of France is Paris.' Best for precise, fact-seeking queries.
• Abstractive Summary: A concise natural language description. Example: 'This section details the historical and political reasons for Paris being named the capital of France.' Better for broader, explanatory queries. The choice depends on the expected query patterns and the nature of the source content.

The process follows a strict pipeline:

• Ingestion: Document is split into chunks.
• Summarization: An LLM generates a summary for each chunk.
• Dual Embedding: Both the summary and the full chunk are embedded.
• Indexing: Both vectors are stored in the vector database, with the summary vector as the primary searchable field.
• Retrieval: A user query is embedded and matched against summary vectors. The associated full-chunk text is then fetched and injected into the LLM's context window.