Contextual Compression

Selects the most relevant contiguous sentences or passages from a source document based on their similarity to the user's query.

• Mechanism: Uses embedding similarity or cross-encoder scoring to rank sentences.
• Key Benefit: Guarantees factual consistency because the output is a verbatim subset of the input.
• Example: Retrieving a 50-page legal contract and extracting only the 3 paragraphs discussing 'Indemnification'.

Generates a condensed summary of the source context in the model's own words, similar to human summarization.

• Mechanism: A smaller, efficient model rewrites the long context into a dense, short passage.
• Key Benefit: Can fuse information from disparate sections of a document into a single coherent sentence.
• Risk: May introduce subtle hallucinations or alter nuanced meanings if the compression model is not properly aligned.

The compression logic is strictly conditioned on the user's specific question, ensuring only query-relevant information survives.

• Mechanism: The system computes the cross-attention between the query embedding and every chunk of the document.
• Key Benefit: Prevents the dilution of the context window with high-quality but irrelevant background information.
• Example: For the query 'What is the return policy?', all product specification text is filtered out, leaving only the legal return clauses.

Balances the compression ratio against the information loss, ensuring the compressed context retains maximum utility for the downstream task.

• Mechanism: Treats compression as a mathematical trade-off between the token budget (rate) and the semantic fidelity (distortion).
• Key Benefit: Provides a tunable knob for engineers to set a strict token limit while minimizing the degradation of the final answer quality.
• Use Case: Critical in Retrieval-Augmented Generation (RAG) where a fixed top-k of retrieved chunks must fit within a strict context window limit.

Leverages the instruction-following capabilities of a language model to perform compression via a zero-shot or few-shot prompt.

• Mechanism: A prompt like 'Compress the following text to 100 tokens while preserving key entities' is passed to a lightweight LLM.
• Key Benefit: No fine-tuning required; highly flexible and adaptable to new domains instantly.
• Example: Using GPT-4o-mini to compress a verbose customer support transcript before passing the dense summary to a more expensive reasoning model.

Iteratively compresses long documents by summarizing chunks in parallel and then summarizing the summaries.

• Mechanism: A Map-Reduce style approach where a document is split, each part is compressed, and the combined compressed outputs are compressed again.
• Key Benefit: Overcomes the context window limits of the compressor model itself, allowing for the compression of arbitrarily long texts.
• Use Case: Summarizing a 500-page technical manual into a single page of dense technical context for an AI agent.