Context Window Optimization

Applies lossless and lossy compression algorithms to legal text before insertion into the context window.

• LLMLingua: Uses a small language model to remove non-essential tokens while preserving legal entities and citations
• Selective Context: Drops low-perplexity sentences that contribute minimal information to the legal reasoning task
• Citation Preservation: Compression pipelines are explicitly instructed to never alter or remove case citations, statutory references, or pin-point page numbers

This technique can reduce token consumption by up to 50% without degrading the accuracy of downstream legal analysis.

Organizes retrieved legal documents into a rigid, hierarchical XML or Markdown structure within the prompt to maximize the model's ability to navigate and attribute evidence.

• Document Tagging: Each source is wrapped in <document id='...' title='...'> tags for explicit source tracking
• Hierarchical Chunking: Presents a document summary first, followed by relevant subsections, allowing the model to efficiently locate key passages
• Instruction-Context Separation: Uses clear delimiters to prevent the model from confusing system instructions with retrieved evidence

This structured approach reduces the cognitive load on the model and improves citation accuracy.

Implements a programmatic allocation system that distributes the finite context window across multiple retrieved documents based on their relevance scores.

• Weighted Allocation: Assigns more tokens to highly relevant precedents and fewer to marginally relevant background material
• Reserved Generation Space: Explicitly reserves a fixed percentage of the context window for the model's final reasoning and answer
• Overflow Management: When retrieved evidence exceeds the budget, the system triggers summarization of lower-priority documents rather than truncation

This prevents the most critical authority from being cut off mid-sentence.

Segments legal documents into atomic, self-contained factual propositions rather than arbitrary token windows to eliminate wasted context space.

• Atomic Units: Each chunk expresses exactly one legal fact, holding, or rule statement
• Context Independence: Every proposition is rewritten to include necessary contextual metadata, ensuring it is understandable in isolation
• Deduplication: Identical propositions from different sources are collapsed into a single entry with multiple citations

This fine-grained approach ensures that every token in the context window carries a discrete unit of legal information.

Distributes retrieved evidence across multiple parallel inference calls when a single context window is insufficient, then synthesizes the results.

• Map-Reduce Architecture: Each parallel call analyzes a subset of documents and generates intermediate findings
• Hierarchical Summarization: Intermediate outputs are aggregated and fed into a final synthesis step
• Cross-Reference Resolution: A post-processing step reconciles conflicting citations or interpretations that emerged across parallel windows

This technique effectively multiplies the available context capacity for cases requiring analysis of dozens of precedents.

Engineers the prompt structure to prevent the model's attention mechanism from being diluted by long contexts, a phenomenon known as the 'lost in the middle' problem.

• Recency Bias Exploitation: Places the most authoritative binding precedent at the very end of the context window
• Primacy Bias Exploitation: Places the core legal question and jurisdictional constraints at the very beginning
• Signposting: Inserts explicit transitional markers to re-anchor the model's attention when switching between distinct legal issues

This counteracts the documented tendency of transformer models to overweight the beginning and end of the context while ignoring the middle.

A fine-grained chunking strategy that segments legal documents into atomic, self-contained factual propositions rather than arbitrary token windows. By breaking a paragraph like 'The court held that the defendant was liable. Damages were set at $50,000.' into two distinct chunks, the retriever can precisely place only the relevant holding into the context window, eliminating wasted tokens on adjacent but irrelevant facts. This directly increases the evidence density of the assembled prompt.

A retrieval architecture that decouples the search space from the generation space. The system searches using small, focused sentence chunks to maximize semantic precision, but returns the larger parent paragraph or full section to the generator. This ensures the model receives the necessary surrounding legal context—such as the procedural posture or the full reasoning chain—without the search algorithm being confused by the length and topical diversity of a large document block.

A post-retrieval step where a computationally intensive cross-encoder model re-orders a candidate list of legal documents. Unlike a fast bi-encoder that scores documents independently, a cross-encoder processes the query and passage simultaneously to compute a high-fidelity relevance score. By applying this to the top 100 candidates and selecting only the top 5 for the context window, the system guarantees that every token slot is occupied by the most jurisprudentially relevant text, not just the most semantically similar.

A preprocessing technique that prepends chunk-specific explanatory context to each text chunk before embedding. For example, a chunk stating 'The motion is denied.' is ambiguous in isolation. Contextual Retrieval transforms it to 'In Smith v. Jones, regarding the motion to suppress evidence, the court held: The motion is denied.' This prevents the embedding model from creating a poor vector representation, ensuring the chunk is retrieved for relevant queries and the generator does not waste precious context window space on an ambiguous, useless fragment.

The technique of breaking a complex, multi-faceted legal question into a set of simpler sub-questions that can be answered independently. Instead of stuffing a single context window with evidence for 'Was the contract breached and what are the damages and is the liquidated damages clause enforceable?', the system answers each sub-question with a focused retrieval set. The final synthesis step then combines the answers, effectively parallelizing the evidence gathering and preventing cross-topic contamination within a single overloaded prompt.

A forward-looking active retrieval method that monitors the model's generation confidence in real-time. As the model generates a legal analysis, FLARE detects when it is about to produce a low-probability token—signaling a need for factual grounding. It then proactively searches for the specific missing information and injects it into the ongoing context window. This just-in-time approach avoids pre-loading the entire window with speculative evidence, instead dynamically allocating tokens only when a knowledge gap is detected.