Retrieval-Interleaved Generation

RIG breaks the traditional single-pass RAG paradigm by creating a tight feedback loop between generation and retrieval. The model generates a sentence of legal reasoning, then pauses to formulate a new search query based on its current analytical state.

• Step 1: Generate a reasoning step and identify an information gap
• Step 2: Issue a targeted query to the legal corpus
• Step 3: Integrate the retrieved evidence into the next reasoning step
• Step 4: Repeat until a conclusion is reached

This mirrors how a human lawyer cross-references multiple authorities while drafting an argument.

Unlike static retrieval that uses only the user's original question, RIG generates context-aware queries that evolve as the model's understanding deepens. Each query is conditioned on the chain of reasoning generated so far.

• Uses the partially constructed argument as query context
• Can pivot from a broad statute search to a specific exception clause
• Queries often include newly discovered legal terms of art
• Prevents the model from fixating on an initial, potentially incomplete retrieval set

This dynamic approach is critical for multi-hop legal reasoning where the answer depends on connecting disparate authorities.

RIG implementations often use a confidence threshold to decide when to trigger a new retrieval. The model monitors its own token-level probability distribution and initiates a search when it detects uncertainty.

• High confidence: Continue generating from internal knowledge
• Low confidence: Pause generation, retrieve external evidence
• Prevents unnecessary retrievals that add latency
• Ensures the model seeks grounding precisely when needed

This gating mechanism is a form of model introspection, where the system evaluates its own knowledge boundaries in real-time.

Each retrieval round adds a new link to an explicit evidence chain. The final output is not just a conclusion but a fully traceable path showing how each retrieved document contributed to the reasoning.

• Document A: Established the governing statute
• Document B: Provided the interpretive precedent
• Document C: Addressed a jurisdictional exception
• Document D: Confirmed the application to the current facts

This architecture directly supports citation grounding and makes the model's reasoning auditable, a non-negotiable requirement in legal AI systems.

Standard RAG performs a single retrieval before generation begins, providing a static context window. RIG interleaves multiple retrievals throughout the generation process.

Standard RAG:

• Retrieve once → Generate once
• Context is fixed and cannot adapt
• Struggles with multi-hop questions

Retrieval-Interleaved Generation:

• Retrieve → Generate → Retrieve → Generate
• Context expands dynamically based on discovered needs
• Excels at building logical chains across documents

This makes RIG particularly suited for legal entailment tasks requiring synthesis across multiple authorities.

Forward-Looking Active REtrieval (FLARE) is a prominent RIG implementation. It monitors the model's generation and proactively retrieves when a low-probability token is anticipated.

• Uses the model's own logits as an uncertainty signal
• Generates a temporary next sentence to use as a search query
• Retrieves relevant documents before committing to the final sentence
• Replaces the temporary sentence with a grounded version

FLARE demonstrates how RIG can be implemented without an external classifier, using the model's intrinsic confidence signals to drive retrieval decisions.

An iterative search process where the answer to an initial query is used to formulate a secondary query to find connecting authority. This enables the construction of a logical evidence chain across multiple documents.

• Mechanism: The retriever uses the output of reasoning step n as the input query for step n+1.
• Legal Application: Finding a statute, then retrieving the cases that interpret that statute, then retrieving the cases that distinguish those interpretations.
• Relationship to RIG: Retrieval-Interleaved Generation is the generative counterpart to multi-hop retrieval, where the model actively decides when and what to search next.

Forward-Looking Active REtrieval is a method that monitors the model's generation confidence during decoding. When the model is about to generate a low-probability token, it proactively triggers a search query using the unconfident sentence as context.

• Key Distinction: Unlike fixed-interval interleaving, FLARE uses confidence-based gating to decide when to retrieve.
• Query Formulation: The temporary sentence the model was trying to generate becomes the implicit query.
• Legal Use Case: Automatically pausing mid-paragraph to look up a specific case holding when the model lacks internal knowledge of the citation.

A reasoning framework where a language model explicitly generates a sequence of interconnected legal citations to demonstrate the logical derivation of a conclusion from primary authority.

• Structure: Each claim is followed by a parenthetical citation, creating a verifiable reasoning trail.
• Integration with RIG: Retrieval-Interleaved Generation can supply the evidence for each link in the chain, grounding the citation sequence in actual retrieved text.
• Output: Produces a document that mirrors a human lawyer's brief, with every proposition anchored to a source.

A self-reflective architecture that evaluates the relevance of retrieved legal documents before generation. If the initial retrieval quality is scored as low, it triggers a corrective action such as a web search or knowledge graph lookup.

• Retrieval Evaluator: A lightweight classifier assesses the relevance of each retrieved chunk.
• Corrective Actions: Includes knowledge refinement from graphs or large-scale web search for supplementary context.
• Synergy with RIG: Both share the principle of iterative refinement, but CRAG corrects retrieval quality while RIG corrects reasoning gaps.

The technique of breaking a complex, multi-faceted legal question into a set of simpler sub-questions that can be answered independently before synthesizing a final response.

• Decomposition Strategies: Includes sequential (solve A then B) and parallel (solve A and B simultaneously) decomposition.
• Legal Example: Decomposing 'Is the defendant liable for negligence?' into sub-questions on duty, breach, causation, and damages.
• Contrast with RIG: Query decomposition plans all sub-questions upfront, while Retrieval-Interleaved Generation discovers the need for sub-questions dynamically during reasoning.

A dynamic framework that routes a legal query to different processing paths based on its assessed complexity. Simple queries may bypass retrieval entirely, while complex queries trigger multi-step retrieval and reasoning.

• Complexity Classifier: Determines if a query requires no retrieval, single-hop retrieval, or multi-hop interleaved retrieval.
• Routing Logic: Directs the query to the most computationally efficient path.
• RIG as a Path: Retrieval-Interleaved Generation serves as the most complex processing path within an Adaptive RAG system for queries requiring iterative evidence gathering.