Hypothetical Document Embedding (HyDE)

HyDE leverages a Large Language Model (LLM) with a simple instruction prompt to generate a hypothetical document that answers the user's query, even if the generated content contains factual inaccuracies. The key insight is that the structural and semantic pattern of a correct answer is what matters for retrieval, not the factual correctness of the hallucinated text. This process requires no fine-tuning or labeled training data, making it immediately deployable across any domain.

Once the synthetic document is generated, it is passed through a dense embedding model to produce a vector representation. This embedding captures the high-level semantic gist and topical structure of a plausible answer. The vector is then used to query a vector index via Approximate Nearest Neighbor (ANN) search. Real documents that share a similar semantic fingerprint—even with zero lexical overlap—are retrieved, effectively grounding the hallucinated answer in factual source material.

Traditional keyword-based retrieval like BM25 fails when queries and documents use different vocabulary to describe the same concept. HyDE bridges this lexical gap by expanding a terse query into a verbose, document-like representation. For example, a query like 'Why is the sky blue?' becomes a paragraph on Rayleigh scattering. This synthetic paragraph's embedding naturally clusters near real scientific documents, even if the query itself shares few words with them.

Because the LLM generates the hypothetical document based on its pre-trained knowledge, HyDE inherently adapts to the language, style, and structure of the target domain without requiring domain-specific retrieval training. A query about a medical condition generates a synthetic document in clinical prose, which then retrieves real medical literature. This makes HyDE exceptionally effective for domain-specific search where off-the-shelf embedding models may not have been fine-tuned on specialized corpora.

HyDE introduces a latency overhead at query time because it requires a full LLM generation step before the embedding and retrieval can begin. This contrasts with standard bi-encoder retrieval, which embeds the raw query directly. The trade-off is between increased relevance for complex or ambiguous queries and the added inference cost and latency. In production, this is often mitigated by using a smaller, faster instruction-tuned model for the generation step.

HyDE functions as a first-stage retrieval augmenter and composes naturally with downstream re-ranking. The synthetic document embedding retrieves a broader candidate set with high semantic recall. A cross-encoder re-ranker then scores each real document against the original user query to filter out any noise introduced by the hallucinated text. This two-stage pipeline—HyDE for recall, cross-encoder for precision—represents a state-of-the-art approach for factual grounding in RAG architectures.

A family of techniques that enrich a user's original query with additional terms or reformulations to improve retrieval recall. While HyDE generates a synthetic answer document, query expansion typically adds synonyms, hyponyms, or paraphrases.

• Lexical expansion adds keywords using thesauri or statistical co-occurrence
• Neural expansion uses language models to generate alternative query formulations
• Complements HyDE by addressing vocabulary mismatch from a different angle
• Can be applied before or in parallel with hypothetical document generation

The process of decomposing a complex query into sub-questions and iteratively retrieving and synthesizing information across multiple documents. HyDE can be applied at each hop to generate intermediate hypothetical answers.

• Decomposition: Break "What caused the event after X?" into sequential steps
• Iterative retrieval: Use the answer from hop 1 to form the query for hop 2
• HyDE generates a hypothetical intermediate answer to guide the next retrieval
• Essential for questions requiring information aggregation across disparate sources

An adaptive document splitting technique that uses embedding similarity to determine natural break points between sentences. The quality of chunks directly impacts HyDE's ability to match hypothetical documents to real ones.

• Calculates cosine similarity between consecutive sentences
• Identifies similarity drops as topic transition boundaries
• Produces self-contained, coherent chunks rather than arbitrary splits
• Ensures the hypothetical document embedding aligns with well-formed semantic units

The standard metric for comparing embedding vectors in dense retrieval, measuring the cosine of the angle between two vectors. When HyDE generates a hypothetical document embedding, cosine similarity determines which real documents are nearest.

• Range: -1 (opposite) to 1 (identical), with 0 indicating orthogonality
• Normalizes for vector magnitude, focusing purely on directional alignment
• Computationally efficient for high-dimensional vectors
• The core mathematical operation behind every HyDE-powered retrieval