Hypothetical Document Embeddings (HyDE)

Standard retrieval methods embed the user's short, often ambiguous query directly. This creates a vocabulary mismatch because the query's sparse embedding is semantically distant from the rich, descriptive embeddings of the target documents. HyDE solves this by first expanding the query into a full hypothetical document, aligning the search vector's density with the corpus.

HyDE operates in a sequential pipeline:

• Stage 1: Hypothetical Document Generation. A language model (e.g., GPT-3.5) receives the query and is prompted to generate a plausible, detailed answer or document, even if it contains factual errors.
• Stage 2: Vector Similarity Retrieval. The generated hypothetical text is embedded into a dense vector. This vector is used to perform a nearest-neighbor search in the document vector store to find real, relevant documents.

HyDE is fundamentally unsupervised at the retrieval stage. It requires no relevance labels or fine-tuning for the retriever. The core mechanism relies on the property that a generated text answering a query will be vector-space proximal to real documents answering the same query, even if the generated text's factual content is hallucinated. The embedding model captures topical similarity over factual precision.

A counter-intuitive strength of HyDE is its tolerance for hallucination in the generated hypothetical document. Since the generated text is never shown to the user and is used only as a search query, factual errors are irrelevant. The model only needs to capture the linguistic patterns and topical vocabulary of a relevant answer to create an effective embedding for retrieval.

In a standard Retrieval-Augmented Generation (RAG) pipeline, the raw user query is embedded and used for retrieval. HyDE modifies this by inserting a pre-retrieval generation step. This is distinct from query expansion, which adds keywords; HyDE synthesizes a complete, coherent narrative to capture the ideal document's latent structure before any real data is accessed.

HyDE significantly boosts retrieval recall for complex, information-seeking queries by adding contextual depth. However, this comes at the cost of increased latency due to the initial LLM generation call. The technique is most effective when the retrieval corpus contains documents that are stylistically similar to the LLM's generated output and when the query is too short for effective direct embedding.

The foundational framework that HyDE enhances. RAG first retrieves relevant documents from an external knowledge base, then provides that context to a language model to ground its response.

• Standard RAG embeds the user query directly for retrieval
• HyDE-enhanced RAG generates a hypothetical answer first, then uses that embedding
• Bridges the vocabulary gap between short queries and document language

The dual-tower neural architecture that powers HyDE's retrieval step. A bi-encoder independently encodes queries and documents into dense vector representations.

• Query encoder processes the hypothetical document text
• Document encoder pre-computes embeddings for the entire corpus
• Enables fast cosine similarity search across millions of documents
• Contrast with cross-encoders, which process query-document pairs jointly for higher accuracy but slower retrieval

A complementary technique to HyDE that reformulates a user's query by adding related terms, synonyms, or paraphrases. While HyDE generates a full hypothetical document, query expansion makes smaller, targeted additions.

• Lexical expansion adds synonyms and morphological variants
• Pseudo-relevance feedback extracts terms from top initial results
• Often used in conjunction with HyDE for multi-stage retrieval pipelines
• Addresses the same core problem: vocabulary mismatch between queries and documents

The evaluation score that measures whether a generated text is factually consistent with its source documents. Critical for assessing HyDE's downstream impact.

• Measures entailment between retrieved documents and final output
• Detects extrinsic hallucinations not supported by provided context
• Common implementations: Natural Language Inference (NLI) models, LLM-as-judge
• HyDE can improve faithfulness by retrieving documents the original query would have missed

A second-stage refinement process applied after HyDE's initial vector retrieval. A more precise model re-scores candidate documents to push the most relevant results to the top.

• Cross-encoders are the standard re-ranking architecture
• Processes query and document jointly through transformer attention layers
• Computationally expensive, so applied only to top-k candidates (typically 100-1000)
• Significantly boosts retrieval precision after HyDE's high-recall embedding search

Answering complex questions by combining information from multiple distinct documents in a logical chain. HyDE-generated hypothetical documents can serve as an initial reasoning scaffold.

• Requires iterative retrieval where each step informs the next
• HyDE can generate intermediate hypothetical answers for each hop
• Example: 'Which CEO of the company that acquired Instagram was born first?' requires two retrieval steps
• Tests the compositional generalization of retrieval systems