Dense Retrieval

The embedding model is the neural network responsible for converting text (queries and documents) into dense vector representations, or embeddings. These models, such as sentence transformers like all-MiniLM-L6-v2 or text-embedding-3-small, are trained to position semantically similar texts close together in the high-dimensional vector space. The model's quality directly determines the system's semantic understanding and retrieval accuracy. Key considerations include model size, dimensionality (e.g., 384, 768, or 1536 dimensions), and whether it's pre-trained or fine-tuned on domain-specific data.

A vector index is a specialized data structure optimized for Approximate Nearest Neighbor (ANN) search. It enables the rapid lookup of the vectors most similar to a query embedding. Common algorithms include:

• HNSW (Hierarchical Navigable Small World): A graph-based method offering a strong balance of speed and accuracy.
• IVF (Inverted File Index): Clusters vectors into Voronoi cells for coarse-grained filtering.
• IVF-PQ: Combines IVF with Product Quantization to compress vectors, drastically reducing memory usage for massive datasets. Libraries like FAISS, Weaviate, and Qdrant provide implementations of these indices, which are built offline from the document corpus.

The vector store is the persistent storage and retrieval engine that houses the vector index, the raw embeddings, and their associated metadata (like the original document text and IDs). It provides the APIs for indexing (adding vectors) and querying (searching). This component is distinct from the index algorithm; it handles scalability, durability, and often advanced features like filtering, multi-tenancy, and hybrid search. Examples include dedicated vector databases like Pinecone, Milvus, and Chroma, as well as ANN extensions for traditional databases like pgvector for PostgreSQL.

This is the runtime component that accepts a user's natural language query. The query encoder uses the same embedding model to convert the query into a vector. The retrieval interface then takes this query vector and executes a search against the vector index in the store. It handles parameters like the number of results to return (top_k), similarity score thresholds, and any metadata filters (e.g., WHERE year > 2020). The output is a ranked list of document IDs, their similarity scores (e.g., cosine similarity), and the associated metadata or text chunks.

Before documents can be embedded, they must be segmented into meaningful chunks. The chunking strategy is critical, as it defines the unit of retrieval. Common methods include:

• Fixed-size chunking: Simple but can split semantic concepts.
• Semantic chunking: Uses text coherence or embeddings to break at natural boundaries.
• Recursive chunking: Splits by characters, then by tokens, aiming for optimal sizes. The pipeline also handles text cleaning, normalization, and may extract metadata. Poor chunking can severely degrade retrieval performance by creating fragments with incomplete context.

A re-ranker is a secondary, more computationally intensive model that refines the results from the initial vector search. The dense retriever acts as a fast recall stage, fetching a broad set of candidate documents (e.g., top 100). The re-ranker, often a cross-encoder model like cross-encoder/ms-marco-MiniLM-L-6-v2, then evaluates the precise relevance of each query-document pair for superior precision. This two-stage process combines the speed of ANN search with the accuracy of more powerful, slower models, optimizing the overall quality of the final retrieved set.

A neural network, typically a transformer, that converts discrete data (text, images) into continuous, dense vector representations (embeddings). The quality of dense retrieval is fundamentally limited by the embedding model's ability to capture semantic meaning.

• Core Task: Maps semantically similar inputs to nearby points in the vector space.
• Training: Models like BERT, Sentence-BERT, and OpenAI's text-embedding-ada-002 are trained on contrastive or ranking losses to optimize for retrieval tasks.
• Output: A fixed-length vector (e.g., 768 dimensions) where cosine similarity between vectors indicates semantic relatedness.

The overarching information retrieval paradigm that dense retrieval enables. It moves beyond literal keyword matching (lexical search) to understanding the contextual meaning and intent behind queries and documents.

• Contrast with Sparse Retrieval: Does not rely on term frequency (e.g., TF-IDF or BM25). The query "automobile" can retrieve documents about "cars" even if the keyword is absent.
• Implementation: Typically involves creating a dense vector index of all documents and then querying it with an embedded user question.
• Hybrid Search: Often combined with sparse (keyword) retrieval methods to balance recall of exact terms with semantic understanding.

An architecture that uses dense retrieval as its core information-fetching component. RAG grounds a large language model (LLM) by retrieving relevant context from an external knowledge source (like a vector store) before generating an answer.

• Workflow: 1) User query is embedded. 2) Dense retrieval finds relevant document chunks. 3) Retrieved context is injected into the LLM prompt. 4) LLM generates a factually grounded response.
• Key Benefit: Mitigates LLM hallucinations by providing authoritative source material.
• Dependency: The performance of the entire RAG pipeline is critically dependent on the precision and recall of the underlying dense retrieval system.

A structured, graph-based alternative or complement to dense vector retrieval. It represents knowledge as a network of entities (nodes) and their relationships (edges). While dense retrieval finds semantically similar text, knowledge graphs enable explicit logical reasoning and traversal of factual connections.

• Query Method: Uses graph query languages like SPARQL or Cypher, not vector similarity.
• Strengths: Excellent for navigating known relationships (e.g., "find all employees who report to this manager"), enforcing ontological rules, and combining facts.
• Hybrid Approach: Often used with dense retrieval, where a knowledge graph provides structured facts and a vector store provides semantic similarity over unstructured text.