Dense Retrieval

The vector store is the persistent storage and retrieval engine that houses the vector index, the raw embeddings, and their associated metadata (like the original document text and IDs). It provides the APIs for indexing (adding vectors) and querying (searching). This component is distinct from the index algorithm; it handles scalability, durability, and often advanced features like filtering, multi-tenancy, and hybrid search. Examples include dedicated vector databases like Pinecone, Milvus, and Chroma, as well as ANN extensions for traditional databases like pgvector for PostgreSQL.

This is the runtime component that accepts a user's natural language query. The query encoder uses the same embedding model to convert the query into a vector. The retrieval interface then takes this query vector and executes a search against the vector index in the store. It handles parameters like the number of results to return (top_k), similarity score thresholds, and any metadata filters (e.g., WHERE year > 2020). The output is a ranked list of document IDs, their similarity scores (e.g., cosine similarity), and the associated metadata or text chunks.

Before documents can be embedded, they must be segmented into meaningful chunks. The chunking strategy is critical, as it defines the unit of retrieval. Common methods include:

• Fixed-size chunking: Simple but can split semantic concepts.
• Semantic chunking: Uses text coherence or embeddings to break at natural boundaries.
• Recursive chunking: Splits by characters, then by tokens, aiming for optimal sizes. The pipeline also handles text cleaning, normalization, and may extract metadata. Poor chunking can severely degrade retrieval performance by creating fragments with incomplete context.

A re-ranker is a secondary, more computationally intensive model that refines the results from the initial vector search. The dense retriever acts as a fast recall stage, fetching a broad set of candidate documents (e.g., top 100). The re-ranker, often a cross-encoder model like cross-encoder/ms-marco-MiniLM-L-6-v2, then evaluates the precise relevance of each query-document pair for superior precision. This two-stage process combines the speed of ANN search with the accuracy of more powerful, slower models, optimizing the overall quality of the final retrieved set.

The overarching information retrieval paradigm that dense retrieval enables. It moves beyond literal keyword matching (lexical search) to understanding the contextual meaning and intent behind queries and documents.

• Contrast with Sparse Retrieval: Does not rely on term frequency (e.g., TF-IDF or BM25). The query "automobile" can retrieve documents about "cars" even if the keyword is absent.
• Implementation: Typically involves creating a dense vector index of all documents and then querying it with an embedded user question.
• Hybrid Search: Often combined with sparse (keyword) retrieval methods to balance recall of exact terms with semantic understanding.

An architecture that uses dense retrieval as its core information-fetching component. RAG grounds a large language model (LLM) by retrieving relevant context from an external knowledge source (like a vector store) before generating an answer.

• Workflow: 1) User query is embedded. 2) Dense retrieval finds relevant document chunks. 3) Retrieved context is injected into the LLM prompt. 4) LLM generates a factually grounded response.
• Key Benefit: Mitigates LLM hallucinations by providing authoritative source material.
• Dependency: The performance of the entire RAG pipeline is critically dependent on the precision and recall of the underlying dense retrieval system.

A structured, graph-based alternative or complement to dense vector retrieval. It represents knowledge as a network of entities (nodes) and their relationships (edges). While dense retrieval finds semantically similar text, knowledge graphs enable explicit logical reasoning and traversal of factual connections.

• Query Method: Uses graph query languages like SPARQL or Cypher, not vector similarity.
• Strengths: Excellent for navigating known relationships (e.g., "find all employees who report to this manager"), enforcing ontological rules, and combining facts.
• Hybrid Approach: Often used with dense retrieval, where a knowledge graph provides structured facts and a vector store provides semantic similarity over unstructured text.