Vector Database

The fundamental capability that distinguishes vector databases from traditional relational or document stores. Rather than indexing scalar values for exact lookups, vector databases construct specialized data structures—such as Hierarchical Navigable Small Worlds (HNSW) graphs or Inverted File (IVF) indexes—that partition the embedding space into navigable regions. These structures enable sub-linear time complexity during retrieval, avoiding the brute-force O(n) cost of comparing a query vector against every stored embedding. The index is built once during insertion and queried repeatedly, trading increased memory footprint and build time for millisecond-range latency at query time.

Vector databases extend standard Create, Read, Update, and Delete operations to handle embeddings alongside associated metadata payloads. Each vector is stored with a payload—structured JSON containing attributes like timestamps, categories, or source URLs. This enables filtered vector search, where similarity queries are constrained by metadata predicates (e.g., 'find similar products in the electronics category priced under $500'). The database executes a two-phase operation: first applying attribute filters to narrow the candidate set, then performing ANN search within the filtered subset. Efficient implementations fuse these steps to avoid scanning the entire collection.

Storing raw 32-bit floating-point vectors for billion-scale collections demands terabytes of RAM. Vector databases employ quantization techniques to compress embeddings while preserving semantic fidelity:

• Product Quantization (PQ): Decomposes vectors into sub-vectors, clusters each subspace independently, and stores only the nearest centroid IDs. Achieves 4-8x compression.
• Scalar Quantization (SQ): Converts 32-bit floats to 8-bit integers per dimension. Simpler than PQ with 4x compression and minimal recall loss.
• Binary Quantization: Reduces each dimension to a single bit. Extreme compression (32x) suitable for high-throughput, lower-accuracy scenarios. These techniques enable in-memory search over datasets that would otherwise require disk-based storage, maintaining sub-10ms latency.

The foundational algorithm class that makes vector databases viable. ANN trades a small, controlled amount of accuracy for orders-of-magnitude speed improvements when searching high-dimensional spaces. Instead of comparing a query vector against every stored vector—a linear scan that becomes prohibitively slow at scale—ANN algorithms use graph-based or hashing techniques to prune the search space. Key algorithms include HNSW, IVF-PQ, and LSH. Without ANN, real-time semantic search over millions of embeddings would be computationally infeasible.

The primary distance metric used in vector databases to measure semantic relatedness. Cosine similarity calculates the cosine of the angle between two vectors, producing a value between -1 and 1. Critically, it ignores vector magnitude, focusing purely on orientation—this means a short document and a long document with the same semantic meaning will score highly. It is computed as the dot product of two L2-normalized vectors. Alternative metrics include Euclidean distance and inner product, but cosine similarity remains dominant for text embeddings.

A graph-based ANN algorithm that builds a multi-layered proximity graph. HNSW creates a hierarchy where top layers contain long-range edges for coarse navigation, and bottom layers contain short-range edges for fine-grained search. During retrieval, the algorithm greedily traverses from the top layer downward, achieving logarithmic time complexity. It consistently outperforms tree-based and hashing methods on high-recall benchmarks. Implementations like hnswlib are embedded in most production vector databases, including Weaviate and Milvus.

A memory compression technique essential for storing billion-scale embedding collections. PQ decomposes the original high-dimensional vector into multiple lower-dimensional sub-vectors, then quantizes each subspace independently using k-means clustering. Instead of storing full-precision floats, the database stores compact codes referencing cluster centroids. This can reduce memory footprint by 10-30x with minimal recall degradation. Combined with IVF indexing, PQ enables cost-effective semantic search over datasets that would otherwise require terabytes of RAM.

A retrieval strategy that fuses dense vector search with sparse keyword search (BM25) to capture both semantic meaning and exact lexical matches. Dense embeddings excel at understanding concepts but can miss rare terms like product codes or names. Sparse retrieval handles exact matches precisely but fails on paraphrases. Reciprocal Rank Fusion (RRF) is the standard algorithm for merging these result sets into a single ranking. Hybrid search is critical for enterprise RAG applications where both precision and recall are non-negotiable.

A content segmentation strategy that splits documents based on embedding similarity rather than fixed token counts. A sliding window moves through the text, computing cosine similarity between adjacent sentences. When similarity drops below a threshold—indicating a topic shift—a chunk boundary is inserted. This produces self-contained, semantically coherent chunks that align with how language models process information. Semantic chunking significantly improves retrieval precision compared to naive character splitting, especially for long-form technical documentation.