Vector Database

Vector databases use specialized Approximate Nearest Neighbor (ANN) indexing algorithms to organize embeddings for efficient search. Unlike exact search, which is computationally prohibitive in high dimensions, ANN algorithms trade a small amount of precision for massive gains in speed and memory efficiency. Common algorithms include:

• HNSW (Hierarchical Navigable Small World): A graph-based method known for high recall and low latency.
• IVF (Inverted File Index): Clusters similar vectors into partitions (Voronoi cells) to narrow the search scope.
• Product Quantization (PQ): Compresses vectors by splitting them into subvectors and representing each with a centroid ID, drastically reducing memory footprint.

The primary data type is the dense vector embedding—a fixed-length array of floating-point numbers (e.g., 768 or 1536 dimensions) generated by models like BERT or CLIP. The database stores these vectors alongside their associated metadata (e.g., original text, image URL, timestamp). This hybrid storage model allows queries to filter by metadata (e.g., user_id = 'abc') before performing the computationally expensive vector similarity search, a process known as filtered search or pre-filtering.

The fundamental query is a k-Nearest Neighbor (k-NN) or Approximate k-Nearest Neighbor (k-ANN) search. Given a query vector, the system returns the k most similar stored vectors. Similarity is measured using distance metrics, with the choice impacting the geometric interpretation of the vector space:

• Cosine Similarity: Measures the cosine of the angle between vectors, ideal for text embeddings where magnitude is less important than direction.
• Euclidean Distance (L2): Measures the straight-line distance between vector points.
• Inner Product (Dot Product): Related to cosine similarity but affected by vector magnitude. The database internally optimizes computations for these metrics at scale.

To handle billions of vectors, databases implement horizontal scaling via vector sharding. Vectors are distributed across multiple nodes based on their proximity in the vector space (e.g., using the IVF algorithm's clusters) or by metadata. A coordinator node manages the query, fanning it out to relevant shards and aggregating results. This architecture allows capacity and query throughput to scale linearly with added nodes. Systems also manage memory hierarchy, keeping hot indices in RAM and spilling colder data to SSD.

Unlike static ANN libraries (e.g., FAISS), production vector databases support full Create, Read, Update, and Delete (CRUD) operations in real-time. This allows for dynamic applications where the knowledge base evolves:

• Insert: New vectors are added and the index is updated incrementally or via periodic rebuilds.
• Delete: Vectors are marked for deletion; indices are updated asynchronously.
• Update: Handled as a delete followed by an insert of the new vector. This capability is critical for applications like real-time recommendation feeds or chatbots with evolving knowledge.

Ensuring vectors and metadata are not lost is paramount. Features include:

• Write-Ahead Logging (WAL): Guarantees that operations are durable before being acknowledged to the client.
• Snapshotting & Point-in-Time Recovery: Creates consistent backups of the index and data.
• Replication: Synchronously or asynchronously copies data to follower nodes for high availability and read scaling.
• ACID Compliance: For metadata transactions, ensuring operations like filtered searches have a consistent view of the data. These features distinguish a database from an ephemeral, in-memory index.

An Approximate Nearest Neighbor (ANN) index is the core data structure that enables fast similarity search in high-dimensional spaces. Unlike exact k-NN search, which is computationally prohibitive at scale, ANN algorithms trade a small amount of precision for massive gains in query speed and memory efficiency.

• Key Trade-off: Enables sub-second search over billions of vectors by accepting approximate results.
• Common Algorithms: Includes HNSW, IVF (Inverted File Index), and LSH (Locality-Sensitive Hashing).
• Primary Function: The ANN index is what a vector database builds, maintains, and queries to perform semantic search.

Hierarchical Navigable Small World (HNSW) is a state-of-the-art, graph-based algorithm for constructing an ANN index. It is renowned for its high search speed and accuracy.

• Graph Structure: Organizes vectors into a multi-layered graph, where the top layer has few nodes and each lower layer is more densely connected.
• Search Process: Queries start at the top layer, navigating to the nearest neighbor, then proceed down the hierarchy for refinement.
• Performance: Often provides the best recall-speed trade-off for high-dimensional data and is the default algorithm in many vector databases like Weaviate and Qdrant.

Hybrid search is an advanced retrieval technique that combines vector-based (semantic) search with keyword-based (lexical) search to improve overall recall and precision.

• Vector Search: Finds semantically similar items (e.g., 'canine' matches 'dog').
• Keyword Search: Finds items with exact term matches or BM25 relevance.
• Fusion: Results from both methods are combined using algorithms like reciprocal rank fusion (RRF). This is crucial for enterprise search where filtering by exact metadata (e.g., a date or SKU) is as important as semantic understanding.

A unified embedding space is a shared, high-dimensional vector space where embeddings from different data modalities (text, image, audio) are directly comparable.

• Core Concept: Enables cross-modal retrieval (e.g., searching for images with a text query).
• Creation: Built using multimodal models like CLIP (for text-image) or ImageBind (for multiple modalities), which are trained to align different data types.
• Vector Database Role: The vector database stores these aligned embeddings, allowing for joint querying across modalities within a single index.

A knowledge graph is a semantic network that represents entities (nodes) and their relationships (edges). When integrated with a vector database, it creates a powerful neuro-symbolic system.

• Symbolic Reasoning: The graph provides explicit, logical facts and relationships (e.g., Company -> employs -> Person).
• Vector Complement: The vector store provides implicit, semantic similarity and contextual understanding.
• Combined Use Case: A query can first retrieve relevant entities from the knowledge graph and then use their vector representations to find semantically similar concepts, enabling complex, multi-hop reasoning for RAG systems.