Vector Database

The fundamental capability of storing embedding vectors—dense numerical arrays typically ranging from 384 to 4096 dimensions—as first-class data types. Each vector represents a point in a continuous semantic space where proximity equals similarity.

• Stores vectors alongside metadata payloads (source, author, timestamp) for hybrid filtering
• Supports multiple embedding dimensions within a single collection
• Optimized storage engines use product quantization (PQ) to compress vectors by up to 97% while preserving recall
• Example: A 1536-dimension OpenAI embedding occupies ~6KB uncompressed; PQ reduces this to ~192 bytes

Vector databases rely on ANN algorithms to trade a small, controlled loss in recall for orders-of-magnitude gains in query speed. Without ANN, exact k-NN search over millions of vectors would require a brute-force scan of every vector.

• HNSW (Hierarchical Navigable Small World) builds multi-layer graph structures for logarithmic-time traversal
• IVF (Inverted File Index) partitions the vector space into clusters, searching only the nearest centroids
• DiskANN enables billion-scale search from SSD storage rather than expensive RAM
• Typical latency: < 10ms for top-10 retrieval over 1M vectors

Unlike static ANN libraries such as FAISS, vector databases support full create, read, update, and delete (CRUD) operations on live indexes without requiring a full rebuild. This is critical for production systems with continuously changing data.

• Upsert operations insert new vectors or update existing ones by unique ID
• Incremental indexing adds vectors to the HNSW graph or IVF cluster without rebuilding the entire structure
• TTL (Time-To-Live) fields automatically expire and remove stale vectors
• Compaction processes periodically merge small segments and reclaim space from soft-deleted vectors
• Example: Qdrant and Milvus support zero-downtime inserts while serving concurrent queries

The choice of distance metric fundamentally shapes retrieval behavior. Vector databases expose multiple similarity functions, and the correct choice depends on the embedding model's training objective.

• Cosine Similarity: Measures angular distance; normalizes for vector magnitude. Standard for text embeddings from Sentence Transformers and OpenAI
• Euclidean Distance (L2): Straight-line distance; sensitive to magnitude. Common for image embeddings
• Dot Product: Used when embeddings are not normalized; faster computation than cosine
• Inner Product: Equivalent to dot product; preferred for models trained with contrastive loss
• Index structures like HNSW must be configured for the specific metric at creation time

Production vector databases scale beyond single-machine memory limits through horizontal sharding, distributing vectors across multiple nodes while maintaining query consistency.

• Hash-based sharding distributes vectors uniformly by ID for balanced storage
• Range-based sharding partitions by metadata ranges for locality-aware queries
• Replication creates redundant copies of each shard for fault tolerance and read throughput
• Consistent hashing minimizes data movement when nodes join or leave the cluster
• Example: Milvus separates storage and compute, allowing independent scaling of index nodes, query nodes, and data nodes

The neural network that generates the vectors stored in the database. A model like all-MiniLM-L6-v2 maps text into a 384-dimensional space where semantically similar concepts cluster together. The choice of model dictates the embedding dimension and the quality of downstream retrieval; domain-specific fine-tuning often yields significant precision gains.

The standard distance metric used by most vector databases to compare embeddings. It measures the cosine of the angle between two vectors, normalizing for magnitude and focusing purely on directional alignment. A score of 1.0 indicates identical orientation, while 0.0 represents orthogonality. This metric is preferred because it ignores differences in vector length caused by document length variance.

The mechanism that combines vector search with structured predicates. A vector database stores both the high-dimensional embedding and its associated payload—fields like date, author, or status. Queries can pre-filter on these scalar attributes before performing ANN search, enabling precise scoping such as 'find semantically similar documents published after 2024'. This hybrid capability is essential for enterprise access control.

The class of algorithms that makes vector databases viable at scale. Exact k-NN search is computationally prohibitive on millions of high-dimensional vectors. ANN algorithms like HNSW and IVF-PQ sacrifice a small percentage of recall—often less than 1%—to achieve sub-millisecond query latency. This trade-off is the foundational engineering principle of all production vector databases.

The automated workflow that feeds the vector database. It orchestrates:

• Parsing: Extracting text from PDFs, HTML, and DOCX
• Chunking: Splitting documents into semantically coherent segments
• Embedding: Generating vectors via a bi-encoder model
• Upserting: Inserting vectors and metadata into the index This pipeline must handle incremental updates to avoid costly full re-indexing.