Vector Sharding

The shard key is the attribute used to deterministically assign a vector to a specific shard. Common strategies include:

• Random Distribution: Vectors are assigned to shards via a hash function, ensuring an even load but ignoring data locality.
• Dimensionality-Based: Vectors are partitioned based on a specific dimension range (e.g., all vectors where dimension 1's value is between 0.0 and 0.5).
• Metadata-Based: Vectors are grouped by a categorical metadata field (e.g., user_id, tenant_id), ensuring all vectors for a given entity reside on the same shard for efficient filtered searches. The choice of shard key directly impacts query performance and data locality.

A query is broadcast to all shards simultaneously. Each shard performs a local Approximate Nearest Neighbor (ANN) search on its subset of vectors. The local top-K results from each shard are then merged and re-ranked by a coordinator node to produce the final global top-K results. This architecture:

• Reduces Latency: Search time scales with the size of the largest shard, not the total dataset.
• Increases Throughput: Multiple queries can be processed concurrently across different shards.
• Introduces Overhead: The merge phase adds coordination latency and network I/O.

Sharding enables horizontal scaling (scale-out). As the vector dataset grows, new shards can be added to the cluster without downtime through a resharding process. This contrasts with vertical scaling (scale-up), which is limited by single-node hardware. Key mechanisms include:

• Dynamic Rebalancing: Automatically moving vectors between shards when nodes are added or removed to maintain even data distribution.
• Elastic Clusters: Cloud-native vector databases use sharding to automatically provision or decommission compute/storage based on load. The goal is near-linear scalability for both ingest and query workloads.

Sharding is typically combined with replication. Each shard (a primary) has one or more replicas on different physical nodes. If a node fails:

• Reads can be served from replicas.
• Writes are redirected to a promoted replica.
• Data Durability is maintained as copies exist elsewhere. This design provides high availability and data redundancy. The replication factor (e.g., 3x) trades storage cost for resilience. Systems like Raft or Paxos are often used to maintain consistency across replicas of a shard.

Effective sharding maximizes data locality—keeping related vectors together to minimize cross-shard queries. For hybrid or filtered search, where a vector similarity search is combined with metadata filters (e.g., user_id=123), the shard key should align with common filter attributes. If user_id is the shard key, a filtered query for a specific user executes on a single shard, avoiding the broadcast overhead. Poor shard key choice leads to query fan-out, where all shards must be queried and results filtered post-hoc, degrading performance.

Distributed sharding introduces trade-offs between consistency, availability, and partition tolerance (CAP theorem). Common models include:

• Strong Consistency: Reads reflect the latest write across all shards/replicas, often at the cost of higher latency. Uses protocols like two-phase commit.
• Eventual Consistency: Writes propagate asynchronously; reads may temporarily return stale data, offering higher availability and lower latency.
• Causal Consistency: Guarantees that causally related writes are seen by all nodes in the same order. The coordinator node manages transaction ordering, global clock synchronization (e.g., Hybrid Logical Clocks), and conflict resolution.

The process of creating and maintaining redundant copies of vector data across different storage nodes or geographical regions. Its primary purposes are high availability and fault tolerance. If one node fails, a replica can serve queries. It also enables load balancing for read-heavy workloads by distributing queries across replicas. Common strategies include:

• Leader-Follower (Primary-Secondary): Writes go to a primary node and are asynchronously replicated to followers.
• Multi-Leader: Multiple nodes accept writes, requiring conflict resolution.
• Geographic Replication: Copies are maintained in different data centers for disaster recovery and low-latency global access.

The formal guarantee provided by a distributed vector database regarding the visibility and ordering of read and write operations across shards and replicas. This is a critical trade-off in system design, balancing latency with correctness.

Common models include:

• Strong Consistency: A read is guaranteed to return the most recent write. This simplifies application logic but increases latency.
• Eventual Consistency: Replicas will converge to the same state given no new updates, offering lower latency but potential for stale reads.
• Causal Consistency: Guarantees that causally related operations are seen by all nodes in the same order, a middle-ground often used in distributed systems.

The choice directly impacts application behavior when querying a sharded and replicated vector store.

An automated storage architecture that moves vector data between different performance and cost tiers based on access patterns. This optimizes total cost of ownership (TCO) for large-scale embedding storage.

Typical Tiers:

• Hot Storage (SSD/NVMe): For frequently accessed, latency-sensitive vectors and active index structures.
• Warm Storage (HDD/Standard Cloud Disk): For less frequently queried historical embeddings.
• Cold/Archival Storage (Object Storage like S3): For rarely accessed vectors, used for compliance or long-term retention.

Policies automatically promote vectors to hotter tiers upon access and demote them based on inactivity. This is complementary to sharding, which distributes data horizontally.

A design characteristic that minimizes system downtime and ensures continuous operation of the vector database. It is achieved through the combined use of sharding, replication, and automatic failover mechanisms.

Key Components:

• Redundancy: No single point of failure for data (via replication) or compute (via multiple shard nodes).
• Health Monitoring: Continuous checks on node heartbeat, disk space, and network connectivity.
• Automatic Failover: If a primary shard leader fails, the system automatically promotes a healthy replica to leader without manual intervention.
• Load Balancers: Distribute client connections away from unhealthy nodes.

The Service Level Agreement (SLA) for a vector database quantifies its high availability, e.g., 99.9% uptime.

A fundamental durability mechanism where all modifications to vector data (inserts, updates, deletes) are first written to a persistent, append-only log before being applied to the main in-memory index or on-disk structure. This ensures data integrity and crash recovery.

Process:

1. A write request is received.
2. The operation is serialized and appended to the WAL on stable storage (disk).
3. Only after the WAL write is confirmed is the operation applied to the volatile index.
4. On a crash restart, the system replays the WAL to reconstruct the last consistent state.

In a sharded system, each shard typically maintains its own WAL, which is also crucial for replicating operations to follower nodes.

The principle of storing vectors that are frequently queried together on the same physical node, disk, or memory region. The goal is to minimize network latency and I/O overhead during similarity search operations.

How it relates to sharding: A good shard key strategy aims to maximize locality. For example, sharding by a tenant_id or user_id metadata field ensures all vectors for a single entity are co-located on one shard. This means queries for that entity's data are served locally on that node without costly cross-shard fan-out.

Benefits:

• Reduced Query Latency: Eliminates network hops between shards.
• Improved Cache Efficiency: Related vectors are loaded into the same node's memory cache.
• Simplified Transactions: Operations on co-located data can be atomic within a single shard.