Vector Tiered Storage

The core mechanism that dynamically migrates vectors between storage tiers without manual intervention. Policies are defined using:

• Access frequency (hot, warm, cold)
• Time-based rules (e.g., move after 30 days)
• Cost-performance targets The system continuously monitors access patterns and transparently moves data between tiers like in-memory caches, NVMe/SSD, and HDD/object storage.

Defines a clear performance and cost gradient for vector storage:

• Tier 0 (Hot): In-memory or NVMe SSD for sub-millisecond latency. Stores frequently accessed, recent, or critical vectors.
• Tier 1 (Warm): High-performance SSDs or local disks. Balances cost and speed for moderately accessed data.
• Tier 2 (Cold): High-capacity HDDs or low-cost cloud object storage (e.g., S3, GCS). For rarely accessed, archival, or historical vectors with latency measured in seconds.

The query engine automatically routes similarity searches to the correct tier, presenting a unified namespace to the application. Key behaviors include:

• Hot-path optimization: Queries first check the hottest tier (e.g., cache).
• On-demand promotion: Fetching a vector from a cold tier may trigger its promotion to a warmer tier for subsequent accesses.
• Metadata indexing: A lightweight master index tracks vector location across all tiers to enable fast routing decisions.

Administrators define declarative rules that govern the vector lifecycle across tiers. Common policies include:

• Time-to-Live (TTL): Automatically delete vectors after a set period.
• Access-based demotion: Move vectors to a colder tier after a defined period of inactivity.
• Cost caps: Ensure storage costs do not exceed a budget by aggressively moving data to cheaper tiers.
• Priority tagging: Manually pin critical vectors to high-performance tiers.

Dramatically reduces total storage cost by aligning media expense with data value. For example, storing 1TB of vectors:

• 100GB in-memory (expensive, fast)
• 400GB on NVMe SSD (moderate cost)
• 500GB on HDD/object storage (very low cost) This structure enables scaling to petabyte-scale vector collections while keeping operational expenses predictable, as the majority of data resides on the cheapest tier.

Ensures data integrity and crash consistency across all tiers. The system employs a persistent Write-Ahead Log (WAL) on durable storage. All vector insertions and updates are:

1. First written to the append-only WAL.
2. Acknowledged to the client.
3. Asynchronously applied to the appropriate tier's index. This guarantees no data loss if a node fails before the vector is written to its final tier, a critical feature for production systems.

The specialized database engine responsible for the persistent storage, indexing, and retrieval of high-dimensional vectors. It implements the low-level data structures (e.g., LSM-trees, B-trees) and access methods that a tiered storage system uses to move data between performance levels. Key characteristics include:

• High-throughput ingestion for streaming embedding data.
• Optimized disk layouts for vector I/O patterns.
• Integration points for tiering policies to trigger data movement.

A high-speed, low-latency storage layer (typically in-memory like Redis) that holds a subset of frequently accessed vectors or index metadata. It sits logically above the primary tiered storage to accelerate read operations. In a tiered architecture:

• The hottest data resides in the cache for sub-millisecond access.
• It acts as a buffer, reducing load on the primary SSD/HDD tiers.
• Cache misses trigger fetches from lower, slower storage tiers, making cache hit rate a critical performance metric.

A horizontal partitioning strategy that distributes vectors across multiple nodes or disks based on a shard key (e.g., vector ID, tenant ID, or a dimension of the vector). It enables scalability and parallel query execution, which interacts directly with tiered storage:

• Each shard can have independent tiering policies based on its local access patterns.
• Hot shards (frequently queried) may reside entirely on SSD, while cold shards are moved to HDD.
• Enables cost-effective scaling by adding commodity storage nodes for colder data tiers.

A critical durability mechanism for tiered storage systems. All data modifications (inserts, updates) are first written as sequential entries to a persistent, append-only log before being applied to the main vector index. This ensures:

• Crash consistency: The system can recover uncommitted changes after a failure.
• Data integrity across storage tiers during movement operations.
• The WAL itself is often stored on the fastest, most durable tier (e.g., high-performance SSD) to minimize write latency.

The use of scalable, durable cloud object stores (e.g., Amazon S3, Google Cloud Storage) as the coldest or archival tier in a vector storage hierarchy. Characteristics include:

• Extreme durability (99.999999999% typical) at very low cost.
• High latency for retrieval, making it suitable for infrequently accessed historical embeddings or index snapshots.
• Often used for disaster recovery backups of hotter-tier vector data.
• Data is typically retrieved as entire files (e.g., index segments) rather than individual vectors.

The formal guarantee provided by a distributed vector database regarding how and when writes (including tier movements) become visible to subsequent reads. This is crucial in a tiered system where data may be replicated or moved asynchronously. Common models include:

• Strong Consistency: A read returns the most recent write, guaranteeing data is in sync across all tiers and replicas.
• Eventual Consistency: Writes (like a move to cold storage) will propagate to all replicas over time, allowing for temporary stale reads but higher availability.
• The chosen model directly impacts the complexity and performance of tiering operations.