Data Tiering

Data tiering is fundamentally governed by automated policies that dictate when and how data is moved. These policies are based on explicit rules or learned access patterns.

• Rule-Based Policies: Use static criteria like creation time, last access time, or data type (e.g., 'move logs older than 30 days to archive').
• Access-Pattern Learning: More advanced systems use machine learning to analyze usage frequency and predict future needs, proactively tiering data.
• Policy Triggers: Movement can be triggered by events like a cache reaching capacity, a scheduled job, or a change in data state (e.g., marked 'inactive').

The strategy relies on a hierarchy of storage tiers, each with distinct cost-performance trade-offs. Data is placed on the tier that best matches its current value and access needs.

• Hot Tier (Performance): High-speed, low-latency storage like RAM or NVMe SSDs. Used for active working sets, recent context, or frequently accessed knowledge. Highest cost per GB.
• Warm Tier (Balanced): Slower SSDs or high-performance cloud object storage. Holds data accessed moderately or periodically.
• Cold/Archive Tier (Economy): High-latency, high-capacity storage like HDDs, tape, or cloud archive services (e.g., AWS Glacier). For compliance data, old logs, or rarely accessed historical experiences. Lowest cost per GB.

A core characteristic is that data movement between tiers should be transparent to the consuming application or agent. The system handles the physical location, presenting a unified logical view.

• Location Abstraction: The agent queries memory using an ID or semantic search; the tiering system retrieves it from the appropriate tier without the agent needing to know where it was stored.
• On-Demand Promotion: When cold data is requested, it is automatically 'promoted' back to a hotter tier (e.g., loaded into a vector cache), often with some retrieval latency.
• Asynchronous Demotion: Moving data to a colder tier is typically done asynchronously in the background to avoid blocking agent operations.

The primary engineering driver for data tiering is infrastructure cost reduction. By ensuring only the most valuable data resides on expensive storage, total ownership cost is minimized.

• Example: Storing 1TB of embeddings in RAM may cost ~$400/month, while the same data in a cold object storage tier may cost <$5/month.
• Trade-off Management: Engineers configure policies to balance retrieval latency (performance) against storage cost. The goal is to maximize the ratio of performance to cost.
• Predictive Tiering: Advanced systems predict when data will be needed next, moving it to a warmer tier just-in-time to avoid paying for unnecessary high-performance storage.

Data tiering works in concert with cache eviction policies but operates at a different granularity and timeframe. While eviction removes data, tiering relocates it.

• Eviction as a Trigger: When an LRU policy evicts an item from a hot in-memory cache, a tiering policy may decide to write it to a warm disk-based tier instead of discarding it entirely.
• Hierarchical Eviction: A multi-tier system may have eviction policies at each level. An item evicted from L1 (RAM) moves to L2 (SSD), and only when evicted from the final tier is it truly deleted.
• Policy Coordination: The tiering logic must understand the semantics of the data to make intelligent movement decisions, not just recency/frequency.

For autonomous agents, data tiering manages the lifecycle of context, experiences, and learned knowledge.

• Short-Term/Working Memory: Resides in the hot tier (e.g., the LLM's context window or a fast vector cache). This is the agent's immediate focus.
• Long-Term/Episodic Memory: Moved to a warm tier (e.g., a vector database on SSD) after the interaction ends. Can be retrieved via semantic search for future sessions.
• Archival Knowledge & Logs: Infrequently accessed training data, interaction histories for compliance, or old policy versions are stored in a cold tier.
• State Serialization: An agent's entire operational state can be serialized and tiered during periods of inactivity, enabling cost-effective persistence.

A predetermined algorithm that determines which items to remove from a cache when it reaches its capacity limit. Unlike data tiering, which moves data between storage classes, eviction policies delete data to make space.

• Key Examples: LRU (Least Recently Used), LFU (Least Frequently Used), FIFO (First-In, First-Out).
• Primary Goal: Maximize cache hit rate by retaining the most valuable data.
• Contrast with Tiering: Eviction is a removal decision; tiering is a relocation decision based on cost/performance trade-offs.

A high-performance storage engine data structure that implicitly implements a form of tiering. It batches writes in a fast, memory-resident component (the memtable) and periodically merges sorted data into slower, immutable files on disk.

• Tiering Mechanism: Data automatically flows from a fast, mutable tier (memory) to slower, immutable tiers (disk).
• Optimization: Excellent for write-heavy workloads (e.g., databases like Apache Cassandra, RocksDB).
• Contrast: LSM trees are a specific architectural implementation, while data tiering is a broader strategy applicable to many storage systems.

A storage optimization technique closely related to tiering logic. Data modifications are written only to a fast cache tier initially. The update is deferred to the slower backing storage tier until the cache block is evicted or explicitly flushed.

• Performance Benefit: Reduces latency for write operations by acknowledging writes to fast storage first.
• Risk: Introduces a consistency window where the durable store is stale; a system crash before flush can cause data loss.
• Use Case: Common in CPU caches, RAID controllers, and distributed file systems.

A principle defining the subset of total data a process actively needs within a specific time interval to operate efficiently. Data tiering policies often use the working set to decide what belongs in the performance tier.

• Core Idea: Keep the working set in fast storage (e.g., RAM, SSD) to minimize access latency.
• Policy Driver: Tiering systems monitor access patterns to identify the active working set versus colder, archival data.
• Application: Foundational to operating system memory management (paging) and database buffer pool design.

A database isolation technique that maintains multiple versions of data items. While not tiering itself, MVCC creates a natural tiering requirement: recent, active versions need fast access, while older versions can be moved to cheaper storage or purged.

• Version Proliferation: Can lead to significant storage overhead from historical data.
• Tiering Integration: Database systems often tier old MVCC snapshots or UNDO logs to lower-cost object storage.
• Example: PostgreSQL's vacuum process cleans up old row versions, a form of internal lifecycle management.

The lowest-cost, highest-latency tier in a data tiering hierarchy, designed for data that is rarely accessed but must be retained for compliance or historical analysis.

• Characteristics: Very low cost per GB, high retrieval latency (minutes to hours), and often higher durability.
• Access Model: Typically requires a restore operation before data becomes readable in a hotter tier.
• Technologies: Amazon S3 Glacier, Azure Archive Storage, Google Cloud Storage Coldline. Data is moved here by tiering policies after a defined period of inactivity.