Cache Consistency

Strong consistency is a guarantee that any read operation will return the most recent write for a given data item. This model ensures that all clients, regardless of which cache node they access, see the same data at the same time after an update.

• Mechanism: Typically implemented via synchronous write-through or write-invalidate protocols. A write is not considered complete until it has been propagated to all replicas and caches.
• Trade-off: Provides linearizability and simplifies application logic but introduces higher latency for write operations and can reduce overall system availability.
• Use Case: Critical for financial transactions, inventory management systems, or any scenario where reading stale data could cause a business logic failure.

Eventual consistency is a model where, after an update, the system guarantees that all replicas will converge to the same value if no new updates are made for a period of time. Reads may temporarily return stale data.

• Mechanism: Updates are propagated asynchronously. Common in distributed systems using anti-entropy protocols, gossip, or conflict-free replicated data types (CRDTs).
• Trade-off: Offers high availability and low-latency writes but requires applications to tolerate temporary state divergence.
• Use Case: Ideal for social media feeds, DNS systems, and collaborative editing tools where immediate global consistency is not required.

Causal consistency is a model that preserves the "happened-before" relationships between operations. If operation A causally affects operation B (e.g., a reply to a comment), then any process that sees B will also see A.

• Mechanism: Achieved by tracking causal dependencies, often using vector clocks or version vectors to tag operations.
• Trade-off: Stronger than eventual consistency but weaker than strong consistency. It prevents confusing anomalies while allowing concurrent, unrelated writes to proceed independently.
• Use Case: Well-suited for chat applications, comment threads, and notification systems where the causal order of events is critical for user experience.

Read-your-writes consistency (also known as session consistency) is a guarantee that a process will always see the effects of its own previous writes within the same session, even if other users see stale data.

• Mechanism: Often implemented by pinning a user's session to a specific cache replica or by tracking a client's write sequence number and ensuring subsequent reads reflect at least that version.
• Trade-off: Provides a good user experience for single users without the global coordination overhead of strong consistency.
• Use Case: Essential for web applications where a user expects to see their own profile updates, shopping cart changes, or posted content immediately after submission.

Monotonic read consistency ensures that if a process reads a particular version of a data item, all its subsequent reads will return that same version or a more recent one. It prevents time from moving backward for a single reader.

• Mechanism: The system tracks the most recent version seen by a client and ensures future reads are at least as fresh. This can be managed via client-side tokens or server-side session state.
• Trade-off: Prevents confusing behavior where a user sees newer data and then older data, but does not guarantee they see the latest data globally.
• Use Case: Useful in systems like news feeds or product listings where a user should not see items disappear and reappear upon refresh.

Consistent prefix (or prefix consistency) guarantees that reads will observe a sequence of writes in an order that respects their global commit order. It prevents seeing a subset of transactions out of sequence.

• Mechanism: The system ensures that if write B depends on or occurs after write A, a client will never see B without also having seen A. This is often managed through logical timestamps.
• Trade-off: Stronger than eventual consistency but achievable with less coordination than strong consistency. It is crucial for maintaining logical integrity.
• Use Case: Critical in database replication, distributed ledgers, and systems where operations build upon previous state, such as applying a series of configuration changes.

Cache invalidation is the active process of marking cached data as stale or obsolete, triggering its removal or refresh. It is the primary mechanism for enforcing strong consistency by ensuring subsequent requests fetch fresh data from the primary source.

• Explicit Invalidation: The application or an external event (e.g., a database update) directly deletes the specific cache key.
• Time-Based Invalidation: Uses a Time-To-Live (TTL) to automatically expire entries after a set duration.
• Challenge: Known as one of the two hard problems in computer science, improper invalidation is a common source of bugs leading to stale data.

Eventual consistency is a relaxed cache consistency model where updates to the primary data source are propagated to all caches asynchronously. It does not guarantee that all reads will immediately see the latest write, but it does guarantee that all replicas will converge to the same, most recent state given enough time without further writes.

• Trade-off: Sacrifices strong consistency for higher availability and lower latency.
• Use Case: Ideal for agent-side caching of non-critical, read-heavy data where minor staleness is acceptable (e.g., user profile avatars, product descriptions).
• Mechanism: Often implemented via change data capture (CDC) streams or periodic cache refresh jobs.

A write-through cache is a caching pattern that enforces strong consistency by writing data synchronously to both the cache and the primary data source (e.g., a database). The write operation is only considered complete after both writes succeed.

• Guarantee: The cache always contains the latest data, ensuring subsequent reads are consistent.
• Performance Impact: Introduces higher write latency compared to write-behind patterns, as the application waits for the slower primary storage.
• Agent-Side Use: Crucial for agent tools that modify critical state (e.g., updating a CRM record) where data integrity is paramount.

Cache coherence is the property that guarantees all copies of the same data across multiple caches in a distributed system maintain a consistent view. It prevents different agents or application instances from reading different (stale) values for the same logical data item.

• Scope: A system-level concern, broader than single-cache consistency.
• Protocols: Implemented via protocols like MESI (Modified, Exclusive, Shared, Invalid) in CPUs or via distributed consensus in software caches like Redis.
• Agent Challenge: In a multi-agent system, ensuring cache coherence is critical to prevent agents from acting on conflicting, outdated information.

A cache stampede (or thundering herd) is a performance failure scenario where the simultaneous expiration or invalidation of many cached items causes a sudden, overwhelming surge of requests to hit the primary data source. This can lead to latency spikes, timeouts, and even cascading failures in the backend system.

• Cause: Often triggered by a scheduled TTL expiration or a service restart.
• Mitigation: Techniques include staggered expiration (jitter), background refresh, lease mechanics, or using a Bloom filter cache to coordinate a single regenerating request.
• Consistency Link: Aggressive cache invalidation policies can inadvertently trigger stampedes.

A deterministic cache stores the results of pure, side-effect-free functions where the same input arguments always produce the identical output. This property guarantees that a cached result is always valid for a given input, simplifying consistency concerns.

• Foundation for Semantic Caching: Enables caching of LLM inferences or API calls where the request can be canonicalized into a deterministic key.
• No Invalidation Needed: Because the output is purely a function of its input, the cache never becomes inconsistent—only potentially unnecessary if the underlying logic changes.
• Agent Application: Ideal for caching the results of idempotent tool calls, mathematical computations, or data transformations performed by an agent.