Cache Stampede

The core mechanism of a cache stampede is the thundering herd effect. When a hot cache key expires, all concurrent requests for that data simultaneously miss the cache. Without coordination, every single request thread independently attempts to regenerate the value from the origin database. This massive spike in load can saturate database connections, exhaust CPU, and cause a complete service outage. The problem is particularly acute for items that are expensive to compute or retrieve.

A cache stampede is not caused by steady high traffic but by a sudden, synchronized burst of concurrency targeting a single resource. Common triggers include:

• Expiration of a popular key: A news article, product page, or configuration value that receives thousands of requests per second.
• Cold cache restart: When a cache server is rebooted and all keys are empty, the first requests for every popular item create simultaneous stampedes.
• External events: A flash sale or viral social media post that drives traffic to a previously uncached resource.

The victim of a cache stampede is the origin datastore—the primary database or computation service behind the cache. The origin is typically provisioned for a normal, cache-mitigated load, not the full brunt of all user traffic. A stampede can cause:

• Connection pool exhaustion: All available database connections are consumed by redundant regeneration requests.
• Cascading failure: A slow or unresponsive database causes upstream services to time out, depleting their thread pools and spreading the outage.
• Write amplification: If the regeneration logic involves a write-back to the cache, the stampede can also flood the cache itself with redundant writes.

A primary mitigation strategy is probabilistic early recomputation. Instead of setting a fixed Time-To-Live (TTL), the system triggers a controlled refresh before expiration when a key is accessed. The probability of triggering an early refresh increases as the key approaches its expiry. This ensures that only a single process regenerates the value while the cache is still warm, completely preventing a stampede. This is often implemented using an algorithm like XFetch or a simple randomized check.

Another robust solution is to use an external lock to coordinate the regeneration process. When a cache miss occurs, the first request acquires a distributed lock (e.g., using Redis or etcd) for that specific key. Other concurrent requests wait on the lock or serve a stale value. The lock holder computes the new value, updates the cache, and releases the lock. This guarantees that the origin database is hit exactly once, regardless of the number of concurrent requests.

The stale-while-revalidate pattern instructs the cache to serve a potentially stale item to all requesters while asynchronously triggering a single background refresh. This eliminates client-facing latency and protects the origin by decoupling the read path from the write path. The cache serves the expired data immediately, and the next request after the background refresh completes will receive the fresh value. This is a standard HTTP caching directive (Cache-Control: stale-while-revalidate) that can be implemented at the CDN or application cache level.

A proactive mitigation strategy where the cache randomly expires entries early before their absolute TTL. When a request arrives and the entry is within this early-expiry window, the cache serves the stale value while asynchronously refreshing it from the origin. This spreads out the refresh load over time, preventing the synchronized thundering herd. The probability of triggering a refresh typically increases as the expiry deadline approaches, often implemented using an exponential backoff function tied to the remaining TTL.

A reactive pattern where the first request that encounters a cache miss acquires a distributed lock for that specific key. All other concurrent requests for the same key are forced to wait or served stale data. The lock holder computes the fresh value, populates the cache, and releases the lock. This guarantees that only one request hits the origin database per cache miss. Implementations often use Redis with SETNX or a similar atomic operation, but must handle lock timeouts and dead processes to avoid permanent deadlock.

Instead of waiting for a cache miss, a separate system component—such as a cron job, CDC pipeline, or event listener—pre-computes and refreshes cache entries before they expire. This decouples cache population from user traffic entirely. When a database write occurs, a change data capture event triggers an immediate cache update. This pattern is ideal for hot keys with predictable update patterns and eliminates the stampede condition at its root by ensuring the cache never goes cold for critical data.

A distributed hashing technique that maps keys to cache nodes in a ring topology. When a cache node fails or is added, only a fraction of keys—those mapped to the affected node's range—need to be redistributed. Without consistent hashing, a node failure could invalidate a massive percentage of cached entries, triggering a cluster-wide stampede as all affected keys simultaneously miss. This pattern is foundational to distributed caches like Memcached and Redis Cluster, ensuring that infrastructure changes don't cascade into application-level failures.

A resilience pattern that acts as a safety valve when the origin database is already overwhelmed. The circuit breaker monitors the failure rate of calls to the origin. If the failure rate exceeds a threshold, the breaker trips to an open state, immediately failing all requests without calling the origin. This prevents the stampede from compounding the overload and gives the database time to recover. After a cooldown period, the breaker transitions to a half-open state, allowing a limited number of test requests to probe if the origin is healthy again.

A strategy where writes are applied to the cache first and then asynchronously persisted to the database. While primarily a write-pattern optimization, it indirectly influences stampede dynamics. If the cache holds the authoritative latest state, a cache miss on a read is less catastrophic because the data can be reconstructed from a durable commit log rather than a slow relational query. This pattern is common in systems using Apache Kafka as a log-backed source of truth, where cache population replays the log rather than querying a fragile origin.