Stale-While-Revalidate

The stale-while-revalidate directive is specified within the HTTP Cache-Control header. It defines a grace period, in seconds, during which a cache can serve stale data. The syntax is Cache-Control: max-age=600, stale-while-revalidate=30. Here, max-age=600 means the response is fresh for 600 seconds. After this, it becomes stale, but the stale-while-revalidate=30 directive allows the cache to serve it for an additional 30 seconds while it revalidates in the background.

The primary benefit is a dramatic improvement in perceived latency and a smoother user experience. Instead of the user waiting for a synchronous network fetch on a cache miss, they receive a stale-but-available response instantly. The cache then performs an asynchronous revalidation request. This is critical for agent-side caching where an AI agent's responsiveness is paramount, preventing blocking operations during tool or API calls.

• Instant Response: User/agent gets data immediately, even if outdated.
• Background Updates: Fresh data is fetched without blocking the current operation.
• Reduced Load: Smoothes out request spikes to origin servers by avoiding synchronous herd behavior.

A cache implementing SWR operates through a defined state machine for each cached item:

1. Fresh: Age < max-age. Served directly from cache.
2. Stale (Revalidate Eligible): Age between max-age and max-age + stale-while-revalidate. Item is served stale and a background revalidation request is triggered.
3. Stale (Expired): Age > max-age + stale-while-revalidate. A normal cache miss occurs; the request must synchronously fetch from the origin.

On a background revalidation, the cache issues a conditional request (e.g., with an If-None-Match ETag header). If the origin responds with 304 Not Modified, the cached item's freshness is reset. If the origin responds with new data (200 OK), the cache is updated for the next request.

SWR differs fundamentally from other caching strategies:

• vs. Cache-Aside: In cache-aside, the app handles the cache. On a miss, the user request blocks until the data is fetched. SWR is often implemented at a lower level (e.g., CDN, HTTP client) and provides non-blocking stale serves.
• vs. Time-To-Live (TTL) with Hard Expiry: A simple TTL (max-age) makes data unavailable after expiry, causing latency. SWR adds a grace period.
• vs. must-revalidate: The must-revalidate directive requires synchronous validation with the origin once stale, blocking the user. SWR explicitly allows asynchronous revalidation.

In agent-side caching, SWR is ideal for data where immediate availability is more critical than absolute freshness. Examples include:

• Tool/API Response Caching: Caching the result of a database query or external API call (e.g., weather, stock price) that an LLM agent uses. The agent gets a fast, possibly slightly old, answer while the cache updates.
• Semantic Cache Results: Storing the results of previous LLM inferences or embeddings. A semantically similar user query can receive a stale cached completion while a fresh one is generated in parallel.
• Configuration & Schema Data: Caching external API schemas (OpenAPI) or tool definitions that change infrequently. The agent operates without delay while definitions are revalidated.

Effective use requires careful configuration and awareness of trade-offs:

• Setting Grace Periods: The stale-while-revalidate value must be tuned based on data volatility. Too long risks serving very outdated data; too short reduces the benefit.
• Background Request Throttling: Implement logic to deduplicate concurrent revalidation requests for the same resource to prevent stampedes.
• Error Handling: If the background revalidation request fails, the stale data typically continues to be served until the grace period expires. Monitoring is needed to detect persistent origin failures.
• Consistency Model: SWR provides eventual consistency. It is not suitable for data where strong consistency is required (e.g., financial transactions).

The cache-aside pattern is a foundational caching strategy where the application code (or AI agent) explicitly manages the cache. It is the most common pattern used in conjunction with logic like stale-while-revalidate.

• Flow: The application first checks the cache for data. On a cache miss, it fetches data from the primary source, stores it in the cache, and then returns it.
• Contrast with SWR: Cache-aside handles the population of the cache. Stale-while-revalidate defines the validation and refresh logic for cached data that already exists.

Time-To-Live (TTL) and the HTTP max-age directive define the absolute freshness lifetime of a cached item. This is the foundational timer that stale-while-revalidate extends.

• Mechanism: Once a cached response's age exceeds its max-age, it is considered stale.
• SWR Interaction: The stale-while-revalidate directive provides a grace period (e.g., stale-while-revalidate=60) after the max-age expires. During this period, stale data can be served while revalidation occurs.

Eventual consistency is a data consistency model where updates are propagated asynchronously. Systems using stale-while-revalidate often operate under this model.

• Principle: The system does not guarantee that all reads will immediately reflect the latest write, but it guarantees that all replicas will converge to the same state if no new updates are made.
• SWR Alignment: Serving stale data during revalidation means clients may temporarily see an older state. Once the background fetch completes, the cache is updated, moving the system toward consistency.

A read-through cache is a caching pattern where the cache itself is responsible for loading data on a miss. This contrasts with the application-managed cache-aside pattern.

• Architecture: The application always calls the cache. If data is absent, the cache fetches it from the primary source, stores it, and returns it.
• SWR Potential: A read-through cache can be enhanced with stale-while-revalidate logic. When a request for stale data arrives, the cache can immediately return the stale value and internally trigger an asynchronous refresh.

A cache stampede (or thundering herd problem) is a performance failure scenario that stale-while-revalidate helps mitigate.

• Scenario: When a popular cache item expires simultaneously for many clients (e.g., at the end of its TTL), a sudden surge of requests floods the primary data source.
• SWR as a Solution: By allowing stale data to be served during revalidation, SWR decouples user requests from the refresh trigger. Only one asynchronous revalidation request is typically made, even while thousands of users are served stale data.