Write-Back Cache

The core principle of a write-back cache is deferred updating. When a processor writes data, the operation updates only the cache copy. The corresponding location in main memory is not updated at that moment. The modified cache line is marked as dirty (or modified). The actual write-back to main memory occurs only when that cache line is evicted (replaced) to make room for new data. This reduces memory traffic and write latency significantly compared to a write-through policy.

Each cache line in a write-back cache is associated with a dirty bit (or modified flag). This single bit of metadata is crucial for correct operation.

• Set to 1 (Dirty): Indicates the cached data is newer than the copy in main memory.
• Set to 0 (Clean): Indicates the cached data is identical to main memory. On eviction, the cache controller checks this bit. If dirty, the entire line must be written back to memory. If clean, the line can be simply discarded, saving bandwidth. This mechanism ensures data consistency while minimizing unnecessary writes.

Write-back caches are designed to minimize off-chip memory traffic, a key goal for power and performance in accelerators. Multiple writes to the same cache line before eviction result in only one final write-back to main memory. This is especially beneficial for workloads with high write locality, where data is updated repeatedly. By coalescing writes, the cache acts as a write buffer, dramatically reducing the bandwidth required on the memory bus compared to a write-through cache, which writes every store operation to memory.

In multi-core or multi-accelerator systems, the deferred writes of a write-back policy introduce complexity for cache coherence. Since the most recent data may reside only in a private cache, a coherence protocol (like MESI) is required to track line state and manage ownership. Before another processor can read a memory location, any dirty copy in a remote cache must be retrieved or written back. This generates snoop traffic and potential latency for shared data, making the design of the coherence protocol critical for system performance.

Write-back caches are typically paired with a write-allocate policy on a cache miss. When a write misses the cache:

1. The target cache line is allocated (fetched from memory).
2. The write updates the newly allocated line in the cache.
3. The line is marked as dirty. This policy leverages spatial locality, assuming subsequent writes (or reads) will target nearby data in the same line. The alternative, no-write-allocate (or write-around), writes directly to memory on a miss, which is less common with write-back as it forfeits the benefit of caching the written data.

A significant trade-off of the write-back policy is volatility risk. Dirty data that has not yet been written back to persistent main memory (like DRAM) resides only in the volatile cache. A sudden power failure or system crash can lead to permanent data loss or inconsistent system state. This is mitigated in critical systems by:

• Flush instructions: Software can explicitly write back dirty lines.
• Battery-backed caches: Using non-volatile memory for the cache itself.
• Periodic flushing: Operating system or hardware mechanisms to periodically write back dirty data.

A contrasting cache policy where every write operation updates both the cache and the main memory simultaneously. This ensures memory consistency but increases write latency and memory bandwidth consumption compared to a write-back policy.

• Key Difference: Write-through provides simpler cache coherence at the cost of performance for write-heavy workloads.
• Use Case: Often used in systems where data integrity is paramount and writes are infrequent, or where a simpler coherence protocol is required.

A critical property in multi-core or multi-accelerator systems that ensures all caches have a consistent view of shared memory. Write-back caches introduce complexity for coherence protocols because modified data exists only in the cache until eviction.

• Protocols: Snoopy and directory-based protocols (like MESI) track cache line states (Modified, Exclusive, Shared, Invalid) to manage updates and invalidations.
• NPU Context: In a system with multiple NPU cores, maintaining coherence between their local caches and shared memory is essential for correct parallel execution.

A status flag associated with each cache line in a write-back cache. The dirty bit indicates whether the data in the cache has been modified relative to the copy in main memory.

• Eviction Trigger: When a cache line needs to be replaced, the cache controller checks the dirty bit. If set (dirty), the line must be written back to main memory. If not set (clean), it can simply be discarded.
• System Efficiency: This mechanism is what allows a write-back cache to minimize unnecessary memory traffic, writing back only modified data.

A small, high-speed FIFO queue that sits between a write-back cache and main memory. Its purpose is to decouple the cache from the slower main memory during write-back operations.

• Function: When a dirty cache line is evicted, the data is placed into the write buffer. The cache is immediately freed for new data, while the buffer handles the asynchronous write to memory.
• Performance Benefit: This hides the latency of the memory write, preventing the cache from stalling while waiting for the write-back to complete.

An alternative on-chip memory structure common in NPUs and DSPs. Unlike a hardware-managed cache, a scratchpad is explicitly managed by software (the programmer or compiler).

• Key Contrast: Data movement to/from a scratchpad is deterministic and controlled, eliminating the unpredictability of cache misses. Write-back caches are automatic and transparent.
• Trade-off: Scratchpads offer predictable, low-latency access for known data patterns but require more complex programming. Write-back caches simplify programming but introduce variable latency.

An event where requested data is not found in the cache, forcing access to a slower level of the memory hierarchy. Write-back policy influences the handling of certain miss types.

• Write Miss Policy: On a write miss, a system may use write-allocate (fetch the line into cache, then update it) or no-write-allocate (write directly to memory, bypassing cache). Write-back caches typically use write-allocate.
• Performance Impact: A write-back operation triggered by an eviction can delay the servicing of a subsequent cache miss, as the memory channel is occupied.