Out-of-Order Execution

The foundational hardware algorithm enabling out-of-order execution. It uses reservation stations to hold instructions until their operands are available, implementing register renaming to eliminate false data dependencies (WAR and WAW hazards). When an operand becomes ready, it is broadcast on the Common Data Bus (CDB) to all waiting reservation stations simultaneously, allowing dependent instructions to execute immediately without waiting for a write-back to the register file.

A critical technique that maps architectural registers to a larger pool of physical registers, eliminating write-after-write (WAW) and write-after-read (WAR) hazards. By dynamically allocating a new physical destination register for every write, the processor breaks false dependencies that would otherwise serialize execution. This is essential for exposing instruction-level parallelism in AI tensor operations where intermediate results are frequently overwritten.

A circular hardware queue that ensures precise exceptions and in-order retirement. Instructions are allocated ROB entries in program order, execute out-of-order, and then commit their results to the architectural state strictly in program order. If an instruction triggers an exception, all subsequent instructions in the ROB are flushed, preserving the illusion of sequential execution. The ROB size directly limits the instruction window.

To keep pipelines full, processors speculatively execute instructions past unresolved branches using branch prediction. If a misprediction occurs, the speculative state is discarded. However, these discarded paths leave measurable microarchitectural side-effects in caches and branch predictors. This is the root cause of Spectre and Meltdown vulnerabilities, where attackers exfiltrate secrets by training the branch predictor and measuring cache timing differences.

Out-of-order memory access creates a divergence between program order and observed memory order. Modern architectures define a memory consistency model (e.g., x86-TSO, ARM's relaxed model) that specifies what reorderings are legal. Memory barriers (fences) like MFENCE or DMB enforce ordering constraints, preventing reordering of loads and stores. For AI security, fences are critical to ensure that security-critical checks are not bypassed by speculative loads.

The instruction window is the pool of decoded instructions from which the scheduler selects ready operations. A larger window exposes more instruction-level parallelism (ILP), allowing the processor to find independent work to hide long-latency operations like matrix multiplications. AI inference workloads, dominated by dense linear algebra, exhibit high ILP, making large instruction windows and deep out-of-order pipelines critical for achieving high throughput on CPUs.

A CPU optimization where the processor predicts the likely path of a branch and executes instructions along that path before the branch condition is resolved. If the prediction is correct, the results are committed, saving cycles. If incorrect, the speculatively executed instructions are discarded and the pipeline is flushed.

• Branch Prediction Unit (BPU): Hardware that learns branch patterns to guess outcomes
• Spectre Vulnerability: Exploits speculative execution to leak data via cache-timing side channels
• Transient Instructions: Operations executed speculatively that leave microarchitectural traces even when discarded

A hardware optimization that proactively loads data from main memory into the CPU cache before the program explicitly requests it. Prefetchers analyze memory access patterns—such as sequential or strided reads—to anticipate future loads.

• Hardware Prefetchers: Integrated into the memory controller, operating transparently to software
• Software Prefetching: Explicit PREFETCH instructions inserted by compilers
• Cache-Timing Attack Vector: Prefetch operations can be abused to prime or evict cache lines, leaking information about secret-dependent access patterns

The algorithm that determines which cache line to remove when a new block must be loaded into a full cache set. This policy directly impacts hit rates, latency, and the predictability of memory access for AI inference workloads.

• LRU (Least Recently Used): Evicts the line accessed furthest in the past; common but complex to implement
• Pseudo-LRU: A tree-based approximation requiring fewer bits per cache set
• RRIP (Re-Reference Interval Prediction): Predicts how soon a line will be reused, often outperforming LRU for workloads with mixed access patterns
• Security Implication: Eviction-based attacks like Prime+Probe exploit deterministic replacement policies to infer victim cache usage

An attack that extracts secrets by observing physical or microarchitectural side effects of computation—timing, power consumption, electromagnetic emissions, or cache state—rather than breaking the algorithm mathematically.

• Cache-Timing Attacks: Measure access latency to infer whether a specific cache line was loaded by a victim process (Flush+Reload, Prime+Probe)
• Spectre/Meltdown: Exploit speculative and out-of-order execution to transiently access secrets and encode them into cache state
• Mitigations: Constant-time programming, cache partitioning (CAT), and serializing instructions like LFENCE that prevent speculative leakage

A concurrency paradigm that guarantees system-wide progress without using mutual exclusion locks. Algorithms rely on atomic CPU instructions like Compare-and-Swap (CAS) or Load-Linked/Store-Conditional (LL/SC) to manipulate shared data.

• Wait-Free: Every thread completes in a bounded number of steps—the strongest guarantee
• Lock-Free: At least one thread makes progress in any finite time period
• ABA Problem: A thread reads value A, another changes A→B→A, and the first thread's CAS succeeds incorrectly—solved with tagged pointers
• Memory Ordering: Out-of-order execution requires explicit memory fences (std::memory_order) to enforce visibility guarantees across cores

A stop-the-world event in managed runtimes (Java, Go, .NET) where all application threads are suspended while the garbage collector reclaims dead objects. These pauses introduce unpredictable latency spikes that are catastrophic for real-time AI inference serving.

• Mark-Sweep-Compact: Traditional algorithm causing pauses proportional to heap size
• Generational GC: Collects short-lived objects frequently in a small nursery, reducing pause frequency
• Pauseless/ZGC/Shenandoah: Modern collectors performing most work concurrently, targeting sub-millisecond pauses
• Relevance to OoO: GC pauses disrupt the instruction pipeline, flushing the reorder buffer and negating the throughput gains of out-of-order execution