Speculative Execution

The branch predictor is the core component that guesses the direction of conditional branches (e.g., if-then-else) before the condition is evaluated. Modern predictors use two-level adaptive algorithms that track branch history patterns:

• Branch Target Buffer (BTB): Caches the destination addresses of previously taken branches
• Pattern History Table: Stores 2-bit saturating counters to track branch tendencies
• Global History Register: Tracks the outcome of the last N branches to identify correlated patterns

Prediction accuracy on modern processors exceeds 97%, enabling deep pipelines to stay filled with useful instructions.

Instructions executed speculatively operate in a transient window—a temporary architectural state that is architecturally invisible. If the prediction is correct, results are retired and become visible. If incorrect, the pipeline is flushed:

• All register and memory changes are discarded
• The program counter is reset to the correct path
• The flush penalty is typically 10-20 clock cycles

Critically, microarchitectural side effects—such as cache state changes—are not rolled back during a flush, creating the fundamental leakage vector exploited by Spectre-class attacks.

Although the architectural state is cleaned during a misprediction flush, the cache hierarchy retains residual traces of speculatively executed loads. This creates a measurable timing side channel:

• A speculatively executed load brings data into L1/L2/L3 cache
• An attacker can probe cache lines using Flush+Reload or Prime+Probe techniques
• Faster access times reveal which lines were cached by the transient execution
• This timing difference—often < 100 cycles—encodes secret data bits

This mechanism is the foundation of Spectre v1 (Bounds Check Bypass) and Meltdown attacks.

To execute loads and stores out of order, the processor must predict whether a load depends on an earlier store. The memory disambiguator speculates that loads are independent of prior in-flight stores:

• If a load address overlaps with a prior store address, a memory ordering violation occurs
• The pipeline is flushed and the load is replayed
• Spectre v4 (Speculative Store Bypass) exploits this by training the disambiguator to falsely predict independence
• This allows a load to speculatively read stale data from a store buffer before the store commits

Mitigations include Speculative Store Bypass Disable (SSBD) and memory ordering fences.

Indirect branches (calls through function pointers) use the Branch Target Buffer (BTB) to predict the target address. Spectre v2 (Branch Target Injection) poisons this prediction:

• The BTB is indexed by the virtual address of the branch instruction
• An attacker trains the BTB to mispredict a victim's indirect branch to a gadget address
• The victim speculatively executes the attacker-chosen gadget, leaking data through cache side channels
• Mitigations include Retpoline (replacing indirect branches with returns) and Indirect Branch Restricted Speculation (IBRS)

This attack can cross privilege boundaries, including user-to-kernel and guest-to-hypervisor transitions.

MDS attacks exploit fill buffer and line fill buffer structures shared between logical cores in simultaneous multithreading (SMT) environments:

• Speculative loads that fault or assist still populate internal processor buffers
• ZombieLoad, RIDL, and Fallout leak data from these buffers across security domains
• Data can include AES keys, passwords, or kernel memory
• Mitigations require microcode updates to flush buffers on context switches and disabling SMT in high-security environments

These vulnerabilities demonstrate that speculation leaks data even when the load never architecturally completes.

A processor paradigm where instructions are dynamically reordered for maximum pipeline efficiency rather than strict program order. While speculative execution predicts branches, out-of-order execution rearranges independent instructions to avoid stalls.

• Enables instruction-level parallelism by executing ready instructions immediately
• Uses a reorder buffer to commit results in original program order
• Combined with speculation, creates complex microarchitectural state that side-channel attacks exploit
• Modern AI inference servers rely heavily on this for throughput

An attack class that extracts secrets by measuring physical side-effects of computation—timing, power draw, electromagnetic emissions, or sound—rather than breaking cryptographic primitives directly.

• Spectre and Meltdown are canonical examples exploiting speculative execution
• Cache-timing attacks measure access latency to infer data locations
• AI model weights and training data are high-value targets
• Defenses include constant-time programming and hardware isolation like Trusted Execution Environments

A hardware optimization that proactively loads data into cache before the processor explicitly requests it, anticipating future memory access patterns. While distinct from speculative execution, prefetch mechanisms share similar cache-state leakage vulnerabilities.

• Hardware prefetchers track access strides and patterns automatically
• Software prefetch instructions give programmers explicit control
• Attackers can trigger prefetch to prime caches and measure timing differences
• Critical for AI inference latency—prefetching model weights reduces Time to First Byte

The algorithm governing which data is removed when a cache fills, directly impacting both performance and the attack surface of speculative execution vulnerabilities.

• LRU (Least Recently Used) is common but predictable by adversaries
• Random replacement policies reduce deterministic cache-timing leaks
• Cache partitioning isolates sensitive AI model regions from untrusted processes
• Eviction patterns can be weaponized in Prime+Probe attacks to infer victim memory access

A secure enclave within the main processor that guarantees code and data confidentiality even against a compromised operating system. TEEs are a primary defense against speculative execution attacks on AI workloads.

• Intel SGX and AMD SEV provide hardware-grade memory encryption
• Enclaves isolate model weights during inference, preventing cache-timing snooping
• Confidential computing extends TEE principles to cloud AI deployments
• Performance overhead from enclave transitions must be balanced against security requirements

A physical attack vector that deliberately induces hardware errors—voltage glitching, clock manipulation, electromagnetic pulses—to bypass security checks or extract secrets from AI accelerators.

• Voltage glitching can skip instruction execution during model loading
• Laser fault injection targets specific transistors on decapped chips
• Combined with speculative execution bugs, faults can amplify information leakage
• Defenses include hardware redundancy, error-correcting codes, and tamper-resistant packaging