Roofline Model

The fundamental x-axis metric, measured in FLOPs/Byte, representing the ratio of total floating-point operations to total data movement (bytes transferred to/from DRAM).

• Low Intensity (< 10): Memory-bound kernels like element-wise additions or ReLU.
• High Intensity (> 100): Compute-bound kernels like large matrix multiplications or convolutions.
• This metric is derived by dividing the total arithmetic operations in a layer by the number of bytes accessed from main memory.

The flat horizontal ceiling representing the processor's maximum FLOPS (Floating-point Operations Per Second). This is a hard limit determined by the clock speed and the number of parallel arithmetic units (e.g., GPU CUDA cores or NPU MAC arrays).

• A workload hitting this roof is compute-bound.
• Optimization strategy: Reduce operations via pruning, quantization, or algorithmic simplification.
• The roof is flat because compute throughput is independent of data reuse; it only cares about raw math.

The sloped diagonal line rising from the origin, representing the maximum rate at which data can be transferred from DRAM to the compute units, measured in GB/s.

• The slope of this line is exactly the peak memory bandwidth.
• A workload sitting on this slope is memory-bound.
• Optimization strategy: Increase data reuse via kernel fusion, tiling, or increasing batch size to amortize memory loads.

Sub-roofs that sit below the absolute peak, representing realistic bottlenecks imposed by specific hardware features or instruction sets.

• SIMD Ceiling: Limited by vectorized instruction width (e.g., AVX-512 vs. NEON).
• Tensor Core Ceiling: The maximum throughput when using specialized matrix engines, often requiring specific data layouts (e.g., N:M sparsity).
• Locality Ceiling: A bandwidth sub-roof for specific cache levels (L1, L2), showing the penalty for missing the cache.

The critical intersection where the memory-bound slope meets the compute-bound flat roof. This point defines the minimum operational intensity required to achieve peak performance on the given hardware.

• If a kernel's intensity is to the left of the ridge point, it is memory-bound and cannot run faster than the bandwidth allows.
• If it is to the right, it is compute-bound and is wasting memory bandwidth.
• This is the primary diagnostic marker for hardware-software co-design.

The model is not generic; a unique roofline must be generated for each target platform (e.g., a specific Arm Cortex-M4 MCU vs. an NVIDIA Jetson Orin).

• Memory Roofs are measured using STREAM benchmarks.
• Compute Roofs are measured using peak theoretical FLOPS or empirical GEMM benchmarks.
• For edge AI, the model often includes energy rooflines (TOPS/Watt) to identify the most efficient operating point for battery-powered devices.

The ratio of floating-point operations to bytes of data movement, measured in FLOPs/Byte. This is the x-axis of the Roofline Model and determines whether a kernel is compute-bound or memory-bound. High operational intensity means more compute per data element, favoring arithmetic throughput. Low operational intensity indicates a memory-bound workload where performance is limited by bandwidth rather than peak FLOPs.

Often used interchangeably with operational intensity, this metric specifically quantifies the number of arithmetic operations performed per byte of data transferred from off-chip memory. It is critical for determining the roofline ridge point—the threshold where a workload transitions from memory-bound to compute-bound. Maximizing arithmetic intensity through kernel fusion and data reuse is a primary optimization strategy for edge accelerators.

A workload whose performance is constrained by memory bandwidth rather than compute capacity. On the Roofline Model, these kernels lie to the left of the ridge point. Optimization strategies include:

• Kernel fusion to reduce intermediate data movement
• Loop tiling to improve cache locality
• Weight pruning to reduce memory footprint
• Quantization to decrease bytes per element

A workload whose performance is limited by the peak floating-point throughput of the processor. These kernels sit to the right of the ridge point on the Roofline Model. Optimization focuses on:

• SIMD vectorization to maximize instruction-level parallelism
• Winograd transformations to reduce multiply-accumulate operations
• Operator fusion to minimize kernel launch overhead
• Mixed-precision training to leverage faster lower-precision arithmetic

The maximum theoretical rate at which data can be transferred between the processor and off-chip DRAM, measured in GB/s. This forms the sloped portion of the Roofline Model. For edge devices like the Cortex-M7, bandwidth may be as low as a few GB/s, making it the dominant bottleneck for signal processing workloads that operate on large IQ sample buffers with low operational intensity.

The operational intensity threshold where the memory bandwidth ceiling and compute ceiling intersect on the Roofline Model. It is calculated as Peak FLOPs ÷ Peak Bandwidth. Workloads below this point are memory-bound; workloads above are compute-bound. Understanding the ridge point for a target accelerator like an NPU or DSP is essential for selecting appropriate model architectures and compression techniques.