Resource Contention

High-speed, low-latency memory (L1, L2 cache) shared among processor cores. Contention arises when multiple threads' working sets exceed the cache capacity, leading to cache thrashing and a low cache hit rate.

• Impact: Dramatic increase in latency as data is fetched from slower main memory.
• Optimization: Adjust tile size selection in kernel code to match cache line size and capacity, promoting temporal and spatial locality.

A small, fast, software-managed memory block on an NPU/GPU multiprocessor, shared by all threads in a workgroup. Contention occurs when threads within a block compete for limited banks, causing bank conflicts that serialize access.

• Consequence: Reduced effective bandwidth of this critical resource.
• Solution: Pad data structures in memory to avoid bank conflicts and carefully partition data access patterns among threads.

The arithmetic logic units that perform computations. Contention here manifests as compute-bound performance, where the pipeline is saturated with arithmetic operations. While desirable, inefficiency arises from thread divergence, which causes serialized execution within a warp/wavefront.

• Detection: High occupancy but suboptimal compute throughput.
• Tuning: Use loop unrolling and adjust the vectorization factor to maximize ALU utilization and minimize divergent control flow.

The hardware pathways (e.g., NoC - Network-on-Chip, crossbars) that route data between cores, caches, and memory partitions. Contention on these fabrics creates latency and reduces effective bandwidth for all connected units.

• Profile Via: Execution trace analysis showing irregular memory access latencies.
• Design Consideration: A key differentiator in NPU architecture; software must be aware of non-uniform memory access (NUMA) characteristics.

Resources like barriers, atomic operations, and locks used for thread coordination. High contention on these primitives forces threads to wait, causing pipeline stalls and reducing parallelism.

• Common in: Reduction operations, parallel data structure updates.
• Best Practice: Minimize global synchronization, use hierarchical (tree-based) reduction patterns, and leverage warp/wavefront-level primitives where possible.

A performance state where a computational kernel's execution time is limited by the speed or bandwidth of the memory subsystem, not the arithmetic units. The processor's compute cores are frequently idle, stalled while waiting for data to be loaded from or stored to memory. This is a primary symptom of resource contention for memory channels or cache space.

• Key Indicator: Low compute unit utilization despite high theoretical FLOPS.
• Example: A matrix multiplication kernel where data access patterns prevent efficient use of cache, causing constant fetches from high-latency global memory.

A performance state where a kernel's execution time is limited by the speed of the arithmetic logic units (ALUs). The processor is saturated with mathematical operations, and memory accesses are relatively infrequent or perfectly overlapped with computation. This is the ideal state for maximizing NPU throughput and indicates minimal resource contention for memory bandwidth.

• Key Indicator: High ALU utilization and memory subsystem activity well below its peak bandwidth.
• Example: A deeply nested loop performing dense floating-point calculations on data already resident in registers or L1 cache.

The systematic process of identifying the primary limiting factor that restricts the overall performance of a system or application. For NPU workloads, this involves using profiling tools to measure metrics like compute throughput, memory bandwidth, and latency to determine if the kernel is compute bound, memory bound, or limited by other factors like instruction throughput or synchronization overhead.

• Goal: Pinpoint the exact shared resource (e.g., L2 cache, memory controller) causing contention.
• Method: Compare achieved performance against the hardware's theoretical peak for each subsystem.

The percentage of memory access requests that are successfully served from a cache (e.g., L1, L2, shared memory) instead of requiring a slower access to main memory. A low cache hit rate is a direct cause of resource contention for DRAM bandwidth and significantly increases latency.

• Impact: A drop from 95% to 80% hit rate can double or triple effective memory access time.
• Optimization: Improving data locality through tiling and memory access pattern adjustments is key to raising the hit rate and reducing contention.

The ratio of actively executing warps (or wavefronts) to the maximum number that can be resident on an NPU's streaming multiprocessor. High occupancy helps hide memory latency by providing more threads that can be switched to when others are stalled. However, excessive occupancy can itself cause resource contention for on-chip resources like registers and shared memory, leading to register spilling or shared memory bank conflicts.

• Trade-off: Auto-tuning tools balance occupancy against per-thread resource usage to find the optimal point that minimizes overall contention.

The automated process of searching a parameter space to find the kernel configuration that yields optimal performance for specific hardware. It directly addresses resource contention by empirically testing different configurations (e.g., workgroup size, tile dimensions, loop unroll factors) to find the balance that minimizes stalls and maximizes utilization of shared resources.

• Method: Uses techniques like grid search, random search, or Bayesian optimization to navigate the configuration space.
• Outcome: Discovers parameters that optimally partition limited resources like shared memory and register files among threads to avoid contention.