Execution Trace

The sequential record of every machine instruction executed by the NPU's cores. This is the most fundamental component, showing the exact flow of control.

• Opcode and Operands: Captures the operation type (e.g., FMA, LOAD, STORE) and its associated data sources/destinations.
• Program Counter (PC): Tracks the memory address of each instruction, enabling reconstruction of the execution path.
• Branch Resolution: Logs the outcome of conditional branches, revealing control flow divergence and potential thread divergence inefficiencies.

This stream is critical for verifying compiler output and identifying unexpected instruction sequences.

A detailed record of all data movement operations, including addresses, sizes, and latencies. This is paramount for diagnosing memory-bound kernels.

• Access Pattern: Shows whether loads/stores are contiguous (enabling memory coalescing) or scattered.
• Hierarchy Level: Indicates whether an access hit in L1/L2 cache or required a fetch from global memory, directly impacting the cache hit rate.
• Bank Conflicts: Flags simultaneous accesses to the same memory bank within a shared memory region, which serialize execution.

Analyzing this log reveals opportunities to optimize data layout, tiling, and prefetching.

A hierarchical record of subroutine invocations, including entry/exit timestamps and parameters. This contextualizes low-level events within the program's logical structure.

• Call Graph: Maps the relationships between functions, showing which high-level operations lead to specific kernel launches.
• Nested Timing: Allows for attribution of execution time to specific functions, facilitating hotspot identification.
• Kernel Launch Parameters: Records the configuration (grid size, block size) for each dispatched kernel, which is the target for auto-tuning.

This stack is essential for creating flame graphs and understanding performance at the application level.

Metrics collected from specialized processor registers that count low-level architectural events. These provide quantitative insight into hardware resource utilization.

• Compute Metrics: Floating-point operations (FLOPs), integer operations, and compute throughput (e.g., FLOPS achieved).
• Memory Metrics: Cache misses at each level, memory bandwidth utilization, and DRAM transaction counts.
• Execution Metrics: Pipeline stall cycles, warp/wavefront issue efficiency, and occupancy levels.

Integrating these counters with the instruction stream enables precise bottleneck analysis, distinguishing between compute-bound and memory-bound phases.

A temporally ordered log of events with nanosecond-resolution timestamps. This reveals parallelism, dependencies, and idle periods.

• Kernel Execution Windows: Visualizes when each kernel starts and stops, showing potential for concurrent kernel execution.
• Memory Transfer Phases: Shows overlapping of computation with host-to-device (H2D) and device-to-host (D2H) data transfers under asynchronous execution.
• Synchronization Points: Records barriers, locks, and atomic operations that can cause threads to wait, indicating resource contention or serialization.

This timeline is crucial for optimizing workload scheduling and achieving full NPU utilization.

Ancillary data that provides essential context for interpreting the raw trace, ensuring reproducibility and accurate analysis.

• Hardware Identification: NPU model, driver version, and core clock frequency at capture time.
• Software Environment: Compiler version, framework (e.g., TensorFlow, PyTorch), and kernel binary hash.
• Input Tensor Characteristics: Shapes, data types (e.g., FP16, INT8), and memory layouts of the workload's inputs.
• Trace Capture Parameters: Sampling rate (for sampling profiler traces) or instrumentation mode (for instrumentation profiler traces).

This context allows performance engineers to correlate behavior across runs and different hardware platforms.

Execution traces are the primary tool for distinguishing between compute-bound and memory-bound kernels. By analyzing the sequence of instructions and memory accesses, engineers can pinpoint the exact cause of pipeline stalls. For example, a trace showing frequent, long-latency accesses to global memory followed by idle compute units clearly indicates a memory bandwidth bottleneck. This analysis directly informs optimization strategies like memory coalescing or increasing cache hit rates.

Traces reveal the dynamic execution path of individual threads within a warp or wavefront. This is critical for identifying thread divergence, where conditional branches (e.g., if/else statements) cause threads within the same warp to serialize their execution, drastically reducing parallel efficiency. By visualizing which threads are active at each cycle, engineers can restructure algorithms or data layouts to maintain high occupancy and ensure all threads in a warp follow the same execution path as much as possible.

Auto-tuning systems like a kernel tuner use parameter search across a configuration space (e.g., workgroup size, tile size) to find optimal settings. An execution trace provides the definitive proof of why one configuration outperforms another. Instead of just seeing a final runtime, engineers can examine traces to verify that the chosen configuration successfully increased memory coalescing, reduced pipeline stalls, or improved cache hit rates. This moves optimization from a black-box search to an explainable, engineering-driven process.

The most detailed use case involves analyzing the exact pattern of loads and stores. Traces can show:

• Non-coalesced memory accesses that fracture a single wide transaction into many small, inefficient ones.
• Bank conflicts in shared memory that serialize access.
• Inefficient reuse patterns that cause unnecessary data movement across the memory hierarchy. By studying these patterns, engineers can apply transformations like loop tiling (tile size selection) or data layout restructuring to create sequential, aligned access patterns that maximize memory bandwidth utilization.

For complex workloads using asynchronous execution or concurrent kernels, traces provide a timeline view of kernel launches, data transfers, and synchronization events. This allows engineers to:

• Identify gaps of idle hardware between kernels.
• Detect resource contention between concurrently running kernels.
• Verify that computation is effectively overlapped with data transfer (hiding latency).
• Optimize the scheduling order and dependencies to maximize NPU utilization and overall throughput.

Execution traces serve as ground-truth data for building and calibrating analytical performance models. By correlating observed execution times and stall cycles in the trace with kernel parameters (e.g., input size, workgroup size) and hardware events, engineers can create models that predict performance for new configurations. This is a key technique in advanced auto-tuning methods like Bayesian Optimization, where an accurate model guides the search towards promising regions of the configuration space without exhaustive profiling.

A hardware register that counts low-level events within a processor, such as cache misses, floating-point operations (FLOPs), or branch mispredictions. These counters provide the raw, quantitative data that feeds into the analysis of an execution trace.

• Key Metrics: Instructions per cycle (IPC), L1/L2 cache hit rates, memory read/write throughput.
• Use Case: Correlating high-level trace events (e.g., a function call taking too long) with microarchitectural causes (e.g., a high rate of last-level cache misses).

A software tool that measures the execution time, resource utilization, and associated hardware performance counters for a computational kernel running on an NPU. It provides a focused, timed view of a kernel's execution, which is a critical subset of a full system trace.

• Outputs: Timeline of kernel launches, memory copies, and GPU/NPU activity.
• Tools: NVIDIA Nsight Systems, AMD ROCprofiler, Intel VTune Profiler for GPUs/XPUs. NPU vendors provide analogous SDK tools.

The process of using profiling data—including execution traces—to locate the specific functions, loops, or instructions within a program that consume the most execution time or resources. This is the primary diagnostic goal of trace analysis.

• Method: Aggregate time spent per call stack or code region from the trace.
• Visualization: Often presented using Flame Graphs, where the width of horizontal bars represents the proportional time spent in each function and its descendants.

The systematic process of identifying the primary limiting factor that restricts overall performance. An execution trace is the essential dataset for this analysis, revealing whether a workload is compute-bound, memory-bound, or limited by synchronization or control flow.

• Compute-Bound: ALUs are saturated; trace shows high occupancy but low memory throughput.
• Memory-Bound: Compute units are often idle; trace shows high memory transaction counts and potential stalls.
• Latency-Bound: Execution is stalled waiting for dependencies (e.g., data from main memory).

The automated process of searching a configuration space (e.g., tile size, unroll factor, workgroup size) to find the parameters that yield optimal performance for a given kernel on specific hardware. Execution traces and performance counters are used to evaluate each candidate configuration.

• Search Methods: Grid search, random search, or more advanced techniques like Bayesian Optimization.
• Goal: Automatically derive the optimal parameters that a human engineer would seek through manual trace analysis and experimentation.

An analytical or machine-learned model that predicts the execution time or resource usage of a computational kernel based on its parameters, hardware characteristics, and input data. Execution traces from prior runs are often used as training data to build or validate these models.

• Purpose: To guide auto-tuning searches efficiently, avoiding the need to physically execute every possible configuration.
• Types: Roofline models (analytical), linear regression models, or neural networks trained on trace/counter data.