Performance Counter

These are the foundational timing and throughput metrics.

• Clock Cycles: The total number of processor clock ticks consumed. This is the raw measure of execution time, directly impacted by pipeline stalls and branch mispredictions.
• Instructions Retired: The count of instructions successfully completed. The ratio of Clock Cycles per Instruction (CPI) is a key efficiency metric; a higher CPI indicates more stalls and lower utilization.
• Example: A model layer with a high CPI may be suffering from frequent data cache misses, stalling the pipeline while waiting for memory.

Memory access patterns dominate performance on data-intensive ML kernels. Counters track activity at each cache level.

• L1/L2 Cache Misses: Occur when requested data is not found in the small, fast caches, forcing a fetch from slower main memory (SRAM/Flash). This is a primary cause of pipeline stalls.
• Cache Accesses & Hits: The total number of read/write requests and the subset served by the cache. The cache hit rate is critical for performance.
• Use Case: Profiling a convolutional layer might reveal a high L1 data cache miss rate, indicating inefficient memory access patterns in the loop structure.

Modern MCUs use branch prediction to keep pipelines full. Mispredictions cause costly flushes.

• Branch Instructions: Total number of conditional jumps (if/else, loops).
• Branch Mispredictions: Counts when the processor guessed the wrong execution path. Each mispredict can waste 5-10 clock cycles flushing the pipeline.
• Impact: Control-heavy pre/post-processing code or models with data-dependent branches (e.g., certain activation functions) can suffer significantly from mispredictions, increasing non-deterministic latency.

Beyond caches, counters monitor interaction with main memory and the memory controller.

• DRAM/SRAM Accesses: Counts reads/writes to main memory. High access counts correlate with high energy consumption.
• Memory Bus Stalls: Cycles where the processor is ready but waiting for the memory bus or controller.
• Translation Lookaside Buffer (TLB) Misses: Occur when the virtual-to-physical address translation isn't cached, requiring a slow page table walk. Critical for systems with Memory Management Units (MMUs).

These counters provide insight into the inner workings of the CPU's execution engine.

• Micro-ops (μops): Many complex instructions are broken down into simpler μops. Counting them gives a finer-grained view of workload.
• Pipeline Slots: Tracks utilization of the pipeline's execution ports (e.g., integer, floating-point, load/store units). Identifies which functional units are bottlenecks.
• Resource Stalls: Counts cycles where μops could not be dispatched due to lack of a required resource (e.g., all load buffers are full).

Some advanced performance monitoring units (PMUs) correlate architectural events with power draw.

• Active Cycles: Counts cycles where the core or specific units are powered and active.
• Voltage/Frequency Transitions: Can track events related to Dynamic Voltage and Frequency Scaling (DVFS), which impacts performance and energy.
• Integration: By correlating cache miss counts with current draw measurements, engineers can attribute energy spikes directly to inefficient memory access patterns.

Inference latency is the total time delay from input presentation to prediction output for a single model inference. It is a critical real-time system metric, often measured in milliseconds or microseconds. For microcontroller deployments, latency is broken down into:

• Data Preprocessing Time: Time to format sensor data.
• Model Execution Time: Core forward pass, measured by performance counters.
• Post-processing Time: Time to interpret the model's output tensor. High latency can render a system unusable for applications like anomaly detection or keyword spotting.

Peak memory usage is the maximum RAM/SRAM consumed during inference, including model weights, activation buffers, and runtime overhead. This is a hard constraint for microcontrollers, which may have only tens to hundreds of kilobytes of SRAM. Exceeding available memory causes crashes. Profiling tools track:

• Static Memory: Allocated for weights and constants.
• Dynamic Memory: Temporary buffers for layer activations.
• Stack/Heap Usage: Runtime and framework overhead. Optimization techniques like activation buffering and in-place operations directly target reducing this peak.

Energy per inference is the total electrical energy consumed for a single forward pass, typically measured in microjoules (µJ). It is the product of average power (watts) and inference time (seconds). This is the ultimate efficiency metric for battery-powered devices. Measurement requires:

• Precise Power Monitoring: Using a digital multimeter or specialized energy profiler.
• Steady-State Measurement: Isolating the inference task from system idle states.
• Voltage Regulation Awareness: Accounting for changes in supply voltage. Performance counters for CPU cycles and cache misses provide the computational correlates of energy consumption.

A profiling tool is software that instruments a system to measure runtime behavior. For TinyML, these tools interface with hardware performance counters and other sensors to collect data. Key capabilities include:

• Event Tracing: Logging specific microarchitectural events (cache misses, instructions retired).
• Timeline Visualization: Displaying function and layer execution over time.
• Resource Heatmaps: Showing memory or energy use across the model graph. Examples include Arm Streamline (for Cortex-M), Segger SystemView, and framework-specific tools like TensorFlow Lite Micro's debug logs. They are essential for moving from macro metrics to root-cause analysis.

Worst-Case Execution Time is the maximum possible time an inference task could take under all permissible operating conditions. It is a safety-critical metric for real-time systems. Determining WCET involves:

• Static Analysis: Analyzing the control flow graph of the model and runtime.
• Measurement-Based Profiling: Running the model with diverse inputs under varying conditions (temperature, voltage).
• Accounting for Variability: Considering cache thrashing, memory bus contention, and interrupt handling. Performance counter data on cycle counts and cache behavior under stress tests are vital inputs for WCET analysis.