Hardware-in-the-Loop (HIL) Testing

This is the actual microcontroller unit (MCU) or embedded processor that will be deployed in the final product. It executes the complete firmware stack, including the TinyML model inference engine, using its native memory, clock speed, and potential hardware accelerators (e.g., NPU, DSP). In HIL testing, this hardware is the system under test (SUT), and its performance metrics—inference latency, peak memory usage, and energy per inference—are measured directly, not estimated.

A high-performance computer that runs a real-time operating system (RTOS) or a real-time kernel. Its primary functions are:

• Plant Model Execution: Runs high-fidelity mathematical models that simulate the physical environment (e.g., motor dynamics, sensor noise, vehicle physics).
• I/O Simulation: Generates precise, time-synchronized electrical signals (analog/digital/PWM) that mimic real sensors, feeding them to the target hardware.
• Deterministic Timing: Guarantees that simulation cycles execute within strict, sub-millisecond time steps, ensuring the deterministic execution required for valid real-world testing.

Specialized electronic boards that act as the electrical bridge between the RTSC and the target MCU. They perform critical signal conversion and conditioning:

• Digital/Analog (D/A) Converters: Translate simulation outputs from the RTSC into voltage/current signals for the MCU's ADC pins.
• Analog/Digital (A/D) Converters: Read output signals (e.g., PWM control signals) from the MCU and digitize them for the simulation.
• Protocol Interfaces: Provide direct connections for communication buses like CAN FD, SPI, I2C, and UART, allowing the simulated environment to interact with the MCU's communication peripherals.

The central orchestration and measurement layer. This software suite runs on a host PC and manages the entire test campaign:

• Scenario Scripting: Automates complex test sequences, including edge cases and fault injections (e.g., sensor dropout).
• Data Logging & Visualization: Captures time-series data from the target MCU (internal states, outputs) and the simulation.
• Performance Profiling: Integrates with the target's debug probes or internal performance counters to collect detailed layer-wise profiling data, measuring WCET, cache misses, and NPU utilization. Tools like Percepio Tracealyzer or vendor-specific profilers are commonly used here.

A hardware debugger (e.g., J-Link, ST-LINK, DAPLink) connected to the MCU's SWD or JTAG port. It is essential for non-intrusive profiling and calibration:

• Flash Programming: Deploys the firmware and TinyML model to the target.
• Real-Time Trace: Streams execution trace data (e.g., ETM, ITM) to the host for detailed timing analysis and statistical profiling.
• Live Variable Monitoring: Allows observation and modification of model parameters or system variables in real-time without stopping execution, crucial for tuning.

A precision instrument placed in series with the power supply to the target MCU. It provides granular energy consumption data, which is critical for TinyML benchmarking:

• Measures Dynamic Power: Captures current spikes during active computation (inference) and idle states.
• Calculates Energy per Inference: By synchronizing with the test automation software, it precisely measures the joules or microjoules consumed for a single inference or a batch.
• Identifies Power Anomalies: Helps detect issues like excessive static power leakage or unexpected high-current states that could lead to thermal throttling in the final product.

HIL testing is essential for verifying that a TinyML model meets its Worst-Case Execution Time (WCET) and deterministic execution requirements under real hardware conditions. It subjects the microcontroller to worst-case sensor inputs and system loads to measure:

• Tail latency (P95, P99) under variable conditions.
• End-to-end latency from sensor input to actuator output.
• Consistency of inference timing to prevent system failures in time-critical applications like motor control or anomaly detection.

This application directly measures the energy per inference and system-level power draw on the target hardware, which is often impractical in pure software simulation. HIL setups integrate precision power meters and thermal sensors to profile:

• Dynamic power vs. static power breakdown during active inference.
• The impact of model compression techniques like quantization on total energy use.
• Thermal throttling behavior and its effect on sustained throughput (FPS).
• Battery life projections under realistic duty cycles.

HIL testing exposes the TinyML system to real-world electrical and sensor imperfections that are difficult to model digitally. This validates the model's resilience against:

• Analog-to-digital converter (ADC) quantization noise and non-linearities.
• Voltage regulator ripple and supply noise on low-power rails.
• Electromagnetic interference from other on-board components.
• Non-ideal sensor data (e.g., from MEMS accelerometers or microphones) that differs from clean training datasets.

HIL verifies the complete software stack, ensuring the model's inference engine correctly interfaces with the microcontroller's real-time operating system (RTOS), drivers, and communication buses. This tests:

• Memory allocation conflicts between the model runtime and other tasks.
• Correct handling of interrupts during inference without data corruption.
• Data flow over SPI/I2C for sensor input and output signaling.
• Integration with wireless stacks (BLE, LoRaWAN) for sending inference results.

A HIL test bench automates the execution of a golden dataset on the physical hardware as part of a continuous integration pipeline. This provides:

• Deterministic accuracy validation after any code or toolchain update.
• Detection of performance regressions caused by compiler changes or library updates.
• Validation that model efficiency metrics (accuracy, latency, memory) hold after deployment.
• A hardware-correlated ground truth for cross-referencing with simulator results.

HIL testing pushes the hardware to its limits to identify bottlenecks and validate resource forecasts. Profiling tools and performance counters on the MCU are used to measure:

• Peak memory usage (SRAM/Flash) during concurrent operation.
• Utilization of available compute units (CPU, NPU, DSP).
• Cache behavior and memory bandwidth saturation to identify if the workload is compute-bound vs. memory-bound.
• Stability under maximum MACC count workloads for extended periods.

The maximum possible time a specific task, such as a model inference, could take to complete under all permissible operating conditions. For HIL testing, analyzing WCET is non-negotiable for validating deterministic execution in safety-critical or real-time embedded systems. It ensures the model and software will always meet timing deadlines, even under worst-case data inputs and system states.

The total time from the initial sensor data acquisition or system trigger to the final actionable output or actuation. HIL testing must measure this holistic metric, which encompasses:

• Sensor readout and signal preprocessing
• Inference latency on the target MCU
• Post-processing and decision logic
• Communication to actuators This is the true measure of system responsiveness, distinct from isolated inference timing.

The maximum amount of RAM (SRAM) consumed during inference, including model weights, activations, and intermediate buffers. HIL testing directly validates this against the target microcontroller's physical memory limits. Profiling tools measure this in real-time on the hardware to catch memory leaks, stack overflows, and inefficient memory management that could cause system failure.

The total electrical energy, measured in microjoules (µJ) or millijoules (mJ), consumed to complete a single forward pass. HIL testing with precise power measurement hardware (e.g., precision ammeters) is the only way to capture this metric accurately, accounting for:

• Dynamic power from compute operations
• Static power from leakage
• Power spikes from memory access This data is essential for battery life estimation and power-aware design.

A system's property of producing identical outputs and exhibiting identical timing behavior for identical inputs across repeated runs. This is a primary goal verified by HIL testing. Non-determinism can arise from:

• Uninitialized memory
• Interrupt-driven concurrency
• Non-fixed-point arithmetic
• Cache effects HIL tests must stress the system to prove deterministic behavior, which is foundational for reliable, debuggable embedded systems.