Bare-Metal Inference

Standard operating systems introduce non-deterministic latency through context switching, interrupt handling, and scheduler preemption. Bare-metal execution removes these sources of jitter entirely, ensuring that the time from IQ sample arrival to classification output is constant and predictable.

• Context switch overhead: Eliminates 1-10 µs of unpredictable delay per preemption event
• Interrupt coalescing: Direct interrupt vectoring to the inference routine without kernel mediation
• Cache pollution prevention: No OS processes evicting model weights from L1/L2 cache

The compiled model binary owns the processor's entire memory map, DMA engines, and hardware accelerators without virtualization overhead. This enables zero-copy data flows from the RF front-end's ADC directly into the neural network's input tensors.

• Physical memory addressing: Weights and activations placed in tightly-coupled memory (TCM) for single-cycle access
• DMA-driven ingestion: IQ samples streamed directly from ADC FIFOs to inference buffers without CPU involvement
• NPU/FPGA co-processor control: Direct register-level access to hardware accelerators without driver stacks

The neural network is compiled and linked directly into the firmware binary, executing from non-volatile memory immediately upon power-up. There is no separate model loading phase, file system, or dynamic linker delay.

• XIP (Execute-in-Place): Model weights accessed directly from flash memory without copying to RAM
• Static tensor allocation: All activation buffers pre-allocated at compile time, eliminating runtime memory fragmentation
• Sub-100ms boot-to-inference: Full classification capability available within milliseconds of power-on reset

Bare-metal deployment relies on ahead-of-time (AOT) compilation that performs whole-program optimization across the model graph and hardware target. The compiler applies aggressive optimizations impossible in just-in-time or interpreter-based runtimes.

• Operator fusion: Multiple neural network layers collapsed into single optimized kernels
• Loop unrolling and vectorization: Explicit SIMD instruction generation for the target ISA (e.g., ARM NEON, AVX-512)
• Static scheduling: Instruction ordering optimized for the specific processor pipeline depth and issue width
• Dead code elimination: Unused activation paths removed entirely from the binary

Classification is initiated directly by hardware interrupts rather than polling loops or OS-level event mechanisms. A burst detector in the FPGA fabric asserts an interrupt line that vectors the processor directly into the inference routine.

• Zero-latency triggering: Interrupt-to-first-instruction latency measured in processor clock cycles
• Priority inversion prevention: No OS scheduler that could delay the inference ISR
• Atomic execution: Critical sections protected by hardware interrupt masking rather than software mutexes

Without an OS, the entire software stack—including the model, preprocessing routines, and minimal runtime—fits within the tightly constrained memory of embedded processors. This enables deployment on microcontrollers and small FPGAs unsuitable for embedded Linux.

• Total binary size: Often under 1 MB including quantized model weights
• RAM usage: Only activation tensors and a single inference buffer, typically under 256 KB
• No heap fragmentation: All memory statically allocated, eliminating garbage collection pauses
• Power efficiency: Lower clock rates and reduced memory access translate to extended battery life for field-deployed sensors

A compression technique that reduces the numerical precision of a neural network's weights and activations from 32-bit floating point to 8-bit integer (INT8). This is a prerequisite for bare-metal deployment, as it drastically reduces model size and enables execution on processors without floating-point units.

• Reduces memory footprint by up to 4x
• Enables integer-only arithmetic on embedded cores
• Requires quantization-aware training to minimize accuracy loss

A Real-Time Operating System provides deterministic task scheduling with guaranteed worst-case execution times. Unlike general-purpose OSes, an RTOS ensures that the inference task receives priority and completes within its latency budget on every cycle.

• Preemptive, priority-based scheduling
• Eliminates unpredictable context-switching delays
• Critical for meeting hard real-time constraints in electronic warfare

A technique that digitizes the radio frequency signal directly at the antenna without analog down-conversion. When paired with bare-metal inference, this eliminates entire stages of analog hardware and their associated latency, pushing the digital boundary as close to the signal source as possible.

• Removes mixers, local oscillators, and IF stages
• Requires high-speed ADCs (multiple GSPS)
• Enables fully software-defined processing chains

A memory management technique where data is transferred between processing stages by passing pointers rather than physically copying the data. In a bare-metal context, this eliminates the DMA overhead and memory bus contention that would otherwise consume precious microseconds.

• Data stays in a single memory region
• Only the pointer address is passed between stages
• Essential for maintaining deterministic latency in streaming pipelines

The architectural practice of moving computationally intensive tasks from a general-purpose CPU to a Field-Programmable Gate Array. For bare-metal inference, the entire neural network can be synthesized directly into FPGA fabric as a custom hardware accelerator, achieving true wire-speed classification.

• Parallel execution of all network layers
• No instruction fetch or decode overhead
• Can implement custom bit-width arithmetic for maximum efficiency

A hard real-time constraint ensuring that the time from signal reception to classification output is constant and predictable. This is the primary motivation for bare-metal inference, as operating systems introduce non-deterministic jitter from interrupts, schedulers, and background processes.

• Every inference completes in exactly the same number of clock cycles
• No variance due to cache misses or OS preemption
• Critical for time-sensitive electronic warfare and tactical SIGINT