INT8 Inference

Converts 32-bit floating point weights and activations to 8-bit integers through a quantization mapping. The fundamental equation is q = round(x / scale) + zero_point, where scale is a floating-point factor and zero_point is an integer offset that maps real zero to the quantized domain. This reduces model size by 4x and enables integer-only arithmetic units on FPGAs and embedded processors to execute matrix multiplications directly.

A training methodology that simulates quantization effects during the forward pass while maintaining full-precision weights for gradient updates. The model learns to compensate for quantization noise by inserting fake quantization nodes that round and clip values at every layer boundary. This typically yields 1-3% higher accuracy than post-training quantization for complex modulation classification tasks where subtle signal features must be preserved.

Per-tensor quantization uses a single scale and zero-point for an entire weight tensor, while per-channel quantization assigns unique parameters to each output channel. Per-channel is critical for modulation classifiers because weight distributions vary significantly across convolutional filters processing I and Q components. This granularity preserves dynamic range in sensitive layers at the cost of slightly more calibration overhead.

INT8 inference replaces floating-point multiply-accumulate operations with integer matrix multiplication using the equation output = (input_int8 * weight_int8) * scale_accumulation + bias. Modern FPGA DSP slices and ARM NEON SIMD instructions can execute multiple 8-bit operations per cycle. A single Xilinx DSP48E2 slice can perform two INT8 multiply-accumulates simultaneously, enabling massive parallelism on resource-constrained edge devices.

The quantization scale is determined through calibration—running representative IQ sample batches through the model to observe activation value ranges. Common methods include:

• MinMax: Maps the full observed range linearly
• MovingAverageMinMax: Smooths range estimates across batches
• Histogram (entropy): Minimizes KL divergence between original and quantized distributions For RF signals with high peak-to-average power ratios, histogram calibration prevents outlier samples from compressing the useful dynamic range.

INT8 inference maps efficiently to diverse edge hardware:

• FPGA: Xilinx DPU and FINN frameworks synthesize custom INT8 dataflow architectures
• ARM Cortex-M: CMSIS-NN library provides hand-optimized INT8 convolution kernels
• Edge TPU: Google's ASIC executes INT8 operations natively at 4 TOPS
• NVIDIA Jetson: TensorRT optimizes INT8 inference with layer fusion and kernel auto-tuning Each target requires specific quantization schemas and graph optimizations for peak throughput.

The execution of a compiled INT8 neural network directly on a processor without an operating system. This eliminates OS scheduling jitter and context-switching overhead.

• The model is compiled to native machine code for the target architecture.
• Achieves the absolute lowest possible inference latency budget.
• Common in high-stakes electronic warfare systems requiring deterministic latency.