Fixed-Point Arithmetic

The core concept of fixed-point representation is an implicit binary point (or radix point) positioned at a fixed location within an integer's bit sequence. Unlike floating-point, this point does not move. The format is defined as Qm.n, where:

• m: Number of integer bits (including sign bit for signed numbers).
• n: Number of fractional bits.
• Total bits: m + n.

For example, in Q1.15 format (common for 16-bit signed numbers), 1 bit is for the sign/integer and 15 bits are for the fraction. The value 1.0 is represented as the integer 32768 (2^15).

The position of the binary point creates a strict, predictable trade-off between dynamic range and precision.

• Range: The maximum representable value is ±(2^(m-1) - 1) / 2^n for signed formats.
• Precision: The smallest representable difference between numbers is 1 / 2^n (the value of the least significant bit - LSB).

Choosing the Q-format is a critical design decision. Using more fractional bits (n) increases precision for small numbers but reduces the maximum magnitude that can be represented. This requires careful analysis of the data flow in a neural network to prevent overflow (exceeding range) and minimize quantization error.

All arithmetic is performed using standard, fast integer operations (add, subtract, multiply, bit-shift) native to all microcontrollers, bypassing the need for a Floating-Point Unit (FPU).

• Addition/Subtraction: Operands must share the same Q-format. The binary points align automatically, and the result is in the same format.
• Multiplication: Multiplying two numbers in formats Qa.b and Qc.d results in a product with format Q(a+c).(b+d). The raw product has double the bit width. A right shift (or rounding) by b+d bits is typically required to bring the result back to the target accumulator format, a key step in optimized kernels.
• Division: Often avoided or implemented via multiplication by a reciprocal.

Since values are integers representing scaled real numbers, every fixed-point variable has an associated scaling factor S = 2^(-n). Arithmetic operations manipulate these implicit scales.

• The core task of fixed-point software is to track these scaling factors and perform corrective re-scaling (via shifts or multiplications) to maintain correctness.
• For example, multiplying two numbers with scales S1 and S2 yields a result with scale S1*S2. To output a value with the original scale S1, a division by S2 (or multiplication by 1/S2) is needed. On hardware, this is efficiently done with pre-calculated shifts or fixed-point multipliers.

A major implementation challenge is preventing overflow, where a result exceeds the representable range of its bit-width.

• Accumulators: Intermediate results, especially in dot products, are often kept in wider bit-widths (e.g., 32-bit or 40-bit accumulators for 16-bit operands) to prevent overflow before final scaling.
• Saturation Arithmetic: Instead of wrapping around (which causes severe errors), results that overflow are clamped (saturated) to the maximum or minimum representable value. Most microcontroller DSP extensions provide saturating add/subtract instructions.
• Design Requirement: The fixed-point format (Qm.n) must be chosen to accommodate the maximum possible value in any tensor during inference, often determined via profiling or calibration on representative data.

Fixed-point arithmetic enables neural network inference on microcontrollers that lack a Floating-Point Unit (FPU). By representing numbers as integers with an implicit binary point, it leverages the microcontroller's native integer Arithmetic Logic Unit (ALU), which is smaller, faster, and more power-efficient. This is critical for ultra-low-power devices where an FPU would be prohibitively expensive in terms of silicon area and energy consumption. For example, an Arm Cortex-M0+ core can execute fixed-point operations using its standard integer pipeline, avoiding the need for a larger, more power-hungry Cortex-M4F or M7 with an FPU.

Replacing 32-bit floating-point values (FP32) with lower-bit fixed-point integers drastically reduces both model size (Flash footprint) and runtime memory (RAM footprint).

• Weights & Constants: Storing weights as INT8 instead of FP32 yields a 4x reduction in persistent storage.
• Activations & Buffers: Using INT16 or INT8 for intermediate tensors cuts peak RAM usage, often the primary constraint on microcontrollers. This compression allows more complex models to fit within the limited memory of devices like the ESP32 (520KB SRAM) or an nRF52840 (256KB RAM), enabling applications like keyword spotting or anomaly detection that would otherwise be impossible.

Fixed-point operations provide deterministic execution time, a key requirement for real-time embedded systems. Unlike floating-point operations, which can have variable latency due to normalization and rounding, integer arithmetic completes in a known, constant number of clock cycles. This predictability is essential for:

• Hard real-time deadlines in motor control or sensor fusion.
• Accurate power budgeting for battery-operated devices.
• Static scheduling of the neural network's compute graph, where all operations and memory transfers are planned at compile-time for zero runtime overhead.

Fixed-point arithmetic is the runtime execution mechanism for quantized models. The process involves:

1. Calibration: A representative dataset determines the scaling factor and zero-point for each tensor.
2. Conversion: FP32 weights and activation ranges are mapped to integers (e.g., INT8).
3. Inference: All operations (matrix multiplies, convolutions) are performed using integer math. Frameworks like TensorFlow Lite for Microcontrollers (TFLM) and CMSIS-NN provide optimized kernels for these quantized, fixed-point operations on targets like Cortex-M processors.

The core challenge is managing numerical precision and dynamic range. Developers must choose the Q-format (e.g., Q7.8 for 7 integer and 8 fractional bits) for each tensor. Key techniques include:

• Static Analysis: Determining the required integer and fractional bits for weights and activations to prevent overflow and minimize precision loss.
• Fixed-Point Scaling: After each integer operation (e.g., a 32-bit accumulator from an INT8 multiply), the result must be re-scaled and saturated back to the target fixed-point format. Libraries handle this using efficient bit-shift operations instead of costly divisions.
• Mixed Precision: Using higher precision (e.g., INT16) for sensitive layers and lower precision (INT8) for others to balance accuracy and performance.

Fixed-point arithmetic is ubiquitous in production TinyML systems:

• Wake-Word Detection: Models like Google's Hey Google or Amazon's Alexa run on microcontrollers using fixed-point math to continuously analyze audio with minimal power.
• Industrial Predictive Maintenance: Vibration analysis models on STM32 MCUs use fixed-point FFTs and neural networks to detect anomalies in real-time.
• Computer Vision on MCUs: Image classification models for simple gesture recognition are quantized to INT8 and run using fixed-point convolutions.
• Sensor Hub Fusion: Combining data from accelerometers, gyroscopes, and magnetometers for orientation estimation requires efficient, fixed-point Kalman filters or tiny neural networks.

Quantization is the overarching model compression technique that enables fixed-point arithmetic. It reduces the numerical precision of a neural network's weights and activations from 32-bit floating-point (FP32) to lower-bit formats like INT8 or INT4. This directly enables the use of fixed-point math by:

• Decreasing model size (e.g., 4x reduction for INT8).
• Reducing memory bandwidth requirements.
• Accelerating inference on hardware with efficient integer arithmetic units. Fixed-point arithmetic is the practical execution method for quantized models.

Integer quantization is the specific process of constraining all model parameters and activations to integer values. This is the direct prerequisite for pure fixed-point arithmetic. Key characteristics include:

• Eliminates floating-point operations entirely.
• Uses integer-only arithmetic units, which are ubiquitous, smaller, and more power-efficient in microcontrollers.
• Requires scaling factors to map integer values back to their original floating-point ranges during dequantization. It is the most common target for microcontroller deployment due to its hardware compatibility.

Static memory allocation is a critical memory management strategy for deterministic microcontroller inference. All buffers for model weights, activations, and intermediate tensors are pre-allocated at compile-time.

• Eliminates runtime overhead of dynamic allocation (malloc/free).
• Prevents memory fragmentation, ensuring reliable long-term operation.
• Enables precise calculation of RAM footprint, which is essential for resource-constrained devices. This works in tandem with fixed-point arithmetic to create a fully predictable, low-latency inference pipeline.

Kernel optimization is the low-level tuning of fundamental neural network operation implementations for a specific microcontroller's CPU core. Fixed-point arithmetic provides the numerical foundation, but kernels determine execution speed.

• Hand-tuned assembly or C for operations like matrix multiplication, convolution, and pooling.
• Exploits processor-specific features like SIMD (Single Instruction, Multiple Data) instructions to process multiple fixed-point integers in parallel.
• Minimizes pipeline stalls and maximizes register usage. Optimized fixed-point kernels (e.g., in CMSIS-NN) are essential for achieving real-time performance.

Footprint analysis is the measurement of memory consumption, directly impacted by fixed-point quantization.

• Flash Footprint: The non-volatile memory needed to store the model. Quantization to INT8 reduces this by ~75% vs. FP32.
• RAM Footprint: The peak volatile working memory required during inference. Fixed-point activations reduce this size, and techniques like in-place computation (overwriting input buffers with outputs) minimize peak usage. Minimizing both footprints is a primary goal of fixed-point arithmetic, enabling models to fit on devices with as little as 32KB of RAM.