Low-Precision Arithmetic

Integer quantization converts 32-bit floating-point (FP32) weights and activations to 8-bit integers (INT8), reducing the model's memory footprint by ~75% and enabling faster integer-only arithmetic on hardware lacking floating-point units.

• Mechanism: A linear mapping scales and shifts the floating-point range into the 8-bit integer range [-128, 127].
• Impact: Enables efficient inference on microcontrollers but introduces quantization error that can degrade accuracy if not managed.
• Example: A 1MB FP32 model becomes ~250KB after INT8 quantization.

Reduced-precision floating-point formats, like 16-bit (FP16) and Brain Floating Point (BF16), maintain a dynamic range suitable for training while using half the bits of FP32.

• FP16: Uses 1 sign, 5 exponent, and 10 mantissa bits. Prone to underflow/overflow during training.
• BF16: Uses 1 sign, 8 exponent, and 7 mantissa bits. Matches FP32's exponent range, making it more stable for training than FP16.
• Use Case: Mixed-precision training uses FP16/BF16 for most operations and FP32 for master weights and critical reductions to maintain stability.

Quantization-Aware Training (QAT) is a process where the model is trained with simulated quantization noise, allowing it to learn parameters robust to the precision loss incurred during deployment.

• Process: During forward passes, weights and activations are quantized and de-quantized, injecting simulated error. The backward pass uses the Straight-Through Estimator.
• Advantage: Produces models that retain significantly higher accuracy when converted to low-precision formats compared to Post-Training Quantization (PTQ).
• Critical for: Deploying complex models (e.g., vision transformers) on highly constrained TinyML devices.

Modern neural processing units (NPUs) and microcontroller accelerators contain specialized digital signal processors and multiply-accumulate (MAC) units optimized for specific low-precision formats.

• Common Formats: INT8, INT16, and FP16 are natively supported by most edge AI accelerators (e.g., Arm Ethos-U55, Cadence Tensilica VP6).
• Compiler Role: Frameworks like TensorFlow Lite for Microcontrollers and Apache TVM compile models to execute these low-precision operations efficiently on the target hardware.
• Performance Gain: Using the hardware's native precision (e.g., 8-bit MAC) can yield order-of-magnitude improvements in operations per joule compared to emulated floating-point.

Quantization-Aware Training (QAT) is a model compression technique where a neural network is trained with simulated low-precision (e.g., 8-bit integer) arithmetic. This process allows the model to learn to compensate for the precision loss and quantization error during training, resulting in higher accuracy when deployed in low-precision formats compared to simpler post-training quantization.

• Key Mechanism: Simulates quantization by inserting fake quantization nodes into the model's computational graph during the forward pass.
• Benefit: Produces models that are robust to the numerical distortions of integer arithmetic, crucial for maintaining accuracy in TinyML deployments.
• Trade-off: Requires a full or fine-tuning training cycle, which is more computationally expensive than post-training methods.

Post-Training Quantization (PTQ) is a model compression technique that converts a pre-trained model's weights and activations from high-precision floating-point (e.g., 32-bit) to lower-precision formats (e.g., 8-bit integers) after training is complete. It is a faster, deployment-oriented process that requires only a small calibration dataset.

• Process: Involves analyzing the range of activations (calibration) to determine optimal scaling factors (quantization parameters).
• Primary Use: Dramatically reduces model size and accelerates integer-only inference on microcontrollers and edge TPUs.
• Limitation: Can lead to higher accuracy degradation than QAT, especially for models with narrow weight distributions or sensitive activation functions.

Integer-Only Inference is the execution of a fully quantized neural network using exclusively integer arithmetic operations, eliminating the need for floating-point hardware units. This is the ultimate goal of model quantization for deployment on ultra-low-power microcontrollers (MCUs).

• Requirement: The entire model pipeline, including any necessary rescaling operations, must be implemented with integer math.
• Hardware Benefit: Enables efficient execution on MCUs that lack Floating-Point Units (FPUs), reducing power consumption and silicon area.
• Software Stack: Relies on specialized kernels in frameworks like TensorFlow Lite for Microcontrollers and CMSIS-NN for Arm Cortex-M processors.

BFloat16 is a 16-bit floating-point format designed by Google for machine learning workloads. It preserves the dynamic range of a standard 32-bit float (FP32) by using the same 8-bit exponent, but truncates the mantissa to 7 bits. This makes it highly effective for training and inference.

• Design Philosophy: Dynamic range over precision. Excellent for representing gradients and weights where magnitude is more critical than fine-grained precision.
• Hardware Support: Natively supported by modern AI accelerators like Google TPUs, NVIDIA GPUs (Ampere+), and Intel CPUs (AMX).
• Use Case: Increasingly used for federated edge learning where 16-bit precision offers a favorable balance between communication cost, compute efficiency, and model accuracy compared to FP32.