Symmetric Quantization

The core characteristic of symmetric quantization is that the representable range is centered on zero. This is defined by a maximum absolute value (|max|) that is identical for both positive and negative sides of the range (e.g., [-127, 127] for INT8). This symmetry eliminates the need for a separate minimum value parameter, simplifying the quantization math and hardware implementation.

Symmetric quantization uses a single scale factor (S) to map between floating-point and integer values. The conversion formulas are simplified:

• Quantization: q = round(r / S)
• Dequantization: r ≈ q * S

Where r is the real (float) value and q is the quantized integer. The absence of a zero-point (Z) term (or Z=0) reduces computational overhead during inference, as multiplication by S is more efficient than the affine transformation (q - Z) * S used in asymmetric schemes.

This scheme is particularly effective for quantizing neural network weights, which typically follow a symmetric, zero-mean distribution (e.g., from Gaussian initialization). Because the range is equally distributed, it wastes minimal quantization bins on unused value ranges, preserving precision. It is less optimal for activation functions like ReLU, which produce strictly non-negative outputs, leading to a 'wasted' half of the symmetric range.

The mathematical simplicity translates directly to hardware efficiency on microcontrollers (MCUs):

• Reduced Circuitry: Integer arithmetic units do not need to handle a subtraction by a non-zero zero-point for every operation.
• Faster Kernels: Optimized libraries like CMSIS-NN and TensorFlow Lite Micro implement symmetric quantization kernels that leverage this for speed.
• Lower Power: Fewer operations per tensor compute lead to reduced energy consumption, a critical metric for battery-powered edge devices.

Symmetric quantization is applied to standard integer bit widths. The exact integer range depends on whether signed or unsigned integers are used:

• INT8 (signed): Typical range is [-127, 127] (not [-128, 127]) to maintain strict symmetry. The value -128 is often excluded to simplify implementation.
• INT4 (signed): Range [-7, 7].
• UINT8 (unsigned): Not typically used for symmetric quantization, as its range [0, 255] is not zero-centered. Unsigned integers are more common for asymmetric activation quantization.

Determining the scale factor (S) is done during the calibration phase of Post-Training Quantization (PTQ). The process is straightforward:

1. Run a representative dataset through the FP32 model.
2. For each tensor (e.g., a weight matrix or activation layer), collect the maximum absolute value (|max|) observed.
3. Calculate the scale: S = |max| / (2^(b-1) - 1) for signed b-bit quantization. For INT8, this is S = |max| / 127. This method is computationally cheap and does not require retraining, making it a staple for rapid MCU deployment.

Dedicated hardware like the Ethos-U55/65 (Arm), Cadence P6, and Synaptics Katana are designed for quantized inference.

• Fixed-Function Units: Contain integer MAC (Multiply-Accumulate) arrays optimized for INT8/INT16 data.
• Compiler Toolchains: Proprietary compilers (e.g., Arm Vela) take a quantized model and map operations to the accelerator's systolic array, performing static scheduling and memory pooling.
• Power Efficiency: Achieve TOPS/Watt metrics far exceeding general-purpose MCU cores.

Quantization is the foundational model compression technique that reduces the numerical precision of a neural network's weights and activations. This process typically converts parameters from 32-bit floating-point (FP32) to lower-bit integers (e.g., INT8, INT4). The primary goals are to:

• Decrease model size for storage in limited microcontroller flash memory.
• Accelerate inference by leveraging efficient integer arithmetic units.
• Reduce power consumption by simplifying computations. Symmetric and asymmetric quantization are the two principal schemes under this umbrella.

Asymmetric quantization is a scheme where the quantization range is defined by separate minimum and maximum values (min and max). This asymmetry allows it to tightly fit skewed data distributions, such as the output of a ReLU activation function (which has a minimum of zero). Its key characteristics are:

• Uses a non-zero zero-point to represent the real value zero in the quantized domain.
• Requires slightly more complex scaling math during inference, as both a scale factor and a zero-point must be applied.
• Is commonly used for quantizing activation tensors, which often have asymmetric ranges.

Integer quantization constrains all model parameters and activations to integer values, enabling execution on hardware that lacks floating-point units (common in low-cost microcontrollers). This is the practical outcome of applying quantization schemes like symmetric or asymmetric. Its implementation involves:

• Performing all operations (convolutions, matrix multiplications) using integer-only arithmetic.
• Using fixed-point operations to emulate fractional values when necessary.
• Frameworks like TensorFlow Lite for Microcontrollers provide kernels optimized for integer-only inference on Arm Cortex-M CPUs.

Post-training quantization is a workflow for converting a pre-trained, full-precision model into a quantized format without retraining. It is the most common path to deployment for microcontroller inference. The process is:

1. Calibration: A representative dataset is passed through the FP32 model to collect dynamic ranges (min/max) of activations.
2. Parameter Calculation: Scale and zero-point values are calculated for each tensor based on these ranges.
3. Model Conversion: Weights are quantized, and the model graph is modified to include quantization/dequantization nodes. PTQ is fast but may incur a small accuracy drop compared to quantization-aware training.

Quantization-aware training is a technique where quantization error is simulated during the training or fine-tuning process. This allows the model to learn parameters that are robust to the precision loss of subsequent integer deployment. Key aspects include:

• Fake quantization nodes are inserted into the forward pass, rounding values to lower precision but maintaining full-precision gradients.
• The model effectively learns to compensate for quantization noise, typically yielding higher accuracy than post-training quantization for the same bit-width.
• QAT requires a training pipeline and is more computationally expensive than PTQ, but is crucial for pushing the limits of low-bit (e.g., INT4) quantization.

Fixed-point arithmetic is a numerical representation system used to perform fractional math using integer hardware. It represents numbers as integers with an implicit, fixed binary point. This is the computational foundation for running quantized models. Critical concepts are:

• Q-format notation (e.g., Q7.8) defines how many bits are allocated for the integer and fractional parts.
• Operations require careful bit shifting to align radix points during multiplication and addition.
• It provides deterministic, low-latency computation essential for real-time microcontroller inference, avoiding the cost and power of floating-point units.