Asymmetric Quantization

The zero-point is a critical integer parameter in asymmetric quantization that represents the quantized integer value corresponding to the real number zero in the original floating-point range. Its primary functions are:

• Enabling Efficient Padding: It allows zero-valued padding in operations like convolution to be represented exactly in the quantized domain without costly dequantization.
• Handling Asymmetric Ranges: It shifts the quantized range to accurately capture data distributions that are not centered on zero, such as the output of a ReLU activation function (which has a minimum of zero).
• Mathematical Role: In the quantization/dequantization equations, the zero-point ensures the mapping is affine (scale * (integer - zero-point)) rather than just linear, providing the necessary offset.

The scale factor is a positive floating-point number that defines the ratio between a unit step in the quantized integer domain and the corresponding step in the original floating-point domain. It is calculated as: S = (float_max - float_min) / (quant_max - quant_min) For INT8 asymmetric quantization, quant_max is 127 and quant_min is -128.

• Determines Precision: The scale factor directly sets the resolution of the quantization. A larger range of real numbers mapped to the same integer range results in a larger scale and lower precision.
• Per-Tensor vs. Per-Channel: A single scale may be used for an entire tensor (per-tensor), or unique scales can be applied to each output channel of a weight tensor (per-channel), which often yields higher accuracy by providing finer granularity.

This defines the bounded set of integer values available for representing quantized data. For n-bit quantization, the range is typically:

• Signed (e.g., INT8): [-2^(n-1), 2^(n-1) - 1] = [-128, 127]
• Unsigned (e.g., UINT8): [0, 2^n - 1] = [0, 255] The choice of range is hardware-dependent. Asymmetric quantization uses the full range efficiently by mapping the observed minimum (float_min) and maximum (float_max) of the data directly to q_min and q_max, minimizing clipping error compared to symmetric schemes when data is not centered on zero.

Calibration is the data-driven procedure to determine the optimal scale and zero-point for each activations tensor in Post-Training Quantization (PTQ).

• Representative Dataset: A small, unlabeled subset of the training data (typically 100-500 samples) is passed through the pre-trained FP32 model.
• Statistics Collection: For each layer, the minimum and maximum activation values (float_min, float_max) are observed across the calibration dataset. Common methods include:
  • Min/Max: Using the absolute observed min/max.
  • Entropy / KL Divergence: Selecting a range that minimizes the information loss between the FP32 and quantized distributions.
• Parameter Calculation: The collected ranges are then used to compute the final scale and zero-point for deployment.

This is the mathematical framework that governs the conversion between floating-point (FP) and integer (INT) domains. The equations define the relationship:

• Quantization: Q = round(real_value / S) + Z_p
• Dequantization: real_value' ≈ S * (Q - Z_p) Where Q is the quantized integer, S is the scale, and Z_p is the zero-point.
• Integer-Only Arithmetic: A key optimization for microcontrollers is fusing the scale and zero-point operations. During inference, the core computation (e.g., matrix multiplication) is performed entirely with integers. The required scaling is often absorbed into the weights beforehand or handled with fixed-point integer arithmetic, avoiding slow floating-point operations on-device.

Understanding the distinction clarifies why asymmetric is often preferred for activations.

• Typical Hybrid Use: In practice, weights are often quantized symmetrically (simpler math), while activations are quantized asymmetrically to better handle their dynamic, non-negative ranges, providing an optimal balance of accuracy and efficiency.

The most critical application is quantizing layers with ReLU or similar activation functions, which produce strictly non-negative outputs. Symmetric quantization wastes half its integer range on negative values that never occur. Asymmetric quantization shifts the range to start at zero, using the full 8-bit integer range (e.g., 0 to 255 for uint8) to represent the actual activation distribution, minimizing precision loss. This is essential for efficient INT8 inference on vision and sensor models.

Raw data from microcontroller-connected sensors (e.g., accelerometers, microphones, environmental sensors) often has a built-in DC offset or bias. This creates an inherently asymmetric distribution. Applying asymmetric quantization allows the zero-point to align with the sensor's actual zero-signal value, ensuring the integer representation efficiently captures the dynamic range of the signal above and below that baseline without clipping.

When applying post-training quantization (PTQ) to existing floating-point models not trained for quantization, activation ranges are often asymmetric. Asymmetric quantization provides a more accurate mapping during calibration, typically yielding higher accuracy than forcing a symmetric range. This is a pragmatic choice for deploying legacy or third-party models onto edge devices without retraining.

In networks using zero-padding (common in CNNs) or skip connections (e.g., ResNet), the true zero value is semantically meaningful. Asymmetric quantization preserves an exact integer representation for zero via the zero-point. This ensures that padded values and identity mappings in skip connections remain true zeros after quantization, preventing error accumulation and maintaining network stability.

For layers with highly skewed, non-zero-mean output distributions, asymmetric quantization reduces quantization error for a given bit-width. This allows the use of lower precision (e.g., INT8 instead of INT16) to meet strict RAM footprint limits while preserving task accuracy. The computational overhead of the zero-point is a fixed cost outweighed by the memory savings and reduced data movement.

Industry-standard libraries for microcontroller AI, such as CMSIS-NN, provide optimized kernels that support asymmetric (often called "asymm") quantization. Using this scheme ensures compatibility with these highly tuned routines, maximizing performance on Cortex-M cores. Frameworks like TensorFlow Lite Micro (TFLM) default to asymmetric quantization for activations to leverage this hardware support.

Symmetric quantization defines a quantization range that is symmetric around zero. This scheme uses a single scale factor and typically sets the zero-point to zero, simplifying the scaling mathematics. It is most effective for data distributions that are naturally symmetric, such as weight tensors after batch normalization.

• Key Difference: Lacks a separate zero-point to represent the real value zero, making it less optimal for skewed data like ReLU activations, which have a non-negative range.
• Computational Advantage: The zero-point of zero eliminates the need for an additional integer addition during the quantized convolution or matrix multiplication, slightly reducing compute overhead.

The zero-point is a critical integer parameter in asymmetric quantization that represents the quantized equivalent of the real value zero. It bridges the integer and floating-point domains, allowing the scheme to accurately map skewed data ranges.

• Functional Role: Enables efficient padding with zeros in convolutional layers, as the padded value has a direct, meaningful integer representation.
• Calculation: Derived during the calibration phase from the observed minimum and maximum of the tensor's floating-point range. The formula is typically zero_point = round(quant_max - (float_max / scale)).
• Impact: Its inclusion is what defines 'asymmetric' quantization, providing the flexibility to capture the full, often non-zero-centered, dynamic range of activation tensors.

Quantization-aware training is a model training methodology where fake quantization nodes simulate the effects of low-precision arithmetic during the forward pass. This allows the model to learn to compensate for the precision loss inherent to techniques like asymmetric quantization.

• Process: During training, weights and activations are quantized and dequantized on-the-fly using the target bit-width (e.g., INT8) and scheme (e.g., asymmetric). The gradients are calculated through these simulated operations via the straight-through estimator.
• Advantage over PTQ: Typically yields higher accuracy than post-training quantization for aggressive quantization (e.g., INT8 or lower) because the model parameters are optimized for the quantized inference pipeline.
• Use Case: Essential for deploying complex models on microcontrollers where every bit of precision matters for task performance.

Calibration is the data-driven process of determining the optimal quantization parameters—scale and zero-point—for a model's activations. For asymmetric quantization, this involves finding the tensor's minimum and maximum values.

• Procedure: A representative dataset (the calibration set) is passed through the pre-trained floating-point model. Statistics (e.g., min/max, histograms) are collected for each activation layer.
• For Asymmetric Quantization: The collected minimum and maximum values directly define the floating-point range [float_min, float_max], from which the scale and zero-point are calculated.
• Criticality: The choice of calibration data and method (min/max, percentile, entropy) directly impacts the final quantized model's accuracy and robustness.

INT8 inference refers to the execution of a neural network where weights and activations are represented as 8-bit integers. Asymmetric quantization is a primary method for converting models to this format for deployment on microcontrollers.

• Benefits: Provides a 4x reduction in model size compared to FP32 and enables the use of highly efficient integer arithmetic units (ALUs) common in low-power MCUs and Neural Processing Units.
• Performance: On hardware with optimized INT8 pipelines (e.g., using SIMD instructions), inference latency and energy consumption can be dramatically lower than with floating-point.
• Deployment Target: The predominant precision target for production TinyML due to its favorable balance of compression, speed, and accuracy retention.

Dequantization is the reverse operation of quantization, converting quantized integer values back into floating-point numbers. It is a necessary step in the quantization toolchain and during certain hybrid operations.

• Formula: The dequantization operation for asymmetric quantization is defined as float_value = scale * (int_value - zero_point).
• Primary Uses:
  • During quantization-aware training, to convert fake-quantized integers back to floats for gradient computation.
  • In some deployment scenarios, to convert final integer outputs (e.g., classification logits) back to floats for softmax or other post-processing.
• On-Device Consideration: Pure integer-only inference pipelines aim to eliminate dequantization entirely during the main compute graph, performing all operations in the integer domain.