Asymmetric Quantization

Asymmetric quantization uses an affine transformation defined by a scale factor (S) and a zero-point (Z). The zero-point is the integer value that corresponds to the real value zero. This mapping is expressed as:

Q = round( R / S ) + Z

where R is the real (float) value and Q is the quantized integer. The zero-point allows the quantized range to precisely cover the tensor's min and max, which is critical for activations with ReLU non-linearities that produce strictly positive outputs.

This scheme excels when the data distribution is not centered around zero. Common examples include:

• Layer activations after a ReLU function, which have a minimum of 0 and a positive maximum.
• Model inputs like image pixels (0-255 range).

By aligning the quantized grid's minimum with the data's actual minimum, asymmetric quantization avoids wasting representational capacity on unused negative values, leading to a lower quantization error for a given bit-width compared to a symmetric scheme applied to the same data.

The non-zero zero-point introduces additional integer arithmetic during inference. A core operation like a quantized matrix multiply becomes:

Y = S_y * ( (X_int - Z_x) • (W_int - Z_w) ) + Z_y

where • denotes integer matrix multiplication. The need to subtract zero-points from input tensors before the core integer multiply-accumulate (MAC) operation adds overhead. This contrasts with symmetric quantization, where the zero-point is zero, simplifying the operation to Y = S_y * (X_int • W_int) + Z_y. This overhead must be weighed against the accuracy benefit.

Determining the correct min (R_min) and max (R_max) values for the tensor is a critical calibration step. Common methods include:

• Min/Max: Use the absolute observed min and max from calibration data.
• Entropy Minimization (e.g., KL Divergence): Select a range that minimizes the information loss between the float and quantized distributions.
• Percentile / Moving Average: Use a percentile (e.g., 99.9%) to exclude outliers and prevent extreme values from distorting the scale. Poor calibration can lead to excessive clipping (saturation) or wasted precision.

While the math is well-defined, hardware support varies. Many Neural Processing Units (NPUs) and Digital Signal Processors (DSPs) have optimized pipelines for symmetric (zero-point = 0) operations. Implementing asymmetric quantization efficiently requires:

• Hardware support for the zero-point subtraction before the core MAC unit.
• Efficient handling of the affine transformation in the accumulator. Frameworks like TensorFlow Lite and ONNX Runtime provide kernels that implement these operations, abstracting the hardware complexity for common backends (ARM NEON, x86 AVX2).

The efficiency of asymmetric quantization stems from its mapping to efficient integer arithmetic operations in hardware.

• Fundamental Operation: The core computation, Y = X * W, becomes Y_int32 = (X_int8 - zp_x) * (W_int8 - zp_w). The zero-point subtractions are often pre-computed or fused, leaving pure integer Multiply-Accumulate (IMAC) operations.
• Modern CPU SIMD: Instruction sets like ARM NEON and x86 AVX2/AVX-512 include instructions for vectorized 8-bit integer multiplication and addition, which compilers target for quantized kernels.
• GPU Tensor Cores: NVIDIA's Tensor Cores (from Ampere architecture onward) support INT8 precision, allowing massively parallel computation of the quantized integer matrices. The zero-point handling is managed in the kernel's pre-processing logic.

Symmetric quantization constrains the quantized range to be symmetric around zero, which simplifies computation by setting the zero-point parameter to 0. This scheme is defined by a single scale factor. It is most effective when the distribution of values to be quantized is roughly symmetric. For example, weights after batch normalization often have a symmetric distribution, making this scheme efficient. Its primary advantage is computational simplicity, as it eliminates the need for zero-point addition during integer matrix multiplication, a common operation in neural networks.

The scale and zero-point are the fundamental parameters of affine quantization. They define the linear mapping between floating-point and integer domains.

• Scale (S): A floating-point number that determines the step size between quantized levels. Calculated as (float_max - float_min) / (quant_max - quant_min).
• Zero-Point (Z): An integer value that aligns the zero of the integer range with the zero of the float range. It is the quantized representation of the floating-point value 0.0. The dequantization formula is: float_value = scale * (int_value - zero_point). In asymmetric quantization, the zero-point is typically non-zero to accommodate asymmetric data ranges.

Post-Training Quantization (PTQ) is the process of converting a pre-trained, full-precision model (e.g., FP32) to a lower precision format (e.g., INT8) without requiring retraining. Asymmetric quantization is commonly applied as a PTQ technique. The process involves:

1. Calibration: Running a representative dataset through the model to collect statistics (min/max values) for each tensor.
2. Parameter Calculation: Deriving scale and zero-point for each tensor based on the collected ranges.
3. Conversion & Folding: Transforming the model graph, often folding batch normalization layers into weights. PTQ is fast and requires no labeled data, but may incur higher accuracy loss compared to Quantization-Aware Training (QAT).

Calibration is the critical data analysis phase in PTQ that determines the optimal quantization parameters. For asymmetric quantization, the goal is to find the actual minimum and maximum values of each tensor (weights or activations) over a representative dataset. Common calibration methods include:

• Min-Max: Directly uses the observed min/max values. Simple but can be skewed by outliers.
• Entropy / KL Divergence: Adjusts the threshold to minimize the information loss between the original and quantized distributions, often producing a tighter, more robust range. The chosen min/max values directly define the scale and zero-point, making calibration the primary lever for controlling quantization error in PTQ.

Per-channel quantization is a granularity scheme where a unique scale and zero-point are calculated for each output channel of a weight tensor (typically in convolutional or linear layers). This contrasts with per-tensor quantization, which uses one set of parameters for the entire tensor. Per-channel is particularly beneficial for weight quantization because weight distributions can vary significantly across channels. By allowing each channel's unique range to be precisely captured, it dramatically reduces quantization error compared to per-tensor methods. Most modern inference frameworks (like TensorFlow Lite and ONNX Runtime) support per-channel weight quantization for convolutional and linear layers.

TensorFlow Lite (TFLite) Quantization refers to the comprehensive tooling and runtime support within the TFLite framework for deploying quantized models on mobile and embedded devices. It provides full-stack support for asymmetric quantization schemes. Key features include:

• Converter Tools: The TFLiteConverter with options for full-integer quantization (requiring a representative dataset for calibration) and dynamic range quantization.
• Kernel Support: A wide library of optimized integer kernels (e.g., for INT8) that leverage asymmetric quantization parameters.
• Delegate Integration: Seamless execution of quantized models on hardware accelerators like the Google Edge TPU, which are designed for low-precision arithmetic. TFLite's quantization is a primary industry reference for on-device deployment.