Quantization Zero-Point

The Quantization Zero-Point is an integer value that represents the real numerical zero in the quantized domain. Its primary purpose is to align the integer and floating-point ranges in asymmetric quantization schemes. This allows for an accurate mapping of values when the original data distribution is not symmetric around zero, which is common for activations following ReLU functions or other non-negative outputs.

• Core Function: It ensures that the integer value 0 corresponds precisely to a specific floating-point value (often the minimum of the range), preventing a systematic bias in the quantized representation.
• Mathematical Role: In the quantization formula q = round(r / scale) + zero_point, the zero-point shifts the quantized integer values to cover the asymmetric floating-point range.

The zero-point is the defining differentiator between asymmetric and symmetric quantization.

• Asymmetric Quantization: Employs a non-zero zero_point. This is essential when the range of values to be quantized (e.g., a tensor's minimum and maximum) is not symmetric around zero. It uses the full integer range (e.g., 0 to 255 for INT8) efficiently, minimizing quantization error for such data.
• Symmetric Quantization: Sets zero_point = 0. This simplifies computations, as addition/subtraction of the zero-point can be omitted. However, it is only optimal when the value range is symmetric (e.g., -127 to 127). For asymmetric data, it wastes part of the representable integer range, potentially increasing error.

The choice impacts hardware efficiency and model accuracy, making zero-point a key design decision.

The zero-point (z) is intrinsically linked to the quantization scale (s) and the real value range [min_r, max_r].

Key Equations:

• Quantization: q = clamp(round(r / s) + z, q_min, q_max)
• Dequantization: r' = (q - z) * s

Calculation: For a target integer bit-width b (e.g., 8 bits), the integer range is [q_min, q_max] (e.g., [0, 255] for unsigned INT8). The zero-point is calculated to map the real minimum value: z = round(q_min - min_r / s)

This ensures min_r is quantized to q_min and max_r is quantized to q_max, utilizing the entire integer dynamic range. The scale s is derived from (max_r - min_r) / (q_max - q_min).

While the zero-point enables accurate asymmetric quantization, it introduces computational overhead that must be managed by hardware and software stacks.

• Integer-Only Arithmetic: A core goal of quantization is to replace floating-point multiply-accumulate (MAC) operations with integer MACs. With a non-zero zero-point, the standard convolution operation Y = X * W becomes: Y_q - z_y = (X_q - z_x) * (W_q - z_w) This expands to integer operations involving the zero-points of inputs (z_x), weights (z_w), and outputs (z_y).
• Optimization: Modern AI accelerators (e.g., NPUs, Google TPUs) and inference engines (e.g., TensorRT, TFLite) have dedicated hardware or optimized kernels to fuse these zero-point adjustments efficiently, minimizing the performance penalty.
• Trade-off: The added accuracy from using an asymmetric range must justify the slight increase in operation complexity compared to symmetric quantization.

For Post-Training Quantization (PTQ), the zero-point is not guessed but determined empirically through a process called calibration.

• Calibration Dataset: A small, representative set of unlabeled data is passed through the model.
• Range Observation: The minimum and maximum values (min_r, max_r) are observed for each activations tensor (and often pre-determined for weights).
• Algorithm Choice: The method for selecting the range influences the zero-point:
  • Min-Max: Uses the absolute observed min/max. Simple but can be skewed by outliers.
  • Entropy / KL-Divergence: Tries to minimize the information loss between the original and quantized distributions, often leading to a more robust zero-point.
• Result: The chosen min_r and max_r are plugged into the mathematical formulas to calculate the final, static scale and zero_point for static quantization.

The zero-point cannot be understood in isolation; its value and utility are defined by its relationship to other quantization constructs.

• Scale (s): The zero-point and scale are a paired set of parameters. They are always used together to quantize and dequantize values. The scale determines the resolution, while the zero-point determines the offset.
• Quantization Granularity:
  • Per-Tensor: A single zero-point is used for an entire tensor. This is simple and widely supported.
  • Per-Channel: Used primarily for weight tensors in convolutional and linear layers. Each output channel has its own zero-point (and scale), allowing for finer adjustment to the distribution of each filter, significantly improving accuracy.
• Quantization Grid: The zero-point anchors the mapping between the continuous floating-point grid and the discrete integer grid. In uniform quantization, the grid spacing is constant (defined by the scale), and the zero-point selects which integer bin corresponds to real zero.

The Quantization Scale is a multiplicative factor that defines the ratio between a value in the original floating-point range and its corresponding integer in the quantized range. It is calculated as scale = (float_max - float_min) / (quant_max - quant_min). The scale and zero-point work together in the fundamental quantization formula: q = round(r / scale) + zero_point. A smaller scale increases the resolution of the quantized representation but reduces the overall representable range.

Asymmetric Quantization is a scheme where the quantized integer range does not need to be symmetric around zero. This is the method that necessitates a zero-point. It is defined by:

• A scale factor.
• A zero-point, an integer that maps to the real value zero.

This scheme is crucial when the distribution of values to be quantized (like activations after a ReLU function, which are all non-negative) is not symmetric. It allows the full integer range to be used efficiently, minimizing quantization error for asymmetric data by aligning the real zero within the quantized range.

Symmetric Quantization is a scheme where the quantized integer range is symmetric around zero. This simplifies the process by fixing the zero-point to 0. The quantization formula becomes q = round(r / scale). It is typically used for weight tensors, which often have a roughly symmetric distribution around zero. The primary advantage is computational efficiency: integer operations with a zero-point of 0 eliminate the need for additional addition/subtraction steps during matrix multiplication, leading to faster kernel execution on supporting hardware.

Dequantization is the reverse process of converting a quantized integer value back into a (dequantized) floating-point number. It uses the same scale and zero-point parameters established during quantization. The formula is: r = (q - zero_point) * scale. This process is essential for:

• Interpreting the results of integer computations.
• Performing operations that are not efficiently done in integer space.
• Calibrating models during Quantization-Aware Training (QAT). The dequantized value r' is an approximation of the original floating-point value r, with the difference being the quantization error.

A Calibration Dataset is a small, representative set of unlabeled input data (e.g., 100-500 samples from the training set) used during Post-Training Quantization (PTQ). Its purpose is to observe the dynamic range of activations flowing through the model to determine optimal quantization parameters. For static quantization, this data is used to:

• Calculate the min/max ranges for each activation tensor.
• Derive the scale and zero-point for each tensor.
• Potentially choose between symmetric or asymmetric schemes per layer. The quality and representativeness of this dataset directly impact the final quantized model's accuracy.

Integer-Only Inference refers to executing a full neural network forward pass using only integer arithmetic, without converting back to floating-point between layers. Achieving this requires careful design:

• Using symmetric quantization for weights (zero-point = 0) simplifies matrix multiplication.
• For asymmetric quantization of activations, the zero-point must be accounted for in the integer math. The core operation (e.g., for a fully connected layer) becomes an integer matrix multiplication followed by integer additions with fused zero-point terms.
• Non-linear functions like ReLU must be implemented using lookup tables or integer approximations. This mode is critical for deployment on microcontrollers and low-power NPUs without FPUs.