Asymmetric Quantization

The zero-point is an integer value that represents the real numerical zero in the quantized domain. It is the critical parameter that enables asymmetric quantization by offsetting the integer range to align with the floating-point range.

• Purpose: Maps the real value 0.0 to a specific integer, allowing the quantized range to cover both positive and negative values efficiently.
• Calculation: Derived from the real minimum (r_min) and maximum (r_max) and the quantized integer range (e.g., 0 to 255 for 8-bit). Formula: zero_point = round(q_min - r_min / scale).
• Impact: A non-zero zero-point allows for a tighter, more accurate quantization range when the distribution of values is not symmetric around zero, such as with ReLU activations which are always non-negative.

The scale is a positive floating-point number that acts as the multiplicative factor for converting between floating-point and integer representations. It defines the resolution of the quantization.

• Purpose: Determines the size of each quantization bin. Formula: scale = (r_max - r_min) / (q_max - q_min).
• Interpretation: A smaller scale means higher precision, as each integer step corresponds to a smaller change in the original floating-point value.
• Example: If the real range is [-2.5, 5.5] and the quantized range is [0, 255], the scale is (5.5 - (-2.5)) / 255 ≈ 0.0314. Each integer increment represents a change of ~0.0314 in the original data.

The quantization range defines the minimum and maximum integer values available for representing the quantized data. It is defined by the chosen bitwidth.

• Definition: For N-bit quantization, the integer range is typically [0, 2^N - 1] for unsigned integers (uint8) or [-2^(N-1), 2^(N-1)-1] for signed integers (int8).
• Role: This range, combined with the scale and zero-point, determines which real values can be represented and the granularity of the representation.
• Trade-off: A wider real range covered by the same integer range (e.g., 256 values for 8-bit) leads to a larger scale and lower precision, increasing quantization error.

The core mathematical operations of asymmetric quantization are defined by the quantize and dequantize functions using the scale (S) and zero-point (Z).

• Quantize (Float -> Int): q = round(r / S) + Z Clamps q to the integer quantization range.
• Dequantize (Int -> Float): r' = S * (q - Z)
• Key Property: This formulation ensures that r' is an approximation of the original real value r. The zero-point allows the integer Z to exactly represent r = 0.0, minimizing error for zero values, which is common in activations and sparse weights.

Calibration is the process of determining optimal scale and zero-point parameters, typically for activations, using a representative dataset.

• Process: Run a pre-trained model on a small calibration dataset (100-500 samples) and collect the observed ranges (min/max) of activations for each layer.
• Methods:
  • Min-Max: Uses the absolute observed minimum and maximum values. Simple but sensitive to outliers.
  • Entropy / KL-Divergence: Selects a range that minimizes the information loss between the original and quantized distributions. Used in frameworks like TensorRT.
• Outcome: Produces a set of static scale and zero-point parameters for each activations tensor, enabling static asymmetric quantization.

Asymmetric quantization is defined by its key differentiator from symmetric quantization: a non-zero zero-point.

This distinction allows asymmetric quantization to represent the full observed range of data with less quantization error when the distribution is not centered on zero.

Symmetric Quantization is a scheme where the quantization range is symmetric around zero, simplifying the quantization and dequantization formulas by setting the zero-point to 0. This is in direct contrast to asymmetric quantization.

• Key Difference: Does not use a separate zero-point; the integer '0' directly represents the real value 0.0.
• Use Case: Often used for weight tensors where the distribution is roughly symmetric around zero, as it simplifies computation.
• Trade-off: While computationally simpler, it can be less efficient for activations with asymmetric distributions (e.g., after a ReLU function, which produces only non-negative values), as it wastes part of the quantized range.

The Quantization Zero-Point is an integer value that represents the real numerical zero in the quantized domain. It is the defining parameter of asymmetric quantization.

• Function: Aligns the integer and floating-point number lines, allowing the integer value zero_point to map precisely to the float 0.0.
• Calculation: Derived during calibration based on the observed minimum value in the tensor: zero_point = round(-min / scale).
• Impact: Enables precise representation of asymmetric data. For example, it allows all-positive activation values (common in models using ReLU) to utilize the full integer range efficiently, from [0, 255] for INT8, rather than a symmetric range like [-127, 127].

These terms define the granularity at which quantization parameters (scale and zero-point) are applied.

• Per-Tensor Quantization: A single set of parameters is used for all values in an entire tensor. This is simpler but can introduce higher error if the tensor's values have high variance.
• Per-Channel Quantization: A unique set of parameters is applied to each channel (typically each output channel of a weight tensor). This accounts for variation across channels and generally preserves higher accuracy.
• Relation to Asymmetry: Both symmetric and asymmetric schemes can be applied with either per-tensor or per-channel granularity. Asymmetric per-channel quantization is common for convolutional or linear layer weights.

Post-Training Quantization (PTQ) is a compression technique that reduces the numerical precision of a pre-trained model's weights and activations without requiring retraining. Asymmetric quantization is frequently applied as a PTQ method.

• Process: Uses a small calibration dataset to observe activation ranges and calculate optimal scale and zero-point values.
• Static vs. Dynamic: Asymmetric quantization is typically static—the zero-point and scale are fixed after calibration. This differs from dynamic quantization, where activations are quantized on-the-fly during inference.
• Primary Benefit: Enables rapid deployment of efficient INT8 models with minimal engineering overhead, though some accuracy may be lost compared to quantization-aware training.

INT8 Inference is the execution of a neural network model using 8-bit integer precision for weights and activations. Asymmetric quantization is a primary method for converting models to INT8.

• Hardware Acceleration: Modern GPUs (e.g., NVIDIA Tensor Cores) and CPUs have dedicated instructions for INT8 matrix multiplication (DP4A, VNNI), offering up to 4x throughput and memory reduction compared to FP32.
• Dequantization: After integer operations, results are dequantized back to floating-point using the formula: float_value = scale * (int_value - zero_point).
• Toolchain Support: Frameworks like TensorRT, TFLite, and ONNX Runtime provide pipelines to apply asymmetric quantization and execute optimized INT8 kernels.

A Calibration Dataset is a small, representative set of data used during post-training quantization to determine optimal quantization parameters like scale and zero-point for asymmetric schemes.

• Purpose: To observe the actual range (min/max) of activation tensors during a forward pass, which is necessary because activation ranges are data-dependent.
• Size: Typically 100-500 unlabeled samples are sufficient.
• Calibration Algorithms: Simple min-max calibration is common, but more advanced methods exist:
  • Entropy Calibration: Minimizes the information loss (KL divergence) between the original and quantized distributions.
  • Percentile Calibration: Uses a percentile (e.g., 99.99%) of the observed range to ignore outliers and improve robustness.