Uniform Quantization

The defining feature of uniform quantization is a constant step size (Δ) between adjacent quantization bins. This step size is calculated as Δ = (max_value - min_value) / (2^b - 1), where b is the bitwidth. All values within the original floating-point range are mapped to the nearest integer multiple of Δ.

• Mechanism: The range [min_value, max_value] is divided into 2^b equally spaced intervals.
• Implication: The quantization error for any given input is bounded and predictable, but not necessarily minimized for non-uniform data distributions.

Uniform quantization can be implemented with either symmetric or asymmetric quantization ranges.

• Symmetric Quantization: The range is centered on zero (e.g., [-max_abs, +max_abs]). The zero-point is fixed at 0, simplifying the arithmetic: Q = round(r / scale).
• Asymmetric Quantization: The range matches the observed min/max of the tensor ([min, max]). A non-zero zero-point is used to map the real value 0.0 precisely to an integer, which is crucial for activations like ReLU outputs that are always non-negative.

Symmetric is computationally simpler; asymmetric often yields higher accuracy by better utilizing the integer range.

For static uniform quantization, the min and max range values (and thus the scale Δ) are determined once using a calibration dataset. This process is deterministic and offline.

• Process: A representative dataset is passed through the model, and statistics (e.g., running min/max, moving average) are collected for each tensor to be quantized.
• Outcome: Fixed scale and zero-point parameters are embedded into the model, enabling highly optimized, fixed-point inference kernels with no runtime overhead for range calculation.

Uniform quantization is exceptionally hardware-friendly because the constant step size enables the use of integer arithmetic units and vectorized instructions (e.g., SIMD).

• Integer Operations: Convolutions and matrix multiplications decompose into integer multiply-accumulate (MAC) operations, which are significantly faster and more energy-efficient than floating-point MACs on most hardware.
• Compiler Optimization: The predictable, linear mapping allows compilers (e.g., TVM, XNNPACK) and inference engines (e.g., TensorRT, TFLite) to aggressively fuse operations and generate optimized kernels for CPUs, GPUs, and NPUs.

The primary trade-off of uniform quantization is between implementation simplicity/hardware efficiency and potential accuracy loss.

• Pro: Simple, fast, universally supported. The linear mapping requires minimal computational overhead for quantization/dequantization.
• Con: It is suboptimal for long-tailed or non-uniform weight/activation distributions. Many values may cluster in a small part of the range, leading to high quantization error where the distribution is dense and wasted bins where it is sparse.

This limitation is the key driver for non-uniform quantization methods like logarithmic or percentile-based quantization.

Uniform quantization is the industry standard for production inference due to its hardware support.

• INT8 Inference: The most common target. Offers a 4x reduction in model size and memory bandwidth compared to FP32, with typically <1% accuracy drop for well-tuned models. Used for server-side (NVIDIA TensorRT) and mobile (TFLite) deployment.
• INT4 / Low-Bit Quantization: Used for extreme compression on edge devices. Requires more sophisticated Quantization-Aware Training (QAT) to maintain accuracy due to the coarse granularity of only 16 levels.
• Weights-Only Quantization: A popular technique where only model weights are uniformly quantized to INT8/INT4, while activations remain in higher precision (FP16/BF16), offering a good balance of speedup and accuracy preservation.

A quantization method where the discrete levels are not evenly spaced. The step size between bins varies, allowing for finer granularity in regions where the data distribution (e.g., weights or activations) is denser. This can achieve lower quantization error for a given bitwidth compared to uniform quantization but requires more complex, often lookup-based, computations during inference.

• Example: Using a logarithmic scale to place more bins near zero where activation values in a ReLU network often cluster.
• Trade-off: Reduced error vs. increased computational overhead for dequantization.

These are two schemes for defining the quantized range relative to zero.

• Symmetric Quantization: The quantized range is symmetric around zero (e.g., [-127, 127] for INT8). The zero-point is fixed at 0. This simplifies the quantization math but can waste representational capacity if the actual data range is not symmetric.
• Asymmetric Quantization: The quantized range is not centered on zero (e.g., [0, 255]). A separate zero-point integer maps the real value zero into the quantized range. This allows for a tighter fit to asymmetric data distributions (common in activations after ReLU) but adds an extra offset term to calculations.

Uniform quantization can be implemented using either symmetric or asymmetric ranges.

These terms define the granularity at which quantization parameters (scale, zero-point) are applied.

• Per-Tensor Quantization: A single set of parameters is used for an entire tensor. This is simpler and has minimal metadata overhead.
• Per-Channel Quantization: A unique set of parameters is applied to each channel (typically for weight tensors in convolutional or linear layers). This accounts for variation in magnitude across channels, significantly reducing quantization error and is the standard for weight quantization in frameworks like TensorRT and TFLite.

Uniform quantization is typically applied with per-channel granularity to weights for accuracy.

The core mathematical parameters that define a uniform quantization mapping.

• Scale (S): A floating-point multiplier. It defines the ratio between the quantized integer step size and the original floating-point range: S = (float_max - float_min) / (quant_max - quant_min).
• Zero-Point (Z): An integer value. In asymmetric quantization, it represents which quantized bin corresponds to the real value zero. It ensures that real zero is quantized without error, which is crucial for operations like padding.

The quantization formula is: Q = round(real_value / S) + Z. Dequantization is: real_value ≈ (Q - Z) * S.

The primary deployment target for uniform quantization. INT8 inference refers to executing a model using 8-bit integers for both weights and activations.

• Performance Gain: Reduces memory bandwidth and footprint by 4x compared to FP32. Allows for higher throughput as many hardware platforms (GPUs, CPUs, NPUs) have dedicated INT8 vector units that can perform more operations per clock cycle.
• Uniform quantization is the standard method to convert FP32 models to INT8 for efficient deployment via runtimes like TensorRT, OpenVINO, and TFLite.
• The process requires calibration to determine the optimal S and Z for each tensor.

A small, representative set of unlabeled data (typically 100-500 samples from the training set) used during Post-Training Quantization (PTQ).

Its purpose is to observe the range of activation values flowing through the model in order to determine the optimal scale and zero-point parameters for each tensor. The choice of calibration data and the algorithm (e.g., MinMax, Entropy, Percentile) for analyzing the ranges directly impacts the final quantized model's accuracy.

• Critical Role: In static uniform quantization, these ranges are fixed after calibration. Poor calibration data leads to clipping (values outside the range) or excessive quantization error.