Integer Quantization

The core mechanism of integer quantization is the mapping of high-precision floating-point numbers (e.g., FP32) to low-precision integers (e.g., INT8). This reduces the bitwidth of each parameter and activation, directly shrinking the model's memory footprint and enabling the use of faster, lower-power integer arithmetic units on hardware.

• Memory Savings: A model's weights stored in INT8 occupy 75% less memory than FP32.
• Compute Efficiency: Integer matrix multiplication (INT8 GEMM) is typically 2-4x faster than its FP32 equivalent on supported hardware.

The mapping from float to integer is controlled by two learned parameters: the scale and the zero-point. These define the linear transformation.

• Scale (S): A floating-point number that defines the ratio between the quantized integer range and the original float range. Formula: S = (float_max - float_min) / (quant_max - quant_min).
• Zero-Point (Z): An integer value that corresponds exactly to the real value zero in the float domain. It ensures real zero can be represented without error, which is critical for operations like padding.

The quantization formula is: Q = round( R / S ) + Z, where R is the real float value and Q is the quantized integer.

This defines how the quantization range is centered, impacting accuracy and computational overhead.

• Symmetric Quantization: The range is centered on zero. The zero-point is forced to 0, simplifying the dequantization math. This is efficient but can be wasteful if the actual data distribution is not symmetric.
• Asymmetric Quantization: The range is not centered on zero. A non-zero zero-point is used to precisely capture the full, potentially skewed, range of the data (e.g., ReLU activations which are all >=0). This preserves accuracy but adds a small computational cost for the zero-point offset during operations.

This defines the scope over which a single set of quantization parameters (scale, zero-point) is applied.

• Per-Tensor Quantization: A single scale and zero-point is used for an entire tensor. This is simple but can lead to high error if the tensor's values have high variance.
• Per-Channel Quantization: A unique scale and zero-point is applied to each output channel of a weight tensor (common for convolutional and linear layers). This accounts for variation across channels and is the standard for weight quantization in frameworks like TensorRT and TFLite, as it significantly reduces quantization error with minimal overhead.

This characteristic defines when the quantization parameters for activations are determined.

• Static Quantization: The scale and zero-point for activations are pre-calculated using a calibration dataset before deployment. This allows for full graph optimization and kernel fusion, offering the highest inference speed.
• Dynamic Quantization: The scale and zero-point for activations are computed on-the-fly during inference based on the observed range of each input tensor. This is more flexible and accurate for varying inputs but introduces runtime overhead. Often applied to models with dynamic computation graphs or highly variable inputs.

The practical benefit of integer quantization is realized through native hardware support. Modern AI accelerators (GPUs, NPUs, CPUs) have dedicated integer matrix multiplication units.

• INT8 GEMM Cores: NVIDIA Tensor Cores (from Turing architecture onward) and Intel DL Boost (VNNI) provide massive throughput for INT8 operations.
• Kernel Fusion: Inference engines like TensorRT and OpenVINO fuse quantization/dequantization operations with adjacent layers (e.g., Conv + ReLU) into a single, optimized integer kernel, eliminating memory traffic for intermediate results and maximizing speed.
• Compiler-Level Optimization: Frameworks like Apache TVM and XLA perform graph-level transformations to ensure the entire computational graph runs in integer space.

Dedicated neural processing units are architected for integer math. Key examples include:

• Google TPUs: Utilize bfloat16 and INT8 precision with systolic arrays for massive matrix multiplication throughput.
• Apple Neural Engine: An NPU in Apple Silicon (M-series, A-series) optimized for INT8 and INT16 operations, accelerating Core ML models.
• Qualcomm Hexagon DSP: Features a Hexagon Tensor Accelerator (HTA) for INT8 and INT16 inference on Snapdragon platforms.
• Intel AMX: Advanced Matrix Extensions in Xeon CPUs provide INT8 acceleration for server-side inference. These units have dedicated silicon for integer multiply-accumulate (MAC) operations, offering orders of magnitude better performance-per-watt than general-purpose cores.

ML compilers lower high-level models to optimized hardware-specific code, often integrating quantization passes.

• Apache TVM: Its AutoTVM and Ansor schedulers can automatically generate and tune high-performance INT8 kernels for various CPU, GPU, and accelerator backends.
• IREE (MLIR-based): Uses the MLIR compiler infrastructure to represent quantization parameters in its IR and compile for Vulkan GPUs and CPUs, enabling dynamic quantization support.
• XLA (Accelerated Linear Algebra): Google's compiler for TensorFlow/JAX can fuse operations and target TPUs or GPUs with INT8 support, often in conjunction with mixed-precision computation patterns. These compilers perform graph-level optimizations like constant folding and layout transformations that are crucial for maximizing quantized inference speed.

Defines the scope over which a single set of quantization parameters is shared.

• Per-Tensor Quantization: Applies one scale and zero-point to an entire tensor. Simple but less accurate.
• Per-Channel Quantization: Applies unique parameters to each channel of a weight tensor (e.g., each output channel of a convolution). Accounts for variation, preserving higher accuracy, especially for weights.
• Per-Token/Per-Axis: Variations applied to activation tensors.

The mathematical method for mapping floating-point to integer values.

• Symmetric Quantization: The quantization range is symmetric around zero (zero-point = 0). Simplifies computation.
• Asymmetric Quantization: The range is not centered on zero, using a separate zero-point. Better represents asymmetric data (e.g., ReLU activations).
• Uniform vs. Non-Uniform: Uniform uses evenly spaced levels; non-uniform allows denser levels where data clusters, reducing error but complicating hardware support.

Strategies for extreme compression or balanced performance.

• Low-Bit Quantization: Reduces precision to 4-bits or fewer, enabling deployment on microcontrollers and edge devices. Requires sophisticated methods to manage accuracy loss.
• Mixed-Precision Quantization: Applies different bitwidths (e.g., 4-bit, 8-bit, 16-bit) to different layers or tensors. Guided by quantization sensitivity analysis, it optimizes the trade-off between model size, compute cost, and accuracy.