Non-Uniform Quantization

Unlike uniform quantization, which uses a constant step size (Δ), non-uniform quantization employs variable step sizes between quantization levels. This allows the quantization grid to be denser in regions where the data distribution (e.g., weights or activations) is highly concentrated and sparser in regions with few values. The step size is typically determined by a non-linear function, such as a logarithmic or exponential mapping. This characteristic is fundamental to its ability to reduce quantization error for non-uniformly distributed data.

Non-uniform quantization is particularly effective for data that does not follow a uniform or symmetric Gaussian distribution. Many neural network activations (e.g., after ReLU) and certain weight distributions are highly skewed or have long tails. Uniform quantization wastes precious representational capacity on empty regions of the numeric range. By allocating more quantization levels to high-probability regions, non-uniform schemes like logarithmic quantization or those based on K-means clustering (e.g., GPTQ) better capture the statistical properties of the data, preserving model accuracy at lower bitwidths.

A key implementation characteristic is the reliance on Lookup Tables (LUTs). Because the quantized values are not linearly spaced, performing operations like multiplication directly on the integer codes is inefficient. Instead, the integer indices are used to look up pre-computed results in a table. For example, in a matrix multiplication between non-uniformly quantized tensors, the operation A_int * B_int is replaced with a lookup LUT[A_int, B_int]. This trades off computational arithmetic for memory bandwidth, which can be beneficial on certain hardware but adds complexity to kernel design.

The variable step sizes and need for LUTs increase hardware implementation complexity compared to uniform integer quantization. Standard integer arithmetic logic units (ALUs) and vectorized instructions (like Intel VNNI or NVIDIA DP4A) are designed for efficient uniform integer math. Non-uniform quantization often requires custom hardware support or significant software overhead to manage the decoding and LUT accesses. This is a primary reason uniform INT8 quantization remains more prevalent in production, despite the potential accuracy benefits of non-uniform approaches for specific models.

Several established schemes implement non-uniform quantization:

• Logarithmic Quantization: Uses an exponential step size, mapping to powers-of-two. This is highly hardware-friendly as multiplication becomes bit-shift operations.
• K-means / Clustering-Based Quantization: Algorithms like GPTQ or AWQ cluster weight values and assign a centroid to each cluster, creating an optimal non-uniform codebook.
• μ-Law Companding: A companding technique from audio signal processing that applies a logarithmic transformation before uniform quantization, effectively creating a non-linear quantization characteristic. Each scheme makes a different trade-off between accuracy, computational cost, and hardware compatibility.

Non-uniform quantization shines in extreme low-bit quantization scenarios, such as 4-bit or 2-bit inference. At these ultra-low precisions, the limited number of representable values makes the allocation of levels critically important. Uniform quantization's linear spacing often leads to catastrophic accuracy loss for complex models. Non-uniform methods, by concentrating levels where they matter most, can maintain usable accuracy. This makes them a key technique for on-device deployment and memory-constrained edge AI applications where model size is the paramount constraint.

This classic method treats quantization as a clustering problem. It iteratively:

• Assigns data points to the nearest quantization level (centroid).
• Updates each centroid to the mean of its assigned points. The algorithm converges to levels that minimize the mean squared error (MSE). It's optimal for minimizing distortion but computationally intensive for calibration.

This method uses a logarithmic function to map values, creating finer resolution for small magnitudes and coarser resolution for large ones. It is highly effective for data with a long-tailed distribution, such as weight values in deep neural networks. A common implementation is PoT (Power-of-Two) quantization, where levels are restricted to powers of two, enabling efficient hardware implementation using bit-shift operations instead of multipliers.

This method determines quantization levels by analyzing the cumulative distribution function (CDF) of the calibration data. Levels are placed at specific quantiles (e.g., the 1/4, 1/2, and 3/4 points). This ensures an equal number of data points fall into each quantization bin, which is optimal for minimizing information loss rather than just MSE. It is robust to outliers.

OBQ is a greedy, layer-wise method inspired by pruning techniques. It:

• Quantizes one weight at a time to its nearest allowed level.
• Computes and applies a Hessian-informed update to the remaining weights to compensate for the error. This iterative correction makes it highly accurate for low-bit quantization (e.g., 4-bit) of weights, as it accounts for the cross-correlation between parameters.

These are state-of-the-art post-training methods for weight-only non-uniform quantization in Large Language Models (LLMs).

• GPTQ uses approximate second-order (Hessian) information to quantize weights in groups, applying corrective updates to minimize layer-wise error.
• AWQ identifies and preserves a small fraction of salient weights in higher precision (FP16), while quantizing the rest aggresively. It argues that protecting just 0.1%-1% of weights significantly reduces quantization error.

This method integrates the hardware cost model directly into the quantization search process. During Quantization-Aware Training (QAT), a search algorithm (e.g., differentiable NAS) explores a space of non-uniform quantization policies. The loss function includes both task accuracy and a penalty for estimated latency or energy consumption on the target hardware (e.g., a specific NPU), resulting in a Pareto-optimal solution.

Uniform Quantization is a method where the quantized levels are evenly spaced across the representable range, resulting in a constant step size (Δ) between adjacent quantization bins. This is the most common and hardware-friendly approach.

• Mechanism: Maps a floating-point range [α, β] to an integer range using a single, fixed scale factor.
• Trade-off: Simpler and faster to compute but can introduce higher error if the data distribution is not uniform, as it allocates equal precision across the entire range, including sparse regions.

Quantization-Aware Training is a model optimization technique that simulates quantization noise during the training process. This allows the model's weights and activations to adapt to the lower precision, typically recovering accuracy lost by Post-Training Quantization.

• Process: Uses Fake Quantization nodes during forward and backward passes. Gradients are approximated through the non-differentiable quantization function using a Straight-Through Estimator (STE).
• Use Case: Essential for achieving high accuracy with very low bit-widths (e.g., INT4) or for models sensitive to quantization error.

Post-Training Quantization is a compression technique applied to a pre-trained model without requiring retraining. It reduces the numerical precision of weights and activations using a small Calibration Dataset to determine optimal quantization parameters.

• Variants: Includes Static Quantization (fixed calibration) and Dynamic Quantization (on-the-fly activation range calculation).
• Advantage: Fast and requires no labeled data for retraining, making it the most common method for production deployment.

Mixed-Precision Quantization is a strategy that applies different bit-widths to different layers, blocks, or even tensors within a single model. The goal is to optimize the trade-off between compression, speed, and accuracy.

• Methodology: Guided by Quantization Sensitivity Analysis, which identifies layers where precision is critical for task performance.
• Outcome: For example, attention mechanisms might use 8-bit (INT8) while embedding layers remain at 16-bit (BF16), achieving a better accuracy-size ratio than uniform low-bit quantization.

Per-Channel Quantization is a granularity scheme where a unique set of quantization parameters (scale and zero-point) is applied to each output channel of a weight tensor (e.g., each filter in a convolutional layer).

• Contrast with Per-Tensor: Per-Tensor Quantization uses one set of parameters for the entire tensor.
• Benefit: Accounts for variation in weight distributions across channels, significantly reducing quantization error compared to per-tensor methods, especially for depthwise convolutions and fully-connected layers.

Quantization Error is the numerical discrepancy introduced when converting a continuous or high-precision value to a lower-precision, discrete representation. It is the fundamental trade-off in all quantization techniques.

• Composition: Comprises rounding error (from mapping to nearest integer) and clipping error (from values outside the quantizable range).
• Minimization: Non-uniform quantization specifically aims to minimize this error by allocating more discrete levels to regions of the data distribution where values are denser, reducing the expected error more effectively than uniform spacing.