Activation Quantization

A core distinction in activation quantization is whether the quantization parameters are dynamic or static. Dynamic quantization calculates the scale and zero-point for each activation tensor at runtime based on the observed data, offering flexibility for inputs with highly variable ranges. Static quantization determines these parameters once during a calibration phase using a representative dataset, fixing them for all subsequent inferences. Static quantization offers lower runtime overhead and is standard for production deployment, while dynamic quantization is often a fallback for models with unstable activation ranges.

The granularity defines the scope over which a single set of quantization parameters (scale, zero-point) is applied. Common schemes are:

• Per-tensor: A single scale/zero-point is used for an entire activation tensor. This is the simplest but can introduce significant error if the tensor's distribution is non-uniform.
• Per-channel: Primarily used for weight tensors, but some advanced schemes apply it to activations, using separate parameters for each channel. This offers higher accuracy at the cost of increased parameter storage and computational complexity.
• Group-wise: A compromise where parameters are shared across groups of channels or spatial dimensions within a tensor, balancing accuracy and efficiency.

For static quantization, calibration is the critical step to determine optimal quantization parameters. A small, representative dataset (the calibration set) is passed through the model. Statistics are collected on the activation distributions of each target layer. Common calibration algorithms include:

• Min-Max: Uses the absolute minimum and maximum observed values to set the range.
• Moving Average Min-Max: Averages min/max over batches to smooth outliers.
• Entropy Minimization (KL Divergence): Selects a range that minimizes the information loss between the original and quantized distributions, often yielding the best accuracy. Poor calibration is a primary source of quantization-induced accuracy loss.

Not all layers or activations quantize equally. Key architectural considerations include:

• Non-linearities: Functions like ReLU are quantization-friendly as they produce non-negative, often sparse outputs. Functions like SiLU or GELU produce values across a wider range and can be more challenging.
• Residual Connections: Add operations require careful handling to ensure quantized tensors from different branches are aligned to the same scale/zero-point, often requiring requantization steps.
• Attention Mechanisms: The dynamic range of attention scores and softmax outputs in transformers can be large, sometimes necessitating higher precision (e.g., 16-bit) for these specific operations within an otherwise 8-bit model.

The benefits of activation quantization are substantial but come with inherent trade-offs:

• Memory Bandwidth Reduction: Moving 8-bit activations instead of 32-bit floats cuts the required DRAM bandwidth by 75%, a major bottleneck in inference.
• Integer Arithmetic Acceleration: Enables the use of fast, low-power integer units (INT8) on CPUs, GPUs, and NPUs, avoiding slower floating-point math.
• Accuracy vs. Efficiency Trade-off: The primary cost is a potential drop in model accuracy (quantization error). The severity depends on model robustness, calibration quality, and bit-width. A 1-2% accuracy drop for INT8 is often acceptable for massive efficiency gains.
• Increased Design Complexity: Introduces new hyperparameters (calibration method, granularity) and requires specialized toolchains (e.g., TensorRT, TFLite, ONNX Runtime).

Activation quantization's value is realized through hardware support. Key execution contexts include:

• CPU (x86/ARM): Leverages vectorized integer instructions (e.g., AVX2, NEON) for INT8 matrix multiplication, providing a significant speed-up over FP32.
• GPU (NVIDIA): Uses Tensor Cores (e.g., on Ampere, Hopper architectures) for dense INT8 operations, requiring specific kernel implementations and data layouts.
• Neural Processing Unit (NPU): Dedicated AI accelerators are designed from the ground up for low-precision integer math. They feature specialized dataflows and on-chip memory hierarchies that are optimal for quantized models, often achieving order-of-magnitude improvements in performance-per-watt.

Weight quantization reduces the numerical precision of a neural network's stored weight parameters. This directly shrinks the model's memory footprint, as weights typically constitute the majority of a model's size. Unlike activations, weights are static after training, making their quantization more straightforward. Common targets are 8-bit (INT8) or 4-bit (INT4) integers.

• Primary Benefit: Drastically reduces model file size for storage and loading.
• Static Nature: Quantization parameters (scale/zero-point) are calculated once and fixed.
• Combined Approach: Used in conjunction with activation quantization to enable fully integer inference.

Quantization-Aware Training (QAT) is a process where quantization effects are simulated during the training or fine-tuning phase. This allows the model to learn parameters that are robust to the precision loss incurred during subsequent integer conversion.

• Forward Pass: Uses fake quantization nodes that apply rounding and clamping.
• Backward Pass: Employs a Straight-Through Estimator (STE) to approximate gradients through the non-differentiable quantization function.
• Outcome: Produces models that maintain significantly higher accuracy after full integer quantization compared to Post-Training Quantization (PTQ).

Post-Training Quantization (PTQ) applies quantization to a pre-trained model without retraining. It involves analyzing a representative dataset (calibration) to determine optimal quantization parameters for weights and activations.

• Calibration: A small, unlabeled dataset is passed through the model to collect activation range statistics.
• Speed vs. Accuracy: Faster than QAT but may incur higher accuracy loss, especially for activations.
• Static vs. Dynamic: Static PTQ fixes activation ranges after calibration; Dynamic PTQ calculates them at runtime.

Integer quantization is the overarching technique of constraining network parameters (weights and activations) to integer values. This enables execution on hardware with native, high-throughput integer arithmetic units (ALUs), which are more power-efficient than floating-point units.

• Hardware Acceleration: Enables use of integer-only cores in CPUs, GPUs, and dedicated NPUs.
• Full Integer Inference: The goal is to eliminate floating-point operations entirely during inference.
• Dequantization: May still be required for specific operations or layer outputs, adding overhead.

Calibration is the critical data analysis step in PTQ and QAT that determines the quantization parameters. For activations, it involves passing representative input data through the network to observe the statistical distribution of each layer's output.

• Objective: Find the optimal clipping range (min/max) to minimize quantization error.
• Methods: Include min/max, entropy minimization, and percentile-based clipping.
• Granularity: Can be performed per-tensor (one range for all values) or per-channel (more accurate). Poor calibration is a primary source of quantization accuracy loss.

Quantization error is the numerical distortion introduced when mapping a continuous range of floating-point values to a finite set of discrete integer levels. It is the fundamental trade-off in any quantization scheme.

• Sources: Includes rounding error (from nearest-level mapping) and clipping error (from values outside the representable range).
• Propagation: Error accumulates across layers, potentially degrading model accuracy.
• Management: Techniques like bias correction, fine-tuning, and advanced calibration aim to mitigate this error. The error is more pronounced for activations than for weights due to their dynamic range.