Static Quantization

Unlike dynamic quantization, static quantization determines its quantization parameters—specifically the scale and zero-point—offline. This is done by passing a representative calibration dataset through the model to observe the range of activation tensors. These fixed parameters are then embedded into the model, eliminating the need for runtime range analysis.

• Process: A small, unlabeled dataset (e.g., 100-500 samples) is fed through the model.
• Output: Min/max values or histograms are collected for each activation layer.
• Result: A single, constant scale and zero-point is calculated per tensor (or per channel).

Once calibrated, the model's computational graph is transformed and frozen. All floating-point operations for weights and activations are replaced with integer arithmetic where possible. This creates a predictable, static execution path.

• Graph Optimization: The quantized graph often undergoes further optimizations like operator fusion (e.g., Conv + ReLU + Quantize).
• Determinism: The absence of runtime scaling calculations leads to highly deterministic latency.
• Deployment: The final model is typically exported to formats like Quantized ONNX, TFLite FlatBuffer, or engine formats like TensorRT Plan, which are optimized for integer execution.

By moving all quantization parameter calculations to the calibration phase, static quantization achieves minimal overhead during inference. This makes it ideal for high-throughput, latency-sensitive deployments.

• Key Benefit: Eliminates the per-inference cost of computing activation ranges.
• Hardware Advantage: Integer operations (INT8) are typically faster and more energy-efficient than floating-point (FP32/FP16) on dedicated AI accelerators (e.g., NVIDIA Tensor Cores in INT8 mode, Google TPUs, Intel DL Boost).
• Use Case: Perfect for batch processing and high-query-per-second (QPS) serving where every microsecond counts.

The accuracy of a statically quantized model is directly tied to the representativeness of its calibration dataset. If the calibration data does not match the statistical distribution of production inference data, significant accuracy degradation can occur.

• Risk: Quantization error amplifies if runtime activations fall outside the ranges observed during calibration.
• Best Practice: The calibration set should be a random, unbiased sample from the training or validation distribution.
• Tuning: Methods like percentile calibration (e.g., using the 99.9th percentile instead of the absolute max) can provide robustness to outliers.

Static quantization allows for different granularities in applying the scale and zero-point parameters, offering a trade-off between accuracy and complexity.

• Per-Tensor Quantization: Applies a single scale and zero-point to an entire tensor. This is simpler and widely supported but can be less accurate if the tensor's values have a wide or non-uniform distribution.
• Per-Channel Quantization: Uses separate scale and zero-point values for each channel (typically for weight tensors in convolutional or linear layers). This finer granularity better captures the data distribution, often leading to higher accuracy, but requires more parameters and slightly more complex kernel support.

A key design choice in static quantization is whether to use symmetric or asymmetric quantization for weights and activations.

• Symmetric Quantization: The quantized range is centered around zero. The zero-point is fixed at 0, simplifying the integer arithmetic (no zero-point addition needed during convolution). This is commonly used for weight tensors.
• Asymmetric Quantization: The quantized range is mapped to the observed min/max of the tensor, requiring a non-zero zero-point. This is more expressive and is typically used for activation tensors, which often have a non-symmetric distribution (e.g., after a ReLU activation, which is all non-negative).

Dynamic quantization determines the scaling factors (scale and zero-point) for a model's activations at runtime for each individual input. Unlike static quantization, it does not require a calibration dataset for activations, offering flexibility for inputs with highly variable ranges.

• Key Difference: Activation ranges are computed on-the-fly.
• Use Case: Models where activation statistics vary significantly per input (e.g., some NLP models).
• Trade-off: Eliminates calibration overhead but introduces minor runtime computation to determine quantization parameters.

Quantization-aware training is a method where quantization error is simulated during the training or fine-tuning process. Fake quantization nodes are inserted into the forward pass, allowing the model to learn parameters that are robust to the precision loss incurred during subsequent static or dynamic quantization.

• Process: Model trains with simulated rounding/clipping.
• Outcome: Typically achieves higher accuracy than Post-Training Quantization (PTQ).
• Cost: Requires retraining, which adds computational expense compared to calibration-only methods like static quantization.

A calibration dataset is a small, representative sample of the inference data used in static quantization to analyze the statistical distribution (min/max values) of model activations. This analysis determines the fixed scale and zero-point parameters for converting floats to integers.

• Purpose: To compute quantization parameters without labels.
• Size: Typically a few hundred samples are sufficient.
• Criticality: The quality and representativeness of this dataset directly impact the final quantized model's accuracy.

INT8 quantization is the specific practice of representing model weights and activations using 8-bit integers. It is the most common target precision for static quantization due to widespread hardware support.

• Benefit: Reduces model size and memory bandwidth by ~4x compared to FP32.
• Hardware Acceleration: Executes efficiently on integer arithmetic units (e.g., NVIDIA Tensor Cores in INT8 mode, CPU VNNI instructions).
• Static Implementation: In static INT8 quantization, all scale/zero-point values for weights and activations are pre-computed and fixed.

Per-channel quantization is a granular scheme where separate quantization parameters (scale and zero-point) are calculated for each output channel of a weight tensor (e.g., in a convolutional layer). This contrasts with per-tensor quantization, which uses one set of parameters for the entire tensor.

• Advantage: Accounts for varying ranges across channels, typically yielding higher accuracy than per-tensor quantization.
• Complexity: Adds minimal overhead but requires hardware/kernel support.
• Static Application: In static quantization, these per-channel parameters are determined during calibration and remain fixed.

Dequantization is the operation that converts quantized integer values back into floating-point numbers. In a static quantization graph, dequantization nodes are strategically placed where higher precision is required for numerical fidelity.

• Formula: float_value = scale * (int_value - zero_point).
• Role in Static Graphs: While core operations (like matrix multiplies) use integers, their outputs are often dequantized for subsequent non-linear functions or layer outputs.
• Fixed-Point Execution: The pattern of integer ops followed by dequantization is sometimes called "fixed-point" inference.