Quantization Granularity

Per-tensor quantization applies a single set of quantization parameters to an entire tensor. This is the simplest and most coarse-grained method.

• Mechanism: One scale factor and one zero-point are calculated for the whole tensor (e.g., all weights in a layer or all activations from a layer).
• Advantages: Minimal metadata overhead and simple, fast dequantization logic. It is highly efficient for hardware with simple data paths.
• Disadvantages: Accuracy loss can be significant if the tensor's value distribution is wide or non-uniform, as a single scale must accommodate the entire range, leading to higher quantization error.
• Typical Use: Baseline method in many frameworks; suitable for layers with tightly clustered values.

Per-channel quantization (or per-axis) assigns unique quantization parameters to each output channel of a weight tensor in convolutional or fully connected layers.

• Mechanism: For a weight tensor of shape [OC, IC, KH, KW], a separate scale and zero-point is calculated for each of the OC output channels. Activations are typically still quantized per-tensor.
• Advantages: Dramatically improves accuracy over per-tensor by accounting for varying dynamic ranges across channels. This is the standard for weight quantization in production.
• Disadvantages: Increases parameter storage slightly and requires more complex integer-only inference kernels that apply different scales per channel.
• Hardware Support: Modern NPUs and inference engines like TensorRT and TFLite have optimized support for per-channel operations.

Per-token quantization (also called per-activation dynamic quantization) calculates a unique scale factor for each token or input sample in a sequence at runtime.

• Mechanism: The scaling factor for activation tensors is computed dynamically for every input. For a batch of sequences, this is often done per token position.
• Advantages: Excellent accuracy for models with highly variable activation ranges across inputs, such as Large Language Models (LLMs) where token values can vary widely.
• Disadvantages: Introduces runtime overhead for calculating scales and requires support for dynamic quantization in the execution backend. Not suitable for strict integer-only inference without hybrid kernels.
• Typical Use: Dynamic quantization of activations in Transformer-based models to preserve accuracy on diverse textual inputs.

Group-wise quantization partitions elements within a tensor into groups (e.g., subsets of channels or blocks of weights) and applies separate quantization parameters to each group.

• Mechanism: For a weight tensor, elements are grouped along the channel or input dimension. Common configurations are 32, 64, or 128 elements per group.
• Advantages: Offers a middle ground between per-tensor and per-channel granularity. It provides better accuracy than per-tensor with lower metadata overhead than per-channel.
• Disadvantages: Increases kernel implementation complexity. The irregular memory access patterns for loading group-specific scales can reduce hardware efficiency if not carefully optimized.
• Research & Application: Used in advanced compression techniques for very low-bit quantization (e.g., INT4, INT2) to mitigate accuracy collapse.

Block quantization is an even finer-grained approach where quantization parameters are defined for small, contiguous blocks of values within a tensor, such as 4x4 or 8x8 element blocks.

• Mechanism: The tensor is tiled into small blocks, each with its own scale (and sometimes zero-point). This is applied to both weight and activation tensors.
• Advantages: Can achieve very high accuracy for ultra-low precision formats (e.g., 2-bit or 3-bit) by closely modeling local value distributions.
• Disadvantages: Significant metadata overhead (scales must be stored for many blocks) and complex, often inefficient computation patterns that may not map well to standard NPU vector units.
• State of Practice: Primarily a research technique for extreme compression; requires custom hardware or kernels for practical deployment.

The selection of granularity is a systems engineering decision balancing accuracy, latency, and hardware capabilities.

• Accuracy vs. Overhead Trade-off: Finer granularity (per-channel, group-wise) improves accuracy but increases parameter management and can complicate kernel fusion.
• Hardware Constraints: The quantization backend (e.g., TensorRT, Core ML) dictates supported granularities. Per-channel is widely supported; finer methods may require custom operators.
• Model Architecture Sensitivity: Convolutional layers benefit greatly from per-channel weight quantization. Transformer FFN layers may also use per-channel, while attention activations often need per-token scaling.
• Deployment Target: Edge AI and TinyML deployments favoring integer-only inference may prioritize simpler per-tensor or per-channel schemes over dynamic per-token methods to ensure deterministic latency and power usage.

A granularity method where a unique set of quantization parameters (scale and zero-point) is applied to each output channel of a convolutional or fully connected weight tensor. This accounts for varying weight distributions across channels, typically offering higher accuracy than per-tensor quantization at the cost of storing more scale/zero-point values and requiring slightly more complex arithmetic.

• Primary Use: Convolutional layers where weight value ranges differ significantly per filter.
• Trade-off: Improved accuracy vs. increased parameter overhead and computational complexity.

The two fundamental parameters in affine quantization that define the linear mapping between floating-point and integer value ranges. The scale is a floating-point number that determines the step size per integer unit. The zero-point is an integer value that corresponds to the real value zero in the floating-point range, allowing for asymmetric quantization.

• Function: float_value = scale * (int_value - zero_point)
• Granularity Impact: These parameters are defined at the chosen granularity level (e.g., one set per-tensor or per-channel).

Two schemes defining the mapping range. Symmetric quantization constrains the range to be symmetric around zero, often simplifying the zero-point to 0. Asymmetric quantization allows the range to be offset, better capturing skewed data distributions (e.g., ReLU activations that are all non-negative).

• Symmetric: Simpler, faster integer math. Common for weight quantization.
• Asymmetric: More accurate for activations. Requires zero-point arithmetic.
• Granularity Link: The choice influences how scale/zero-point are calculated per granularity unit.

A process that simulates the effects of quantization during the training phase by inserting fake quantization operations. This allows the model to learn to compensate for the expected quantization error, leading to higher accuracy upon deployment compared to Post-Training Quantization (PTQ).

• Key Technique: Uses fake quantization nodes that round and clip values in the forward pass but pass full-precision gradients backward.
• Granularity Integration: The chosen granularity (per-tensor, per-channel) is baked into the simulation during QAT, allowing the model to adapt specifically to that scheme.

A model compression technique that converts a pre-trained model's weights and activations to lower precision without retraining. It uses a calibration dataset to calculate optimal quantization parameters (scale, zero-point) for the chosen granularity.

• Process: Statistics (min/max, histograms) are collected from the calibration forward passes to determine ranges.
• Granularity Criticality: The choice of granularity (e.g., per-channel vs. per-tensor) is a primary hyperparameter in PTQ, directly trading final accuracy for simplicity.

An execution paradigm where the entire neural network inference graph operates using integer arithmetic, eliminating floating-point operations. This requires quantization schemes (like affine quantization) and granularity choices that allow all layers, including activation functions, to be executed with integers.

• Enabling Technology: Crucial for deployment on low-power NPUs, microcontrollers, and edge devices.
• Granularity Constraint: The selected quantization granularity must be compatible with the hardware's integer operation capabilities (e.g., per-channel support).