Per-Channel Quantization

The defining characteristic of per-channel quantization is its granularity. Instead of applying one set of parameters to an entire tensor, it calculates a separate scale and zero-point for each channel. For a weight tensor of shape [C_out, C_in, K_h, K_w] in a convolutional layer, quantization parameters are computed for each of the C_out output channels. This accounts for the fact that weight distributions can vary significantly across different filters or neurons.

Per-channel quantization typically yields higher accuracy compared to per-tensor quantization, especially for INT8 precision. This is because it better accommodates the varying dynamic ranges found across different channels. A channel with a narrow weight distribution can be quantized with high fidelity, while a channel with a wide distribution uses its own parameters, minimizing quantization error across the entire layer. It is the default method for weight quantization in frameworks like TensorRT and TensorFlow Lite for this reason.

For a given channel c, the quantization is defined by:

• Scale (s_c): s_c = (max(w_c) - min(w_c)) / (2^b - 1) for asymmetric, or s_c = max(|w_c|) / (2^(b-1) - 1) for symmetric.
• Zero-Point (z_c): An integer that maps real zero (for asymmetric quantization). The integer quantization of a weight value w in channel c is: q = clamp(round(w / s_c) + z_c, 0, 2^b - 1). Dequantization reconstructs an approximate value: w' = s_c * (q - z_c).

While more accurate, per-channel quantization introduces computational overhead. The inference kernel must load and apply a unique scale factor (and potentially zero-point) per channel during the dequantization step or within the fused integer operation. Modern AI accelerators (e.g., NVIDIA Tensor Cores with INT8, NPUs) have dedicated hardware support to efficiently handle these per-channel scaling operations, minimizing the performance penalty. The increased parameter storage is negligible (just a few extra floats per layer).

This table highlights the core differences:

Per-tensor is simpler but sacrifices accuracy by forcing one range to fit all channels.

It is a standard feature in production inference toolchains:

• TensorRT: Uses per-channel symmetric quantization for weights (default). Calibration determines scales per channel.
• TensorFlow Lite (TFLite): Supports per-channel quantization for convolutional and depthwise convolutional layers via its tf.lite.Optimize API.
• PyTorch (FBGEMM/QNNPACK backends): Torch.ao.quantization supports per-channel weight quantization for torch.nn.Linear and torch.nn.Conv2d modules.
• ONNX Runtime: Provides per-channel quantization through its quantization tool and execution providers. Implementation involves a calibration step to determine per-channel ranges from a representative dataset.

Per-Tensor Quantization is the foundational scheme where a single set of quantization parameters (one scale and one zero-point) is applied uniformly to all values within an entire weight or activation tensor. This contrasts with per-channel methods.

• Simplicity: Easier to implement and requires less metadata.
• Performance: Often leads to simpler, faster kernel implementations on hardware.
• Accuracy Trade-off: Can introduce higher error if the tensor's value distribution varies significantly across channels, as a single scale must accommodate the full range.

Quantization-Aware Training is a process that simulates quantization during training, allowing a model to learn parameters robust to the precision loss incurred during inference. It is often used to recover accuracy when applying aggressive schemes like per-channel quantization.

• Fake Quantization Nodes: Insert operations that mimic rounding and scaling in the forward pass while using the Straight-Through Estimator (STE) for gradients.
• Adaptation: Weights adapt to the quantization error, leading to higher accuracy compared to Post-Training Quantization (PTQ).
• Use Case: Critical for low-bit quantization (e.g., INT4) and for models where PTQ causes significant degradation.

A Calibration Dataset is a small, representative set of unlabeled data used during Post-Training Quantization (PTQ) to observe the runtime distribution of activations and calculate optimal quantization parameters.

• Purpose: Determines the scale and zero-point for activations in static quantization schemes.
• Size: Typically 100-500 samples are sufficient; it does not require labels.
• Impact on Per-Channel: For per-channel weight quantization, calibration is not strictly needed for weights (range is known statically). However, for per-channel activation quantization, calibration data is essential to find per-channel ranges.

These are two schemes defining how the quantized integer range maps to the original float range. The choice interacts with per-channel granularity.

• Symmetric Quantization: The range [-α, +α] is symmetric around zero. The zero-point is fixed at 0. This simplifies computation but can waste precision if the data distribution is not symmetric.
• Asymmetric Quantization: The range [β, γ] is not centered on zero. It uses a non-zero zero-point to precisely map the real zero value. This better captures asymmetric distributions (e.g., ReLU activations).
• Per-Channel Context: Asymmetric per-channel quantization offers the highest granularity and accuracy but requires storing and computing with a zero-point for each channel.

Quantization Granularity defines the scope over which quantization parameters are shared. It is a key axis for trading off accuracy, model size, and computational overhead.

• Levels of Granularity:
  • Per-Tensor: One scale/zero-point per tensor. (Least overhead, potentially lower accuracy).
  • Per-Channel: One scale/zero-point per channel in a weight tensor. (Higher accuracy, moderate overhead).
  • Per-Group/Per-Axis: Parameters shared across sub-groups of elements (e.g., every 64 weights).
• Metadata Cost: Finer granularity (like per-channel) increases the storage for scale/zero-point parameters, though this overhead is typically negligible compared to weight storage.