Model Quantization

A one-shot compression method applied directly to a fully trained model without any retraining or fine-tuning. PTQ uses a small, unlabeled calibration dataset to estimate the dynamic ranges of activations and weights, then applies a quantization scheme.

• Mechanism: Calibrates scaling factors and zero-points by observing tensor value distributions during a forward pass.
• Granularity: Can be applied per-tensor or per-channel for higher accuracy.
• Trade-off: Extremely fast to implement but may cause significant accuracy degradation below 8-bit precision, especially on outlier-heavy models.

A technique that simulates quantization noise during the forward pass of training or fine-tuning, allowing the model to learn parameters that are robust to the precision loss. The backward pass operates on full-precision shadow weights.

• Mechanism: Inserts fake quantization nodes that round values to lower bit-widths, then passes straight-through estimators for gradient computation.
• Recovery: Recovers significant accuracy lost by PTQ, enabling INT4 and even lower bit-widths.
• Cost: Requires full access to the training pipeline and labeled data, making it computationally expensive.

An efficient one-shot weight quantization method based on approximate second-order information. GPTQ quantizes weights layer-by-layer, solving a reconstruction problem to minimize the error between the original and quantized layer outputs.

• Algorithm: Uses the Optimal Brain Quantizer framework, adjusting remaining unquantized weights to compensate for the error introduced by quantizing a single weight.
• Performance: Can quantize a 175B parameter model to 4-bit precision in approximately 4 GPU hours.
• Hardware: Optimized for fast GPU inference, often paired with custom kernels for dequantization.

A weight-only quantization method that identifies and protects salient weights based on activation magnitudes, rather than weight magnitudes alone. AWQ observes that weights corresponding to large activation channels are disproportionately important.

• Mechanism: Applies per-channel scaling to salient weights before quantization to minimize their relative error, effectively shifting the quantization burden to less important weights.
• Efficiency: Does not require backpropagation or reconstruction, making it faster than GPTQ.
• Deployment: Designed for efficient execution with TinyChat, a dedicated inference runtime.

Two fundamental strategies for quantizing activations. Dynamic quantization computes scaling factors on-the-fly for each input during inference, while static quantization pre-computes and freezes scaling factors using a calibration dataset.

• Dynamic: Lower latency for compute-bound models but introduces runtime overhead for range calculation. Ideal for RNNs and LSTMs.
• Static: Faster inference because scaling is baked in, but requires a representative calibration set and can be brittle to out-of-distribution inputs.
• Application: Dynamic is common in CPU inference; static is standard for GPU and edge TPU deployments.

A quantization technique that addresses the activation outlier problem in large language models by mathematically migrating the quantization difficulty from activations to weights. LLMs exhibit systematic outliers in specific activation channels that make standard INT8 quantization fail.

• Mechanism: Introduces a per-channel smoothing factor that scales activations down and weights up proportionally, making both tensors easier to quantize.
• Result: Enables W8A8 (8-bit weight and activation) quantization of models up to 530B parameters without accuracy loss.
• Hardware: Achieves 1.5x speedup and 2x memory reduction on commercial GPUs.