Quantization

The core mechanism of quantization is reducing the numerical precision of model parameters. Standard training uses 32-bit floating point (FP32). Quantization maps these values to 8-bit integers (INT8) or even lower, such as 4-bit (INT4). This directly shrinks the model's memory footprint by up to 4x, enabling deployment on resource-constrained edge devices and reducing bandwidth for federated gradient transmission.

Unlike post-training quantization, QAT simulates the effects of low-precision arithmetic during the forward and backward passes of training. The model learns to adapt its weight distribution to minimize accuracy degradation caused by the quantization error. This method consistently yields higher accuracy than post-training approaches, especially for aggressive bit-widths like 4-bit quantization, making it essential for production-grade compressed models.

Quantization maps floating-point values to integers using an affine transformation defined by two parameters:

• Scale Factor (Δ): A floating-point step size that determines the resolution of the mapping.
• Zero-Point (z): An integer value that corresponds to the real value zero, ensuring that zero is exactly representable. The mapping formula is: r = S(q - Z), where r is the real value and q is the quantized integer. Proper calibration of these parameters is critical to preserving dynamic range.

Two primary mapping schemes exist:

• Symmetric Quantization: Maps the floating-point range symmetrically around zero, using a single scale factor and a zero-point fixed at 0. This simplifies hardware implementation but can waste precision if the data distribution is skewed.
• Asymmetric Quantization: Uses a non-zero zero-point to map the exact minimum and maximum of the data range. This provides better utilization of the quantized range for skewed distributions, such as those produced by ReLU activations, but adds computational overhead.

The granularity of quantization parameters significantly impacts accuracy:

• Per-Tensor Quantization: A single scale factor and zero-point are calculated for an entire weight tensor. This is fast but can cause significant error if the magnitude of weights varies drastically across channels.
• Per-Channel Quantization: Separate scale factors and zero-points are assigned to each output channel of a convolutional filter. This finer granularity preserves the distinct dynamic ranges of different filters, resulting in much higher post-quantization accuracy for vision models.

A communication efficiency technique that reduces the bandwidth required for federated updates by applying lossy compression methods such as quantization or sparsification to model gradients before transmission. This is critical in cross-device federated learning where clients may be on metered or slow connections.

• Quantization-based: Gradients mapped to low-bit representations (8-bit, 4-bit, or even 1-bit)
• Sparsification-based: Only significant gradients transmitted
• Hybrid approaches: Combine both for multiplicative compression

Modern frameworks like QSGD (Quantized SGD) provide theoretical convergence guarantees even with aggressive compression.

The divergence of local model updates from the global optimum caused by heterogeneous, non-IID data distributions across clients. Quantization can exacerbate client drift by introducing additional noise into already-divergent gradient signals.

• Statistical heterogeneity: Different label distributions across clients
• System heterogeneity: Varying compute capabilities and update frequencies
• Quantization error: Reduced precision amplifies local update variance

Mitigation strategies include FedProx (proximal regularization) and SCAFFOLD (control variates) to correct for drift before quantization is applied.

The process of performing local model optimization directly on edge hardware using locally stored data. Quantization is essential here, as edge devices (smartphones, IoT sensors, microcontrollers) have severe memory bandwidth and power constraints that make full-precision training infeasible.

• INT8 training: Uses 8-bit integers for forward and backward passes
• FP16/BF16 training: Half-precision floating point as a middle ground
• Binary/ternary networks: Extreme quantization to 1-bit or 2-bit weights

Frameworks like TensorFlow Lite and PyTorch Mobile provide built-in quantization-aware training pipelines for on-device deployment.