Zero-Point

The zero-point (Z) is the integer value in a quantized representation that corresponds exactly to the real floating-point value of zero. Its primary purpose is to allow for efficient padding with zeros in operations like convolution and to accurately represent data ranges that are not symmetric around zero (e.g., ReLU activations which are always ≥ 0).

• Mathematical Role: It acts as the 'bias' in the affine mapping equation: real_value = scale * (quantized_value - zero_point).
• Hardware Benefit: On microcontrollers, an integer zero-point allows the hardware to use fast, energy-efficient integer arithmetic for operations that would otherwise require costly floating-point logic.

The zero-point is the defining feature that distinguishes asymmetric quantization from symmetric quantization.

• Asymmetric Quantization: Uses a non-zero zero-point to map a floating-point range [min, max] to an integer range [0, 255] for INT8. This captures the full dynamic range of asymmetric data.
• Symmetric Quantization: Forces the zero-point to be 0. It maps the range [-max, max] to [-127, 127]. This simplifies computation but wastes integer bins if the data is not centered on zero.

Key Trade-off: Asymmetric (with zero-point) provides better accuracy for skewed data but adds a subtraction operation. Symmetric is computationally simpler but can lose precision.

The quantization process is defined by a scale (S) and zero-point (Z).

Quantization: q = round(r / S) + Z Dequantization: r = S * (q - Z)

Where:

• r is the real (float) value.
• q is the quantized integer value.
• S (scale) is a positive floating-point number: S = (r_max - r_min) / (q_max - q_min).
• Z (zero-point) is an integer: Z = round(q_max - r_max / S) or Z = round(-r_min / S).

Example: For ReLU activations with r_min = 0.0, r_max = 6.0, quantized to INT8 [0, 255]:

• S = 6.0 / 255 ≈ 0.02353
• Z = round(0 - 0.0 / S) = 0 Here, the zero-point is 0, but it is still part of the asymmetric scheme.

On resource-constrained devices, the zero-point enables critical optimizations:

• Efficient Zero Padding: In convolutional neural networks (CNNs), padding layers with zeros is common. With a zero-point of 0, padding becomes a trivial memory set operation. If the zero-point is non-zero, padding requires filling the tensor with that specific integer value, adding minor overhead.
• Kernel Optimization: Libraries like CMSIS-NN and TensorFlow Lite Micro have hand-optimized kernels that pre-subtract the zero-point from weights during compilation, folding the cost into the weight tensor. This minimizes the runtime overhead of the (q - Z) operation.
• Integer-Only Pipelines: The presence of a zero-point allows the entire inference graph—from input to output—to be executed using integer arithmetic, avoiding any floating-point operations on the microcontroller.

The zero-point is not chosen arbitrarily; it is calibrated using a representative dataset.

Process during Post-Training Quantization (PTQ):

1. Run the floating-point model on a calibration dataset.
2. For each layer's activation tensor, collect the observed minimum (r_min) and maximum (r_max) floating-point values.
3. Calculate the scale S and zero-point Z using the formulas above, clamping Z to the target integer range (e.g., 0 to 255 for uint8).

Considerations:

• If r_min and r_max are symmetric, Z will be near 0.
• For weights, symmetric quantization (Z=0) is often used for simplicity, while activations use asymmetric quantization with a calibrated zero-point.

The zero-point interacts closely with other microcontroller inference techniques:

• Fixed-Point Arithmetic: Zero-point quantization is a form of affine fixed-point representation, where the scale is a power-of-two for hardware efficiency.
• Operator Fusion: When fusing a convolution with a subsequent addition (bias), the bias term must be adjusted to account for the zero-points of the input and weight tensors.
• Static Memory Allocation: Knowing the zero-point at compile-time allows for the pre-calculation of all scaled constants, enabling fully static execution graphs.
• INT8 Inference: The predominant use case for zero-point is in 8-bit integer inference, where it balances precision and computational efficiency on MCUs.