Dequantization

Dequantization is essential during the quantization calibration phase. Engineers run a representative dataset through the model to collect activation statistics. By dequantizing intermediate integer tensors, they can analyze the dynamic range and distribution of values to determine optimal scale and zero-point parameters for minimal accuracy loss. This process validates that the chosen integer bit-width (e.g., INT8) adequately represents the model's numerical behavior.

For many microcontroller applications, the final model output requires floating-point precision for meaningful interpretation. A quantized model might produce an INT8 tensor, but the application needs a probability distribution, a regression value, or a bounding box coordinate. Dequantization of the final layer's output converts these integers back to FP32, enabling correct post-processing. For example, an object detection model's final INT8 outputs are dequantized to obtain precise FP32 coordinates for a display or control system.

Dequantization enables layer-wise debugging and accuracy analysis in integer-only inference pipelines. By selectively dequantizing the output of specific layers, engineers can compare intermediate results against a golden reference from the original FP32 model. This is critical for:

• Isolating which quantized layer introduces the most error.
• Verifying the correctness of custom integer kernels.
• Diagnosing overflow or saturation artifacts that occur when values exceed the representable integer range.

Not all layers in a TinyML model are equally sensitive to precision loss. Hybrid quantization uses different precisions per layer (e.g., INT8 for convolutions, FP32 for attention). Dequantization acts as the bridge between these domains. The output of an INT8 convolutional block is dequantized to FP32 before being fed into a subsequent FP32 layer. This allows for strategic precision allocation, preserving accuracy in critical layers while maintaining efficiency elsewhere.

Many essential algorithms that follow neural network inference are not designed for integer math. Dequantization provides the floating-point inputs required for:

• Non-Maximum Suppression (NMS) in object detection.
• Applying a softmax function to logits for classification.
• Digital Signal Processing (DSP) filters on model outputs. Running these algorithms directly on quantized integers would produce incorrect results. Dequantization is the necessary format conversion before these domain-specific processing steps.

Emerging on-device learning techniques for microcontrollers may involve fine-tuning a quantized model with new sensor data. The forward pass uses integer quantization for efficiency. However, computing gradients for the backward pass often requires higher precision. Dequantizing activations to a floating-point representation (even a narrow one like FP16) enables stable gradient calculation and weight updates, bridging the gap between efficient inference and adaptive learning.

Quantization is the foundational model compression technique that enables dequantization. It reduces the numerical precision of a neural network's weights and activations from 32-bit floating-point (FP32) to lower-bit integers (e.g., INT8, INT4). This process decreases model size, memory bandwidth requirements, and enables the use of efficient integer arithmetic units ubiquitous in microcontrollers. The primary goal is to maintain model accuracy while achieving drastic performance gains.

• Core Process: Maps a continuous range of floating-point values to a finite set of integers.
• Trade-off: Introduces a precision loss that dequantization aims to partially reverse for interpretation.

Integer quantization is the specific form of quantization where all model parameters and layer activations are constrained to integer values (e.g., int8_t in C). This is the direct prerequisite for dequantization. It enables highly efficient inference using low-power, integer-only arithmetic logic units (ALUs) found in all microcontrollers, as opposed to requiring rare and power-hungry floating-point units (FPUs).

• Hardware Alignment: Directly maps to the native data types and operations of Cortex-M and other MCU cores.
• Dequantization Context: The integer tensors produced by this process are the inputs to the dequantization operation.

Calibration is the statistical analysis phase that determines the parameters needed for both quantization and dequantization. A representative dataset is passed through the pre-trained FP32 model to collect the dynamic ranges (min/max) of activation tensors for each layer. These statistics are used to calculate the scaling factor and zero-point for each tensor.

• Purpose: Informs the quantization mapping and provides the constants required for accurate dequantization.
• Output: A set of layer-specific (or channel-specific) scale (s) and zero-point (z) values stored with the quantized model.

These are the two critical parameters that define the linear mapping between integer and floating-point spaces, used in both quantization and dequantization.

• Scaling Factor (Scale): A floating-point value s = (float_max - float_min) / (quant_max - quant_min). It defines the resolution of the quantization.
• Zero-Point (z): An integer that represents the quantized value corresponding to the real number zero. It ensures efficient padding with zeros and handles asymmetric ranges.

Dequantization Formula: The core operation is typically float_value = scale * (int_value - zero_point). These parameters are embedded in the model metadata and used at runtime.

These are the two primary schemes defining the quantization range, which directly impacts the dequantization math and efficiency.

• Symmetric Quantization: The quantization range is symmetric around zero (e.g., [-127, 127] for INT8). The zero-point is fixed at 0, simplifying the dequantization formula to float_value = scale * int_value. This is commonly used for weights.
• Asymmetric Quantization: The range is defined by actual min/max values, leading to a non-zero zero-point. This better captures skewed data (e.g., ReLU activations, which are all >=0) but adds a subtraction in the dequantization step. The formula is float_value = scale * (int_value - zero_point).

INT8 inference is the execution paradigm where dequantization is most frequently applied. It refers to running a neural network where weights and activations are represented as 8-bit integers. This offers a 4x reduction in model size and memory bandwidth compared to FP32 and can leverage highly optimized integer SIMD instructions on modern microcontrollers.

• Performance Target: The primary efficiency goal for many TinyML deployments.
• Dequantization Role: While the core inference (convolution, matrix multiplies) uses INT8, the final layer outputs or specific intermediate results may be dequantized to FP32 for accurate post-processing (e.g., softmax, non-quantizable operations).