Scaling Factor

A scaling factor (often denoted as S or scale) is a floating-point value calculated as: S = (float_max - float_min) / (quant_max - quant_min). It defines the linear mapping relationship: float_value ≈ S * (quantized_int - zero_point). For symmetric quantization, where float_min = -float_max, this simplifies to S = float_max / quant_max.

The scaling factor enables integer-only inference by allowing all computations to occur in the quantized domain. During operations like matrix multiplication, the integer results are rescaled using the product or quotient of the input and weight scaling factors. This avoids floating-point operations entirely on hardware lacking an FPU, using only efficient integer arithmetic and fixed-point rescaling.

• Example: For a fully connected layer, the output scaling factor is often S_out = (S_weights * S_input) / S_output.

The value of the scaling factor directly controls the resolution of the quantization. A larger scale maps a wider range of float values to the same integer range, reducing precision and increasing quantization error. A smaller scale offers higher precision but risks clipping if the float range exceeds the quantized representation. Optimal calibration finds the scale that minimizes the mean squared error (MSE) between original and dequantized values.

The granularity of scaling factor application is a critical optimization:

• Per-Tensor: A single scale (and zero-point) is used for an entire tensor. This is simple but can be suboptimal if the tensor's distribution varies significantly.
• Per-Channel (or Per-Axis): Each channel (e.g., each output channel of a convolutional filter) has its own independent scale and zero-point. This captures intra-tensor variation more accurately, often leading to higher post-quantization accuracy, especially for weights.

In asymmetric quantization, the scaling factor works in tandem with the zero-point (Z). The zero-point is the integer value that maps to the real zero, allowing exact representation of zero (crucial for padding and ReLU activations). The dequantization formula becomes real_value = S * (int_value - Z). The scale and zero-point are determined together during calibration to minimize information loss across the asymmetric data range.

While scaling factors are essential, they introduce overhead. Each factor is a stored float32 value (4 bytes). In a per-channel scheme, this adds 4 * num_channels bytes to the model's flash footprint. During inference, scaling operations require fixed-point integer multiplication and bit-shifts. Optimized kernels (e.g., CMSIS-NN) pre-compute and fuse these scaling operations into adjacent layers to minimize runtime cost. The total overhead is typically negligible compared to the memory savings from INT8 weights and activations.

Quantization is the overarching model compression technique that reduces the numerical precision of a neural network's weights and activations, typically from 32-bit floating-point (FP32) to lower-bit integers like INT8 or INT4. The scaling factor is the critical parameter derived during quantization that defines the linear mapping between the original floating-point range and the quantized integer range. This process decreases model size, memory bandwidth requirements, and enables the use of efficient integer arithmetic units ubiquitous in microcontrollers.

The zero-point (z) is the integer value in asymmetric quantization that corresponds to the real value of zero. It works in conjunction with the scaling factor (s) in the quantization formula: real_value = s * (quantized_value - z). Its primary functions are:

• Enabling exact representation of zero, which is crucial for efficient padding operations in convolutional layers.
• Allowing the quantization range to be asymmetric, which better captures the distribution of data like ReLU activations (which have a minimum of zero but no fixed maximum).

Calibration is the data-driven process used to determine the optimal quantization parameters, specifically the scaling factor and zero-point. It involves running a representative dataset (the calibration set) through the pre-trained floating-point model to collect statistics:

• For weights, the min/max values are typically known directly.
• For activations, the dynamic range is observed by tracking the minimum and maximum values that appear at each layer's output. These observed ranges are then used to calculate the layer-specific scaling factors, making calibration a critical step for maintaining model accuracy after quantization.

Integer-only inference is the execution paradigm where the entire neural network forward pass is performed using integer arithmetic, with no floating-point operations. This is enabled by quantization where all weights and activations are integers, and the scaling factor math is also implemented using fixed-point integer operations. Benefits for microcontrollers include:

• Elimination of power-hungry and area-intensive FPUs (Floating-Point Units).
• Deterministic, faster execution on standard integer ALUs.
• Simplified deployment to the vast majority of MCUs that lack hardware FP support.

Dequantization is the reverse operation of quantization, converting quantized integer values back into floating-point numbers using the formula: float_value = scale * (int_value - zero_point). In microcontroller inference, its role is nuanced:

• During Calibration: Used to simulate quantization error and calculate updated statistics.
• At Model Output: Often required to convert the final integer tensor back to a human-interpretable floating-point value (e.g., a classification score or regression value).
• Fused Operations: In many optimized kernels, dequantization is fused with a subsequent floating-point operation or avoided entirely through rescaling techniques to maintain an integer pipeline.

Fixed-point arithmetic is a numerical representation system used to perform calculations with fractional numbers using only integer hardware. A fixed-point number is essentially an integer with an implied radix point (like a decimal point). It is closely related to scaling factor usage:

• The scaling factor in quantization can be viewed as establishing a fixed-point format for each tensor.
• During integer-only inference, operations involving scaling factors (e.g., requantization) are implemented using fixed-point integer multiplies and bit-shifts, avoiding floating-point math entirely. This is foundational for efficient MCU deployment.