Fixed-Point Arithmetic

A fixed-point number is represented as an integer scaled by an implicit, constant factor. It is defined by the tuple (Q, I, F), where:

• Q is the integer value stored in memory.
• I is the number of integer bits.
• F is the number of fractional bits.
• The total bit-width is N = I + F. The interpreted real value is calculated as: Real Value = Q * 2^{-F}. For example, with F=3 bits, the integer Q=5 represents the real value 5 * 2^{-3} = 0.625.

Fixed-point arithmetic trades dynamic range for deterministic performance and hardware simplicity.

• Precision: Floating-point offers a wide dynamic range (large exponent) with relative precision. Fixed-point offers absolute precision determined by the LSB (Least Significant Bit) value, 2^{-F}.
• Hardware: Floating-point requires complex units (FPUs) for addition and multiplication. Fixed-point uses standard, smaller, and more power-efficient integer ALUs.
• Determinism: Fixed-point operations are bit-exact and reproducible across platforms, crucial for safety-critical systems. Floating-point can have non-associative behavior due to rounding.
• Use Case: Floating-point is for training and high-precision compute. Fixed-point is for deployment-optimized, quantized inference on edge devices.

Mapping floating-point values to fixed-point integers is governed by a affine transformation:

codeCopy
Integer Q = round(Real r / Scale S) + Zero-Point Z
Real r ≈ (Q - Z) * S

• Scale (S): A floating-point number that defines the step size between consecutive integer values. S = (r_max - r_min) / (2^N - 1).
• Zero-Point (Z): An integer that maps the real value '0' precisely into the integer range. This enables efficient padding with zeros in convolutional layers.
• Symmetric vs. Asymmetric: Symmetric quantization sets Z=0, simplifying math but wasting range if data is not centered. Asymmetric quantization uses a non-zero Z for full range utilization.

Fixed-point arithmetic requires managing the scaling factors during computation.

• Addition/Subtraction: Operands must have the same scaling factor (S). If not, one must be rescaled, which involves a costly integer multiplication or shift.
• Multiplication: For Q_a = (r_a / S_a) + Z_a and Q_b = (r_b / S_b) + Z_b: Q_out = S_a S_b / S_out * (Q_a - Z_a)(Q_b - Z_b) + Z_out This requires a integer multiply followed by a rescaling (often a fixed-point multiply or shift).
• Fused Operations: Modern pipelines fuse operations like Convolution, BatchNorm, and Activation into a single integer sequence with a final requantization step, minimizing intermediate rescaling overhead.

Operating within a limited integer range requires careful handling to maintain accuracy.

• Overflow: Occurs when an operation's result exceeds the maximum representable integer (e.g., > 127 for INT8). Mitigated by careful scaling factor selection and potential saturation arithmetic.
• Underflow: Occurs when a result is smaller than the smallest representable quantum, leading to a loss of precision. This is inherent to low-precision formats.
• Rounding: Required after rescaling operations (division by a power-of-two scale). Common modes include:
  • Round to Nearest (Tie to Even): Default, unbiased.
  • Truncation: Simpler, introduces a small negative bias. Hardware often provides specific instructions for these rounding modes.

A training process where quantization error is simulated during the forward pass, allowing the model to learn parameters robust to the precision loss of fixed-point deployment. This involves:

• Injecting fake quantization nodes into the computational graph.
• Using straight-through estimators to approximate gradients for non-differentiable rounding operations.
• Typically yields higher accuracy than Post-Training Quantization (PTQ) but requires retraining resources.

An execution paradigm where all operations in a neural network—including matrix multiplications, convolutions, and activations like ReLU—are performed using integer arithmetic. This eliminates the need for power-hungry floating-point units (FPUs) on edge hardware. Key enablers include:

• Quantized activation functions (e.g., integer-friendly approximations of sigmoid).
• Fused integer layers that combine operations like convolution, batch norm, and activation into a single integer kernel.
• Essential for deployment on microcontrollers and ultra-low-power NPUs.

The process of determining the optimal scale and zero-point parameters for quantizing a model's activations. This is done by passing a representative dataset through the model and analyzing the statistical distribution (min/max, percentile) of each activation tensor. Common methods include:

• Min-Max Calibration: Uses the absolute minimum and maximum observed values.
• Percentile Calibration (e.g., 99.9%): Clips outliers for a tighter, more accurate range.
• Entropy Minimization: Searches for scale/zero-point that minimizes the information loss between float and quantized distributions.

Low-level, hand-optimized software routines that implement neural network operations (like quantized convolution) to exploit the unique features of a target processor. Optimization focuses on:

• SIMD (Single Instruction, Multiple Data) instructions (e.g., ARM NEON, Intel AVX-512) for parallel integer arithmetic.
• Memory access patterns to maximize cache utilization and minimize bandwidth bottlenecks.
• Instruction pipelining to keep the processor's execution units saturated.
• These kernels are the final, performance-critical bridge between a quantized model graph and the silicon.

Two fundamental schemes for mapping floating-point values to integers.

Symmetric Quantization:

• The quantization range is symmetric around zero (e.g., [-127, 127] for INT8).
• Zero-point is fixed at 0, simplifying the dequantization math: float_val = scale * int_val.
• Often used for weights, which are typically centered around zero.

Asymmetric Quantization:

• Uses separate scale and zero-point parameters.
• Formula: float_val = scale * (int_val - zero_point).
• Better for activations (e.g., ReLU outputs that are all non-negative), as it can precisely capture the asymmetric data range.