Quantization-Aware Training

QAT inserts FakeQuant operations into the model graph during training. These nodes simulate the clipping and rounding behavior of integer arithmetic while maintaining floating-point weights. This allows gradients to flow through a straight-through estimator (STE), teaching the model to adapt its weight distribution to minimize the error introduced by the final INT8 conversion.

Unlike Post-Training Quantization (PTQ), which can cause significant accuracy degradation on complex diagnostic models, QAT actively recovers lost fidelity. By exposing the network to quantization noise during training, QAT typically achieves INT8 accuracy within 0.5-1.0% of the original FP32 baseline, making it essential for life-critical medical imaging applications where diagnostic precision cannot be compromised.

During QAT, the optimal scale factor and zero-point for each quantization range are learned dynamically. The model adjusts its activation ranges to fit efficiently within the INT8 representation space. This is critical for medical images like CT scans, where Hounsfield Unit distributions vary widely across anatomy and pathology, requiring adaptive quantization ranges rather than static calibration.

Not all layers are equally sensitive to quantization. QAT workflows often include sensitivity analysis to identify which layers (e.g., the first convolutional layer or final segmentation head) require higher precision. A mixed-precision QAT strategy can keep critical layers at INT16 or FP16 while aggressively quantizing robust layers to INT8, optimizing the energy per inference on edge hardware like the NVIDIA Jetson Orin.

QAT requires a representative calibration dataset of several hundred to a few thousand examples to fine-tune quantization parameters. For medical imaging, this dataset must span the full range of expected clinical variation—different scanner vendors, acquisition protocols, and patient demographics—to prevent out-of-distribution failures when the quantized model is deployed on scanner-side hardware.

Advanced QAT frameworks like TensorRT and OpenVINO incorporate target hardware constraints directly into the training loop. This hardware-aware training simulates the specific integer arithmetic of the deployment accelerator, ensuring that the quantized model's performance is optimized for the exact NPU or FPGA architecture it will run on, rather than a generic INT8 target.

The broader technique of reducing numerical precision of weights and activations from FP32 to INT8 or INT4 to shrink memory footprint and accelerate inference. QAT is the training-time variant; PTQ is the post-training variant.

• INT8 quantization can reduce model size by 4x
• Enables deployment on integer-arithmetic-only accelerators
• Critical for meeting real-time latency budgets on edge NPUs

A model optimization paradigm that incorporates target hardware constraints—latency, power, memory bandwidth—directly into the training loop. QAT is a specific instance where the constraint is low-precision integer arithmetic.

• Uses differentiable proxies for non-differentiable hardware metrics
• Enables co-design of model architecture and deployment target
• Reduces the gap between research accuracy and production performance

A technique that uses lower-precision arithmetic (FP16, INT8) for most operations while retaining FP32 for numerically sensitive layers like the first convolution or final classification head. QAT prepares the model to tolerate this heterogeneous precision scheme.

• Balances speed improvements with minimal accuracy degradation
• Sensitive layers identified through profiling or manual specification
• Supported natively by TensorRT and ONNX Runtime

A compression method where a smaller student model is trained to replicate the behavior of a larger teacher model. Often combined with QAT: the student is trained with quantization simulation to produce a compact, integer-ready diagnostic model.

• Teacher provides soft labels with richer information than hard targets
• Student can be architecturally distinct from teacher
• Effective for preserving diagnostic accuracy in compressed models