Output Divergence

The primary source of divergence in post-training quantization (PTQ). It is the numerical discrepancy introduced when mapping continuous, high-precision floating-point values (e.g., FP32) to a discrete, lower-precision integer representation (e.g., INT8). This process is inherently lossy.

• Mechanism: A quantization function Q(x) = round(x / scale) + zero_point approximates the original value x. The round operation and limited integer range cause rounding error and clipping error.
• Impact: This error propagates through the computational graph, causing small perturbations in activations that can compound across layers, leading to altered final logits and predictions.
• Example: A weight value of 0.317 in FP32 might be quantized to an integer representing 0.3125, introducing a small error that affects the dot product in which it participates.

Divergence caused by the physical removal of network parameters or structures. Pruning eliminates weights, filters, or neurons deemed less important, altering the model's forward pass function.

• Unstructured Pruning: Removes individual weights, creating an irregular, sparse connectivity pattern. This can disrupt subtle, distributed feature representations learned by the original dense network.
• Structured Pruning: Removes entire channels or filters, which is more hardware-friendly but can aggressively remove entire feature detectors, potentially crippling the model's ability to recognize specific patterns.
• Critical Failure: If pruning heuristics (e.g., magnitude-based) incorrectly identify a functionally important weight as unimportant, the resulting structural damage can be severe and non-recoverable without retraining.

Divergence arising from matrix factorization techniques like low-rank approximation. These methods decompose a large weight matrix W (m x n) into the product of smaller matrices, e.g., W ≈ U (m x r) * V (r x n), where r (rank) is much smaller than n or m.

• Mechanism: The low-rank matrices U and V provide a compressed, approximate representation of the original parameter space. The approximation error is the difference W - (U*V).
• Consequence: This forces the model to operate in a reduced-dimensional subspace, limiting its representational capacity. The compressed model can only express a subset of the transformations possible in the original, higher-rank weight matrix, leading to divergent outputs for inputs that relied on the full capacity.

A secondary, compounding cause of divergence. Compression changes the statistical distribution of layer activations, which can break implicit assumptions in the original model's design.

• Batch Normalization Mismatch: PTQ calibrates quantization ranges using a static calibration dataset. If the runtime activation distribution differs from this calibration set, quantization becomes suboptimal, increasing error.
• Non-Linear Function Distortion: Quantization applied before or after non-linear activation functions (e.g., ReLU, GeLU) can change the function's effective shape. For example, quantizing very small positive values to zero can prematurely 'kill' a ReLU activation.
• Cascading Error: Small shifts in early-layer activations are amplified by subsequent non-linearities, leading to large divergences in deeper layers.

Divergence due to the removal of redundant or seemingly parallel computational pathways that actually contributed to model robustness. Modern over-parameterized networks often behave as implicit ensembles.

• Path Redundancy: Original dense networks have multiple, slightly different internal pathways to compute a similar function. Pruning or aggressive quantization collapses these into a single, dominant path, reducing the model's ability to handle varied inputs gracefully.
• Loss of Noise Robustness: The numerous parameters in a large model can provide a smoothing, averaging effect. Compression simplifies this landscape, making the model's output more sensitive to specific input perturbations.
• Example: A pruned vision model might lose alternate edge detectors, making it more susceptible to adversarial noise or occlusion that the original model could handle via its redundant pathways.

In knowledge distillation, divergence is an expected outcome of the student model's limited capacity. The student learns to approximate the teacher's behavior, not replicate it exactly.

• Representation Gap: The smaller student model lacks the parameters to perfectly mimic the teacher's complex function mapping, especially for outlier or difficult samples.
• Soft Target Imperfection: While learning from the teacher's softened class probabilities (soft targets) is more informative than hard labels, the student's final decision boundary will inevitably differ.
• Optimization Challenge: The distillation loss (e.g., KL divergence) is minimized, not zero. The remaining loss represents the fundamental, irreducible divergence between the teacher's knowledge and the student's ability to absorb it.

Model fidelity quantifies the degree to which a compressed model's behavior replicates that of the original, uncompressed model. It is a broader concept than simple accuracy drop, encompassing the similarity of internal activations, attention patterns, and output distributions.

• Primary Metric: Often measured using statistical divergences like KL Divergence or Cosine Similarity between output logits or embeddings.
• Critical Use: Essential for applications where the exact behavior of the model must be preserved, not just its final accuracy score on a test set.

Quantization error is the numerical discrepancy introduced when converting continuous, high-precision values (e.g., FP32) to discrete, lower-precision representations (e.g., INT8). This is a primary, measurable source of output divergence.

• Mechanism: The error acts as additive noise injected into weights and activations, propagating through the computational graph.
• Types: Includes rounding error and clipping error (when values outside the quantizable range are saturated).
• Management: Techniques like quantization-aware training (QAT) simulate this error during training, allowing the model to learn robust parameters.

Compression artifacts are predictable, undesirable distortions or errors in a model's output directly caused by the lossy nature of compression. They are the concrete manifestations of output divergence.

• Examples in Vision: A pruned image classifier might consistently misclassify fine-textured patterns; a heavily quantized generative model may produce blurry or speckled outputs.
• Examples in NLP: A distilled language model may produce less nuanced or more repetitive text.
• Analysis: Identifying systematic artifacts is key to diagnosing which compression technique or layer is causing specific divergence.

Sensitivity analysis is a systematic methodology to identify which components of a neural network are most vulnerable to compression, guiding strategies to minimize output divergence.

• Process: Measures the impact on accuracy or fidelity when compressing individual layers, channels, or attention heads.
• Outcome: Produces a sensitivity profile, enabling mixed-precision quantization (applying higher precision to sensitive layers) or structured pruning (avoiding pruning critical filters).
• Tool: Fundamental for moving from uniform, global compression to heterogeneous, optimized compression schedules.

Accuracy recovery refers to the process of regaining performance lost due to compression (output divergence) through post-compression training techniques. It is the active mitigation phase.

• Primary Technique: Fine-tuning after compression, where the compressed model is trained for a few epochs on the task dataset.
• Advanced Methods: Quantization-aware training (QAT) and Distillation from the original teacher are more integrated recovery processes.
• Goal: Not just to recover baseline accuracy, but to do so efficiently without negating the compression benefits.

Kullback–Leibler Divergence (KL Divergence) is a fundamental statistical metric used to measure output divergence by quantifying the difference between two probability distributions.

• Application in Compression: Compares the output probability distribution (e.g., softmax logits) of the compressed model against the original model's distribution for the same input.
• Interpretation: A lower KL divergence indicates higher model fidelity; the compressed model's "beliefs" are closer to the original's.
• Advantage over Accuracy: Captures the magnitude of uncertainty and distributional shift, even on correctly classified samples.