Model Fidelity

Kullback–Leibler (KL) Divergence is a fundamental statistical measure for quantifying the difference between two probability distributions. In model fidelity analysis, it measures the information loss when comparing the output probability distributions of the compressed model against the original 'golden' model.

• Primary Use: Best suited for classification tasks where models output probability vectors (e.g., softmax outputs).
• Interpretation: A lower KL divergence indicates higher fidelity. A value of 0 means the distributions are identical.
• Calculation: Requires running both models on the same input batch and computing (D_{KL}(P_{original} || P_{compressed})).
• Limitation: Asymmetric and can be sensitive to near-zero probabilities. Often used alongside other metrics.

Cosine Similarity measures the angular alignment between two vectors, making it ideal for comparing embedding vectors or internal activations of neural networks. It is invariant to the magnitude of the vectors, focusing purely on directional similarity.

• Primary Use: Comparing embeddings from language models, feature vectors from vision models, or layer-wise activations.
• Interpretation: Ranges from -1 (perfectly opposite) to +1 (identical direction). Values close to 1 indicate high fidelity.
• Advantage: Robust to scaling differences, which can occur with quantization. Useful for sensitivity analysis to identify which layers degrade most after compression.
• Common Practice: Often computed per layer across a calibration dataset to create a fidelity profile.

Mean Squared Error (MSE) is a direct, point-wise measure of the numerical discrepancy between the outputs of two models. It calculates the average squared difference between corresponding output tensors.

• Primary Use: Regression tasks, or any model where outputs are continuous values (e.g., bounding box coordinates, signal values). Also used to measure quantization error on a per-tensor basis.
• Interpretation: Lower MSE indicates higher fidelity. Provides a clear, absolute measure of numerical deviation.
• Context: A small MSE on logits or embeddings often correlates with minimal accuracy drop. It is a core component for calculating the Signal-to-Quantization-Noise Ratio (SQNR).
• Caveat: Can be dominated by large errors on a few samples; sometimes the Mean Absolute Error (MAE) is used for a more robust measure.

Top-k Accuracy Correlation evaluates fidelity by measuring the agreement in the final, discrete predictions between the original and compressed models. It goes beyond distributional metrics to assess functional equivalence from an application perspective.

• Primary Use: Classification tasks where the final predicted class is the critical output.
• Metrics:
  • Exact Match Rate: Percentage of inputs where the top-1 predictions are identical.
  • Top-k Agreement: Percentage where the true label from the original model's top-k predictions remains in the compressed model's top-k.
• Advantage: Directly tied to user-facing performance. A high correlation here strongly suggests the compressed model will have minimal accuracy drop.
• Analysis: A drop in agreement, especially on borderline cases, signals potential compression artifacts affecting decision boundaries.

Output Divergence is a broad, diagnostic approach that involves analyzing the differences between model outputs on a per-sample basis across a validation dataset. It helps identify systematic failure modes and corner cases.

• Primary Use: Robustness analysis to understand how and on which inputs fidelity breaks down.
• Methodology:
  1. Run both models on a validation set.
  2. Flag samples where the output difference (e.g., KL, MSE) exceeds a degradation threshold.
  3. Analyze the characteristics of these high-divergence samples (e.g., are they out-of-distribution, noisy, or semantically complex?).
• Outcome: Informs whether accuracy loss is uniform or concentrated, guiding targeted accuracy recovery efforts like fine-tuning on identified hard cases.

For specialized applications, generic statistical metrics may be insufficient. Task-Specific Fidelity Metrics are domain-aware evaluations that measure performance degradation on the exact downstream function the model serves.

• Computer Vision (Object Detection): Measure change in mean Average Precision (mAP), Intersection-over-Union (IoU), or recall rates.
• Natural Language Processing: Evaluate changes in BLEU score, ROUGE, or semantic similarity scores for generated text.
• Automatic Speech Recognition: Measure increase in Word Error Rate (WER).
• Signal Processing: Analyze degradation in Signal-to-Noise Ratio (SNR) or perceptual quality scores.
• Critical Role: These metrics define the acceptable loss for a deployment. A model may have high KL divergence but minimal change in mAP, making it viable for its specific task.

A training-time technique that simulates the effects of quantization during the forward pass, allowing the model to learn parameters robust to the precision loss. Unlike Post-Training Quantization, QAT bakes the quantization error into the optimization loop, enabling the model to adapt and recover accuracy.

• Process: Fake quantization nodes are inserted into the computational graph. During training, weights and activations are quantized and de-quantized, but gradients are calculated using the full-precision values.
• Benefit: Typically achieves higher fidelity than PTQ, especially for aggressive quantization below 8-bit.
• Trade-off: Requires retraining from scratch or fine-tuning, which adds computational cost.

A technique where a large, accurate teacher model is used to train a smaller, compressed student model. The student is trained not only on the original task labels (hard labels) but also to mimic the teacher's softened output probability distributions (soft labels).

• Mechanism: The KL divergence between the teacher and student outputs is minimized as part of the loss function.
• Application: Highly effective for creating compact models that retain the nuanced behavior of larger models, improving fidelity beyond what compression alone can achieve.
• Example: A 100M parameter student model distilled from a 1B parameter teacher can often match the teacher's accuracy more closely than a model compressed to 100M parameters via pruning/quantization alone.

A systematic method to identify which parts of a model are most sensitive to compression, guiding heterogeneous compression strategies. Not all layers contribute equally to accuracy loss.

• Process: Layer-wise sensitivity is profiled by compressing individual layers and measuring the resulting accuracy drop. Layers with high sensitivity are assigned higher precision.
• Outcome: Enables mixed-precision quantization, where critical layers (e.g., first and last) remain at 8 or 16-bit, while less sensitive intermediate layers are compressed to 4-bit. This optimizes the compression ratio while staying within an acceptable loss threshold.
• Tooling: Frameworks like NVIDIA's TensorRT and Qualcomm's AI Model Efficiency Toolkit (AIMET) provide automated sensitivity analysis.

A critical accuracy recovery step applied after a compression operation like pruning or post-training quantization. The compressed model undergoes a short, targeted training cycle on the original task dataset.

• Purpose: Allows the model's remaining parameters to adapt and compensate for the perturbations introduced by compression, reducing quantization noise and compression artifacts.
• Protocol: Uses a low learning rate and often a subset of the training data. It is less expensive than full retraining but more effective than calibration alone.
• Best Practice: Essential for achieving viable fidelity when using aggressive compression techniques. The fine-tuned compressed model is then evaluated against the golden model on a validation accuracy benchmark.

The process of selecting and preparing the data used to set quantization parameters (scales and zero-points) in Post-Training Quantization. The quality of this dataset directly impacts fidelity.

• Requirement: The calibration dataset must be representative of the model's operational data distribution. Using unrepresentative data leads to poor range estimation and high quantization error.
• Advanced Techniques:
  • Data-aware calibration: Using a larger, more diverse calibration set.
  • Cross-layer equalization: Algorithms that adjust channel ranges across layers to minimize quantization error.
• Impact: A well-chosen calibration set is the most cost-effective way to improve PTQ fidelity without any retraining.

An advanced pruning methodology that removes entire structural components (e.g., channels, filters, or blocks) rather than individual weights, followed by a potential regrowth phase. This maintains hardware-efficient structures for sparse model inference.

• Fidelity Benefit: Structured pruning creates models that are easier to fine-tune for accuracy recovery compared to unstructured, random sparsity.
• Regrowth Algorithms: Some techniques allow for the iterative pruning and selective regrowth of connections during fine-tuning, dynamically searching for an optimal sparse architecture that maximizes fidelity for a given compression ratio.
• Evaluation: The fidelity of a pruned model is assessed via output divergence metrics and robustness analysis on corner cases.

The measurable decrease in a model's performance on a validation or test set after a compression technique is applied. This is the primary cost of compression and is quantified using the same metrics as the original model evaluation (e.g., top-1 accuracy, F1 score).

• Example: A model's image classification accuracy falling from 94.2% to 92.7% after 8-bit quantization.
• The goal of accuracy recovery techniques like fine-tuning is to minimize this drop.

The numerical discrepancy introduced when converting continuous floating-point values (e.g., FP32) to discrete, lower-bit integer representations (e.g., INT8). This error acts as additive noise injected into the model's weights and activations.

• It is the fundamental source of accuracy degradation in post-training quantization.
• Techniques like quantization-aware training simulate this error during training, allowing the model to learn to compensate for it.

Kullback–Leibler divergence, a statistical measure used to quantify the difference between two probability distributions. In fidelity analysis, it measures the divergence between the output probability distributions of the original model and the compressed model.

• A lower KL divergence indicates higher model fidelity.
• Unlike simple accuracy drop, KL divergence captures changes in the model's confidence and ranking across all possible classes, not just the top prediction.

A systematic evaluation to determine which components of a neural network are most sensitive to compression. This guides heterogeneous compression strategies.

• Layer-wise sensitivity identifies which layers cause the most accuracy drop when compressed.
• Channel-wise or weight-wise sensitivity can guide structured or unstructured pruning.
• Results inform mixed-precision quantization, where critical layers retain higher bit-widths.

In compression-accuracy tradeoff analysis, the set of optimal points on a tradeoff curve where no other configuration can improve one metric (e.g., accuracy) without worsening another (e.g., model size or latency).

• Points on the frontier represent the best possible compromises.
• Engineering decisions involve selecting the specific point on this frontier that meets the application's degradation threshold and resource constraints.

The reference, high-accuracy, uncompressed model against which the performance and fidelity of all compressed variants are rigorously compared. It establishes the performance baseline.

• Serves as the source of truth for all fidelity metrics, including accuracy drop, KL divergence, and output divergence.
• Also referred to as the teacher model in the context of knowledge distillation.