Robustness Analysis

Compressed models often exhibit increased susceptibility to adversarial examples—inputs with small, imperceptible perturbations designed to cause misclassification. Quantization and pruning can remove the model's natural noise robustness, making decision boundaries more brittle. For example, an 8-bit quantized vision model may fail on adversarial images that the original 32-bit FP model correctly classifies. This is a critical failure mode for security-sensitive applications like authentication or fraud detection.

Performance on out-of-distribution (OOD) data often degrades more severely in compressed models than on in-distribution test sets. The loss of parameter precision and network capacity reduces a model's ability to generalize to novel inputs. Key manifestations include:

• Severe accuracy drop on data from a different sensor, lighting condition, or geographic location than the training set.
• Catastrophic failure on corner cases (e.g., rare objects, extreme weather in autonomous driving).
• This failure mode is often missed by standard validation, which uses in-distribution data, necessitating dedicated OOD testing suites.

Low-precision quantization, especially below 8-bit, introduces numerical instability in specific layers. This occurs when the reduced dynamic range cannot accommodate the actual range of activation values, leading to saturation (clipping) or overflow. Consequences include:

• Activation collapse where intermediate feature maps become zero or uniform, destroying information flow.
• Exploding/vanishing gradients during post-compression fine-tuning, hindering recovery.
• Layer normalization and attention layers in transformers are particularly prone. This failure is hardware-dependent, as different NPUs may handle saturation differently.

Compression frequently disrupts a model's calibration—the alignment between its predicted confidence scores and its actual likelihood of being correct. A compressed model may become overconfident in its (often incorrect) predictions. This failure mode is dangerous for human-in-the-loop systems, as high-confidence errors erode trust. It is measured using metrics like Expected Calibration Error (ECE). For instance, a pruned medical diagnostic model might report 99% confidence on a misdiagnosis, whereas the original model was more uncertain.

Compression can amplify existing biases or introduce new ones. Pruning may disproportionately remove neurons responsible for recognizing underrepresented subpopulations in the training data. Quantization can cause higher error rates on specific demographic groups if their data distribution lies in a sensitive region of the activation space. This failure mode requires disaggregated evaluation across protected attributes to detect, as overall accuracy may mask significant performance disparities.

For models processing sequential data (e.g., audio, video, time-series), compression can induce temporal inconsistency. Errors may compound over time, or the model may produce flickering predictions (e.g., an object in video detection appearing and disappearing frame-to-frame). Recurrent layers and attention mechanisms in compressed models are vulnerable. This is critical for real-time applications like speech recognition on devices or video analytics, where smooth, coherent outputs are required.

A model's ability to maintain performance on data that differs statistically from its training distribution. Compression can amplify sensitivity to OOD inputs.

• Key Concern: Quantized or pruned models often lose the generalization capacity to handle novel inputs, leading to sharp accuracy cliffs.
• Evaluation: Test with datasets that have covariate shift (e.g., ImageNet-C for corruptions) or semantic shift.
• Example: A quantized object detector trained on daytime imagery may fail catastrophically on low-light or foggy scenes, whereas the full-precision model degrades more gracefully.

A model's resilience to intentionally crafted, malicious inputs designed to cause misclassification. Compression frequently reduces adversarial robustness.

• Mechanism: Quantization noise and removed parameters can eliminate subtle feature defenses, making compressed models more vulnerable to low-perturbation attacks like FGSM or PGD.
• Critical for: Security applications, autonomous systems, and any deployment facing potential adversarial environments.
• Analysis Metric: Measure the drop in Adversarial Accuracy (accuracy on attacked samples) post-compression versus the baseline model.

Performance on rare, unusual, or extreme input scenarios that are underrepresented in training data. Essential for safety-critical applications.

• Focus: Not just average accuracy, but the worst-case performance. Compression can disproportionately affect these edge cases.
• Evaluation Method: Use specialized test suites or stress test datasets that aggregate challenging scenarios (e.g., small/occluded objects, rare weather conditions).
• Engineering Implication: Defines the acceptable loss threshold; a 2% average accuracy drop may hide a 40% drop on critical corner cases.

The degree to which a model's predicted probability (confidence score) reflects its true likelihood of being correct. Poor calibration is a common robustness failure after compression.

• Problem: A quantized model may become overconfident, outputting high confidence for incorrect predictions, which breaks downstream decision logic.
• Metric: Measured via Expected Calibration Error (ECE) or reliability diagrams.
• Mitigation: Temperature scaling or other post-hoc calibration is often required after compression to restore trustworthy confidence estimates.

The systematic categorization and investigation of how and why a compressed model fails, beyond aggregate accuracy metrics.

• Process: 1. Identify error clusters (e.g., all failures on a specific class). 2. Analyze input characteristics for those failures. 3. Trace back to which compression operation (e.g., pruning of specific filters) contributed.
• Tool: Uses confusion matrices, gradient-based sensitivity maps, and visualization of activations in compressed vs. original models.
• Outcome: Informs targeted accuracy recovery steps, such as protecting sensitive layers from aggressive compression.

The phenomenon where the data encountered during deployment differs from the data used for training and calibrating the compressed model.

• Types: Covariate Shift (input distribution changes), Label Shift (output distribution changes), and Concept Drift (relationship between input and output changes).
• Robustness Test: Apply compression and then evaluate on datasets simulating real-world shift (e.g., new user demographics, sensor degradation).
• Link to Compression: Calibration for quantization is done on a static dataset; if the input distribution shifts, the quantization ranges may become suboptimal, accelerating performance decay.