Compression Ratio

The most direct measurement, comparing the total number of trainable parameters before and after compression. This is a primary indicator of memory footprint reduction.

• Calculation: (Original Parameters - Compressed Parameters) / Original Parameters
• Example: A model reduced from 100M to 25M parameters achieves a 4:1 compression ratio or 75% parameter reduction.
• Significance: Directly correlates with the model's storage size and the memory bandwidth required for loading weights.

Measures the reduction in the physical file size of the serialized model, expressed in megabytes (MB) or gigabytes (GB). This is critical for storage on edge devices.

• Factors: Dictated by parameter count, numerical precision (bit-width), and the efficiency of the model format (e.g., FlatBuffer vs. protocol buffer).
• Example: Quantizing a model from FP32 (32-bit) to INT8 (8-bit) can theoretically achieve a 4x size reduction before any pruning is applied.
• Real Data: The original BERT-base model is ~440MB in FP32. A robustly quantized INT8 version can be reduced to ~110MB.

Measures the decrease in Floating Point Operations required for a single forward pass. This predicts theoretical inference speed-up and energy consumption.

• FLOPs count the multiply-accumulate (MAC) operations. Pruning and low-rank factorization directly reduce this count.
• Limitation: FLOPs are a hardware-agnostic theoretical metric. Real latency gains depend on memory bandwidth and hardware support for sparse operations.
• Use Case: A model pruned to 50% sparsity may achieve a ~2x FLOPs reduction, but the actual speed-up requires a runtime that exploits sparsity.

The most user-facing metric, measuring the reduction in inference time (e.g., milliseconds per prediction) on target hardware. This is the ultimate goal of compression for real-time applications.

• Measurement: Must be performed via on-device profiling using the final runtime (e.g., TFLite, Core ML).
• Factors: Influenced by model size (memory bandwidth), FLOPs, and hardware acceleration for quantized or sparse kernels.
• Example: A model achieving a 3x latency speed-up (e.g., from 30ms to 10ms) enables real-time video processing on a mobile phone.

A critical but often overlooked hardware-level metric. Compression reduces the volume of data transferred from memory to the compute units, which is a major bottleneck and power consumer.

• Mechanism: Smaller models (via quantization/pruning) have smaller weights, requiring fewer bytes to be loaded. Activation quantization further reduces bandwidth for intermediate results.
• Impact: Directly correlates with energy-efficient inference, as memory access often consumes more power than the actual computation.

A specialized metric for pruned models, measuring the percentage of weights or neurons that have been set to zero.

• Calculation: (Number of Zero Parameters) / (Total Parameters)
• Example: A model with a 90% sparsity ratio has 90% of its weights equal to zero.
• Key Insight: A high sparsity ratio does not guarantee proportional speed-up. Realizing the benefit requires sparse tensor formats (like CSR) and hardware with sparse compute support to skip zero operations.

The most common application of the compression ratio is to measure the reduction in model storage size. It is calculated as the size of the original model divided by the size of the compressed model. For example, a 100 MB model compressed to 25 MB has a 4:1 compression ratio (or 75% size reduction). This directly translates to faster over-the-air updates and reduced storage requirements on mobile and IoT devices.

Beyond file size, the compression ratio is applied to parameter count. Techniques like pruning and low-rank factorization remove redundant parameters. A model shrinking from 50 million to 10 million parameters achieves a 5:1 parameter compression ratio. This reduction is critical for fitting models into the limited SRAM of microcontrollers and neural processing units, where memory bandwidth is a primary bottleneck.

For latency-sensitive applications, the ratio of FLOPs (Floating Point Operations) or MACs (Multiply-Accumulate Operations) is paramount. Pruning and quantization can reduce operations by 2-10x. A model going from 3 GFLOPs to 0.5 GFLOPs has a 6:1 compute compression ratio. This metric is used to estimate real-time inference speed and power consumption on target hardware, forming the basis for latency budgets in applications like real-time object detection.

The compression ratio is never evaluated in isolation; it is plotted against model accuracy to create a tradeoff curve. This visualization reveals the Pareto frontier—the set of optimal points where no further compression can be gained without unacceptable accuracy loss. Engineers use this curve to select the operating point that meets both the deployment constraints (size/latency) and the minimum performance requirements for the application.

The achieved compression ratio directly dictates the class of hardware required for deployment. A high ratio may enable a model to run on a microcontroller instead of a mobile GPU, drastically reducing system cost and power draw. This analysis is crucial for CTOs and product architects making bill-of-materials decisions, as it determines whether an AI feature is feasible within the thermal, power, and cost envelope of a mass-market device.

Compression ratio serves as a key benchmarking metric for comparing different compression algorithms and frameworks (e.g., TensorFlow Lite, PyTorch Mobile, NVIDIA TensorRT). When combined with accuracy and latency measurements, it allows teams to objectively select the best toolchain for their specific model and target hardware. A superior tool achieves a higher compression ratio for the same accuracy drop.

The measurable decrease in a model's performance on a validation or test set after applying a compression technique like quantization or pruning. This is the primary cost of compression and is quantified using the original model's performance baseline.

• Key Metric: Typically measured as the absolute or relative reduction in validation accuracy (e.g., Top-1 or Top-5).
• Example: A model's accuracy might drop from 92.5% to 91.0% after 8-bit post-training quantization.
• Analysis: Must be evaluated against an application's degradation threshold to determine viability.

The numerical discrepancy introduced when converting continuous floating-point values (e.g., FP32) to discrete integer representations (e.g., INT8). This is a fundamental source of model degradation.

• Mechanism: Arises from the loss of precision when mapping an infinite range of float values to a finite set of integers.
• Impact: Manifests as quantization noise added to weights and activations, propagating through the network and causing output divergence.
• Mitigation: Techniques like quantization-aware training and careful calibration dataset selection aim to minimize this error.

A systematic evaluation to determine which components of a neural network are most critical for preserving accuracy when compressed. This guides targeted compression strategies.

• Targets: Can be performed per-layer, per-channel, or per-filter to measure layer-wise sensitivity.
• Process: Involves iteratively applying compression to individual components and measuring the resulting accuracy drop.
• Application: Results directly inform mixed-precision quantization and structured pruning by identifying which parts of the model require higher bit-widths or should be left untouched.

A critical accuracy recovery step where a compressed model undergoes limited additional training on a task-specific dataset. This helps the model adapt to the distortions introduced by compression.

• Purpose: To regain model fidelity and close the performance gap with the golden model.
• Process: The compressed model's weights are updated using a low learning rate, often for just a few epochs.
• Contrast with QAT: Unlike Quantization-Aware Training, this is a post-hoc correction applied after compression, not a simulated training process.

In compression-accuracy analysis, the set of optimal points on a tradeoff curve where no configuration can improve one metric (e.g., accuracy) without worsening another (e.g., model size or latency).

• Visualization: Forms the "frontier" or "optimal boundary" of a scatter plot comparing accuracy vs. a compression metric.
• Engineering Goal: To select a compressed model that lies on this frontier, representing the best possible tradeoff.
• Application: Used to compare the efficiency of different compression scheduling strategies and hardware-aware compression techniques.

The final, non-negotiable stage of tradeoff analysis where a compressed model is benchmarked on the actual target hardware (e.g., a mobile SoC or microcontroller).

• Critical Metrics: Measures real-world latency, memory footprint, power consumption, and sustained accuracy under deployment conditions.
• Purpose: Catches discrepancies between theoretical gains (e.g., reduced FLOPs) and practical performance, which can be affected by sparse model inference support and memory bandwidth.
• Outcome: Validates whether the compression ratio translates into tangible benefits for energy-efficient inference.