Structured Pruning

Removes entire convolutional channels or feature maps from a layer, directly reducing the width of the network. Unlike unstructured pruning that zeros out individual weights, channel pruning eliminates both the filters and their corresponding output activations.

• Produces a thinner but complete network architecture
• Immediately reduces FLOPs and memory footprint without sparse matrix overhead
• Typically guided by importance scoring metrics like L1-norm of filter weights
• Example: Pruning 30% of channels in ResNet-50 can reduce parameters by 50%+ with minimal accuracy loss

Operates at the filter level within convolutional layers, removing entire 3D filter kernels. Each pruned filter eliminates one output feature map and cascades to remove corresponding input channels in the subsequent layer.

• Maintains dense tensor operations compatible with standard BLAS libraries
• Enables direct hardware acceleration on CPUs, GPUs, and NPUs without custom sparse kernels
• Pruning decisions propagate forward through the network, creating a dependency chain
• Contrast with unstructured pruning: no need for specialized sparse matrix multiplication hardware

Eliminates entire transformer blocks, residual blocks, or dense layers from deep networks. This coarse-grained approach is particularly effective for over-parameterized models like large language models and deep vision transformers.

• Reduces sequential computation depth, directly lowering inference latency
• Preserves the dense computation pattern of remaining layers
• Often guided by layer importance scoring using Taylor expansion or learned gating mechanisms
• Example: Removing alternating transformer blocks in BERT-base can achieve 2x speedup with <1% accuracy degradation on downstream tasks

Selection of which structures to prune is guided by target hardware constraints rather than purely mathematical importance metrics. This ensures the compressed model maps efficiently to the deployment silicon.

• Latency-constrained pruning: removes structures that contribute most to inference time on the target NPU or CPU
• Memory-budget pruning: targets structures to fit within SRAM or tightly-coupled memory limits
• Energy-aware pruning: prioritizes removal of high-energy operations for battery-operated medical devices
• Uses hardware-in-the-loop feedback during the pruning process to validate real-world speedups

Structured pruning is typically performed iteratively rather than in a single pass. Each pruning step removes a small percentage of structures, followed by fine-tuning to recover accuracy before the next pruning iteration.

• Gradual magnitude decay: progressively reduces the importance of targeted structures during training
• Alternating prune-retrain cycles: prevents catastrophic accuracy collapse from aggressive one-shot pruning
• Learning rate rewinding: resets learning rate schedule after each prune step for optimal recovery
• Typical schedule: prune 10-20% of structures per iteration across 5-10 cycles until target compression is reached

The defining advantage of structured pruning is that it produces a dense, standard-format model that requires no specialized runtime or sparse computation library. The pruned model is simply a smaller version of the original architecture.

• Directly exportable to ONNX, TensorFlow Lite, or Core ML formats
• Compatible with all standard quantization toolchains for further compression
• No dependency on sparse BLAS or custom hardware sparsity engines
• Enables seamless deployment on FDA-cleared medical edge devices with locked-down software stacks
• Critical for regulated environments where runtime modifications require re-certification

Removes individual weights by zeroing out connections below a magnitude threshold, creating a sparse matrix. Unlike structured pruning, this produces irregular sparsity patterns that require specialized sparse hardware or software libraries to accelerate. While it can achieve higher compression ratios with less accuracy loss, the resulting sparse tensors are not natively accelerated by standard edge NPUs or GPUs without dedicated sparse compute support.

Removes entire convolutional filters (kernels) from a layer, which eliminates the corresponding output feature maps. This is geometrically equivalent to removing rows from the weight tensor. Filter pruning is typically guided by:

• L1-norm or L2-norm of filter weights
• Taylor expansion of the loss function
• Geometric median to remove redundant filters near the center of the filter distribution

The result is a thinner network that runs efficiently on commodity hardware.

An automated methodology that discovers optimal network architectures under predefined constraints like latency, memory, or FLOPs. Hardware-aware NAS directly incorporates the target edge accelerator's characteristics into the search objective. Unlike post-hoc pruning, NAS builds efficient architectures from scratch, often discovering non-intuitive layer configurations that outperform manually pruned models on medical imaging tasks.

A complementary compression technique that reduces the numerical precision of weights and activations after training, typically from FP32 to INT8. When combined with structured pruning, the effects are multiplicative: pruning reduces the number of operations, while quantization reduces the cost per operation. Together they can achieve 10-20x total model footprint reduction for deployment on medical wearables with NPU acceleration.