FLOPs Reduction

Systematically removes redundant or low-magnitude connections from a trained neural network. By zeroing out weights below a threshold, the model becomes sparse, eliminating unnecessary multiply-accumulate (MAC) operations during inference.

• Unstructured pruning removes individual weights, leading to irregular sparsity
• Structured pruning removes entire channels or filters, directly reducing tensor dimensions
• Can achieve 50-90% sparsity with minimal accuracy loss when combined with fine-tuning
• Directly maps to sparse matrix multiplication kernels on FPGA fabric

Reduces the numerical precision of weights and activations from 32-bit floating-point to lower bit-widths like INT8 or INT4. This directly cuts the cost of each MAC operation and shrinks the model's memory footprint.

• Post-Training Quantization (PTQ) applies precision reduction without retraining
• Quantization-Aware Training (QAT) simulates quantization during training for higher accuracy
• INT8 inference provides a 4x reduction in model size versus FP32
• Enables use of efficient integer arithmetic units on FPGA DSP slices

Decomposes large weight matrices into the product of two or more smaller matrices using techniques like Singular Value Decomposition (SVD). This replaces expensive full-rank matrix multiplications with cheaper low-rank operations.

• Factorizes a weight matrix W into U × V, where U and V are smaller
• Particularly effective for fully connected layers in modulation classifiers
• Reduces both FLOPs and parameter count proportionally to the chosen rank
• Can be combined with quantization for compound compression effects

Trains a compact student network to mimic the output distribution of a larger, high-accuracy teacher network. The student learns to generalize with far fewer parameters and operations.

• Student is trained on soft labels (teacher's logits) rather than hard ground truth
• Captures inter-class relationships that one-hot labels miss
• Student architectures can be 10-100x smaller than the teacher
• Enables deployment of high-accuracy modulation classifiers on FPGA fabric

Factorizes a standard convolution into a depthwise spatial convolution followed by a pointwise 1×1 convolution. This dramatically reduces the FLOPs required for convolutional layers in modulation recognition networks.

• Standard conv: D_k × D_k × M × N operations per pixel
• Depthwise separable: D_k × D_k × M + M × N operations per pixel
• Used in MobileNet architectures for efficient edge deployment
• Typical FLOPs reduction of 8-9x compared to standard convolutions

Combines multiple consecutive neural network operations into a single computational kernel at the graph level. This eliminates intermediate memory reads and writes, reducing memory bandwidth bottlenecks and kernel launch overhead.

• Fuses Conv + BatchNorm + ReLU into a single operation
• BatchNorm folding mathematically absorbs normalization parameters into preceding weights
• Reduces off-chip DRAM access, critical for FPGA streaming architectures
• A core optimization in TensorRT and Vitis AI compilation flows

A factorized convolutional operation that replaces standard convolution with two sequential layers: a depthwise convolution applying a single filter per input channel, followed by a pointwise convolution (1x1) combining outputs. This factorization reduces FLOPs by approximately 8-9x compared to standard 3x3 convolutions.

• Standard Conv FLOPs: D_k² × M × N × D_f²
• Depthwise Separable FLOPs: D_k² × M × D_f² + M × N × D_f²
• Key architecture in MobileNet and efficient RF classifiers

The systematic removal of redundant or low-magnitude connections from a neural network. Unstructured pruning zeroes out individual weights, creating sparse matrices that require specialized hardware for acceleration. Structured pruning removes entire channels or filters, directly reducing FLOPs on standard hardware.

• Magnitude-based pruning removes weights below a threshold
• Iterative pruning with fine-tuning recovers accuracy
• Can achieve 50-90% sparsity with minimal accuracy loss in modulation classifiers

A graph-level optimization that combines multiple consecutive operations into a single computational kernel. This eliminates intermediate memory reads and writes, reducing memory bandwidth bottlenecks and kernel launch overhead.

• Conv-BatchNorm-ReLU fusion: Folds BN parameters into convolution weights, then fuses activation
• Conv-Bias-ReLU fusion: Combines three operations into one kernel
• Reduces memory transactions by 3-5x for fused sequences, indirectly lowering effective FLOPs through elimination of redundant operations

A compression technique that decomposes large weight matrices W ∈ R^(m×n) into the product of two smaller matrices: W ≈ U × V, where U ∈ R^(m×r) and V ∈ R^(r×n) with rank r ≪ min(m,n). This reduces FLOPs from O(mn) to O(r(m+n)).

• SVD-based decomposition: Uses singular value decomposition for optimal low-rank approximation
• Particularly effective for fully connected layers in modulation classifiers
• FLOPs reduction factor: (m×n) / (r×(m+n))

A pre-deployment optimization that mathematically absorbs batch normalization parameters into the preceding convolutional or fully connected layer's weights and biases. This eliminates the BN computation entirely during inference.

• Fused weight: W_fused = (γ / σ) × W_original
• Fused bias: b_fused = γ × (b_original - μ) / σ + β
• Eliminates 2 multiply-adds per activation plus mean/variance lookups
• Standard practice before quantization and FPGA deployment

A visual performance model that plots attainable performance (FLOPs/s) against operational intensity (FLOPs/byte) to identify whether a neural network workload is compute-bound or memory-bound on a given hardware platform.

• Compute-bound region: Performance limited by peak FLOPs of the accelerator
• Memory-bound region: Performance limited by memory bandwidth
• Guides FLOPs reduction strategy: reduce operations in compute-bound scenarios, reduce memory traffic in memory-bound scenarios
• Critical tool for FPGA deployment optimization of modulation classifiers