Grouped Convolution

In a standard convolution, a filter with dimensions (K \times K \times C_{in}) is applied across all input channels (C_{in}) to produce one output channel. Grouped convolution introduces a group parameter (G). The input channels are divided into (G) distinct groups, each containing (C_{in}/G) channels. A separate set of filters, with dimensions (K \times K \times (C_{in}/G)), is applied to each group. The outputs from all groups are concatenated along the channel dimension.

• Key Formula: For group count (G), the parameter count is reduced by a factor of (1/G) compared to a standard convolution with the same kernel size and input/output dimensions.
• Constraint: (C_{in}) and (C_{out}) must typically be divisible by (G).

The primary advantage of grouped convolution is its drastic reduction in computational cost and parameter count, which is critical for TinyML.

• Standard Convolution FLOPs: ( \approx K \times K \times C_{in} \times C_{out} \times H_{out} \times W_{out} )
• Grouped Convolution FLOPs: ( \approx (K \times K \times (C_{in}/G) \times (C_{out}/G) \times H_{out} \times W_{out}) \times G = (1/G) \times \text{Standard FLOPs} )
• Real-World Impact: For a 3x3 convolution with 128 input/output channels and (G=4), grouped convolution uses 75% fewer parameters and computations. This directly translates to lower latency and SRAM usage on microcontrollers.

Grouped convolution enforces structured sparsity in the connectivity between input and output channels. Channels in one group do not interact with channels in other groups during the convolution. This design choice has significant architectural implications:

• Benefit: It creates efficient, parallelizable computation paths, reducing memory bandwidth requirements.
• Trade-off: It limits cross-channel feature learning. A filter in Group A cannot combine information from an input channel in Group B. This can hinder the model's ability to learn complex, cross-correlated features, potentially impacting accuracy for tasks requiring rich channel interactions.
• Use Case: It is highly effective in early or late layers of a network where learning independent channel-wise features is sufficient, or in extremely wide networks where maintaining some channel interaction within large groups is still possible.

Depthwise separable convolution, a cornerstone of MobileNet, is a special and extreme case of grouped convolution.

• Depthwise Convolution: This is a grouped convolution where the number of groups (G) is set equal to the number of input channels (C_{in}). Each input channel gets exactly one filter, resulting in maximum parameter savings but zero cross-channel interaction at this layer.
• Pointwise Convolution: The depthwise operation is always followed by a 1x1 (pointwise) convolution, which is a standard convolution that fuses the channel-wise features.
• Comparison: Standard Grouped Convolution (e.g., (G=4) or (8)) offers a flexible middle ground between the extreme sparsity of depthwise convolution and the dense connectivity of standard convolution, allowing designers to tune the efficiency/accuracy trade-off.

Grouped convolution is a fundamental building block in several landmark efficient neural networks for embedded systems:

• ShuffleNet: Uses pointwise group convolutions extensively to reduce the cost of 1x1 convolutions. To mitigate the isolation of channel groups, it introduces a channel shuffle operation that permutes channels between groups before the next layer, enabling cross-group information flow.
• ResNeXt: Proposes a split-transform-merge strategy using grouped convolutions within its cardinality parameter, increasing model capacity and accuracy without a proportional increase in computational complexity.
• MobileNet & EfficientNet: While these primarily use depthwise separable convolutions, the grouped convolution principle is their foundational efficiency mechanism.
• Hardware Consideration: On microcontrollers, the reduced parameter count lowers flash storage needs, and the smaller per-group kernel size can improve cache locality during inference.

Selecting the group count (G) is a critical hyperparameter that balances efficiency against model representational capacity.

• Use Grouped Convolution When:
  • Primary constraints are parameter count and compute (FLOPs).
  • The network is sufficiently wide; the reduction in cross-channel interaction per layer is less detrimental.
  • Designing for microcontrollers with severely limited SRAM and flash memory.
• Avoid or Use Minimally When:
  • The task requires learning highly complex, interdependent features from all channels simultaneously (e.g., certain fine-grained classification tasks).
  • The network is already very narrow (small channel count), as further dividing channels can cripple learning.
• Best Practice: Start with a moderate group count (e.g., 4 or 8) and perform hardware-aware neural architecture search (HW-NAS) to find the optimal (G) for your specific accuracy, latency, and memory targets on the MCU.

The primary application of grouped convolution is to drastically reduce the number of parameters and floating-point operations (FLOPs). By dividing input channels into G groups, the computational cost is reduced by a factor of approximately G. For a standard convolution with C_in input channels, C_out output channels, and a kernel size of K x K, the parameter count is C_in * C_out * K * K. With grouped convolution (groups=G), this becomes (C_in/G) * (C_out/G) * K * K * G = (C_in * C_out * K * K) / G. This linear scaling makes it essential for mobile and embedded networks where memory and compute are severely limited.

Grouped convolution is the mathematical foundation for depthwise separable convolution, a cornerstone of efficient architectures like MobileNet. A depthwise convolution is simply a grouped convolution where the number of groups (G) equals the number of input channels (C_in). This applies a single filter per input channel, performing spatial filtering without cross-channel mixing. It is followed by a pointwise convolution (a 1x1 convolution with groups=1) to combine channels. This factorization reduces computations by roughly 8-9x compared to a standard 3x3 convolution, making it the default layer for TinyML vision models.

ShuffleNet uses pointwise group convolutions extensively to reduce the high cost of 1x1 convolutions. However, stacking multiple group convolutions blocks information flow between channel groups. To solve this, ShuffleNet introduces the channel shuffle operation. After a grouped convolution, it systematically rearranges the channels from different groups before the next layer. This allows information to propagate across groups while maintaining the low computational cost of grouped convolutions. This design enables high accuracy with very low FLOP budgets (e.g., < 150 MFLOPs for ImageNet).

The ResNeXt architecture reinterprets grouped convolution as a strategy for increasing cardinality (the size of the set of transformations) rather than just depth or width. Its building block uses a grouped convolution with 32 groups, formulating it as an aggregated residual transformation. This design increases model capacity and accuracy without a proportional increase in parameters, demonstrating that grouped convolutions can be used to build more expressive and efficient networks, not just smaller ones. It showed that boosting cardinality is a more effective way to gain accuracy than going deeper or wider.

On specialized hardware like NPUs (Neural Processing Units) or DSPs, grouped convolutions can be mapped efficiently to parallel compute units. By processing independent groups concurrently, they exploit data parallelism and reduce memory bandwidth requirements. Frameworks like TensorFlow Lite for Microcontrollers and CMSIS-NN include optimized kernel implementations for grouped and depthwise convolutions on ARM Cortex-M CPUs, using techniques like loop unrolling and SIMD instructions. The independent nature of groups makes them ideal for execution on multi-core microcontrollers.

Grouped convolutions enable conditional computation patterns where different parts of the network (groups) can be activated based on the input, saving dynamic inference cost. They are also fundamental to multi-branch architectures like Inception networks, where each branch can be seen as a group processing a different feature representation (e.g., 1x1, 3x3, 5x5 convolutions). In efficient NAS-derived networks, grouped convolutions allow the search algorithm to explore trade-offs between group size and accuracy, leading to hardware-optimal architectures for specific microcontroller memory footprints.

A 1x1 convolution that operates across all input channels to combine or project features into a new channel space. It is the second half of a depthwise separable convolution and is used extensively in efficient architectures for channel mixing. Following a grouped or depthwise convolution, the pointwise convolution reintegrates information across the separated channel groups, allowing for flexible feature recombination with relatively low computational cost.

An operation introduced in ShuffleNet that permutes the channels of a feature map between different groups. It is used after a pointwise group convolution to enable information flow across channel groups. This mitigates a key limitation of stacking multiple group convolutions, where outputs are only derived from a small subset of input channels, by allowing each group in the subsequent layer to receive input from all previous groups.

A mobile-optimized block from MobileNetV2 that uses a linear bottleneck and an inverted structure. It first expands the channel count with a cheap 1x1 convolution, applies a depthwise convolution, then projects back to a smaller channel count. This design preserves representational power in low-dimensional spaces and is more memory-efficient than classic residual blocks, making it ideal for microcontroller deployment.

An automated process for discovering optimal neural network architectures under direct hardware constraints like latency, memory, and energy. HW-NAS search spaces often include operations like grouped convolution, depthwise convolution, and channel shuffle. The search algorithm evaluates candidate architectures directly on the target microcontroller or NPU to find the most efficient design for the specific silicon, a critical step for TinyML deployment.