Ghost Module

A Ghost module splits the convolution process into two efficient parts. First, a regular convolution generates a small set of intrinsic feature maps. Second, a series of cheap linear operations (like depthwise convolutions) applies transformations to each intrinsic map to generate additional ghost feature maps. The final output is the concatenation of both sets, creating more feature channels with far fewer parameters and FLOPs than a standard convolution.

The core efficiency mechanism is the use of linear operations to generate ghost features. Common transformations include:

• Depthwise Convolutions: Applying a single filter per channel.
• Fixed-size Kernels: Using small, efficient kernels (e.g., 3x3 or 5x5).
• Identity Mapping: Simply reusing the intrinsic feature map itself.

These operations are computationally trivial compared to the standard convolutions they replace, as they require no channel mixing and far fewer multiplications.

The primary benefit is a dramatic reduction in resource consumption. For a standard convolution producing n feature maps, a Ghost module might generate m intrinsic maps and use cheap operations to create s ghost maps per intrinsic map, such that n = m * s. The theoretical speedup ratio r_s for parameters and FLOPs is approximately (n * c * k * k) / (m * c * k * k + (s-1) * m * d * d), where c is input channels, k is kernel size, and d is the cheap kernel size. In practice, this achieves 2x to 4x reductions in parameters and FLOPs with minimal accuracy loss.

Ghost modules are stacked into Ghost Bottlenecks, which form the building blocks of GhostNet. A Ghost Bottleneck has two stacked Ghost modules, structured similarly to MobileNetV2's inverted residual block:

1. Expansion Layer: First Ghost module expands the channel count.
2. Depthwise Convolution: An optional depthwise convolution for spatial filtering.
3. Projection Layer: Second Ghost module reduces channels back down.

Stride=2 bottlenecks include depthwise convolution for downsampling. This modular design allows GhostNet to be a drop-in replacement for networks like MobileNetV3.

Both techniques decompose standard convolutions for efficiency, but their approaches differ fundamentally.

Ghost modules can be seen as a feature-level redundancy reduction, while depthwise separable convolution is a kernel-level factorization.

GhostNet is the flagship architecture built entirely with Ghost Bottlenecks. On the ImageNet dataset, GhostNet achieves competitive accuracy with significant efficiency gains:

• GhostNet 1.0x: ~750M FLOPs, 5.2M parameters, 73.9% Top-1 accuracy.
• Outperforms MobileNetV3 at similar FLOP budgets.
• Demonstrates superior latency vs. accuracy trade-off on mobile CPUs (e.g., Huawei Kirin 990).

This proves the Ghost module's effectiveness as a foundational block for constructing state-of-the-art, hardware-efficient networks for embedded deployment.

A factorized convolution that decomposes a standard convolution into two efficient steps: a depthwise convolution (applies a single filter per input channel) followed by a pointwise convolution (a 1x1 convolution to combine channel outputs). This is the foundational operation in MobileNet architectures, drastically reducing parameters and FLOPs compared to standard 3x3 convolutions.

• Key Benefit: Reduces computational cost by a factor of nearly kernel_size².
• Use Case: The primary building block for most modern efficient CNNs targeting mobile phones and microcontrollers.

A mobile-optimized block, introduced in MobileNetV2, that uses a linear bottleneck and an inverted structure for efficiency. It first expands the channel count with a cheap 1x1 convolution, applies a depthwise convolution, then projects back to a lower channel count.

• Design Principle: Expands (→ depthwise → projects) instead of compresses (→ standard conv → expands).
• Linear Activation: Uses linear (ReLU6) activations in the bottleneck to prevent information loss in low-dimensional spaces.
• Purpose: Enables the use of residual connections in very narrow, efficient networks without degrading performance.

An architectural unit that models channel-wise relationships to improve representational power. It performs a squeeze operation (global average pooling) to get channel-wise statistics, then an excitation operation (small MLP with a gating mechanism) to produce per-channel recalibration weights.

• Mechanism: Re-weights feature map channels based on their global importance.
• Overhead: Adds minimal parameters (e.g., two fully-connected layers) for significant accuracy gains.
• Context: While powerful, its non-linearities can be problematic for integer-only accelerators, leading to variants like in EfficientNet-Lite.

The core building block of SqueezeNet, designed for extreme parameter reduction. It consists of a squeeze layer (only 1x1 convolutions) that compresses input channels, feeding into an expand layer that uses a mix of 1x1 and 3x3 convolutions to increase channels again.

• Goal: Achieve AlexNet-level accuracy with 50x fewer parameters.
• Strategy: Reduces input channels to 3x3 convolutions, which are parameter-heavy, and emphasizes 1x1 convolutions.
• Legacy: A pioneering design demonstrating that careful architectural choices can drastically shrink model size without collapsing accuracy.

A structural component that uses 1x1 convolutions to first reduce (compress) and then expand the number of channels. This creates a computational 'bottleneck' within a block, limiting the cost of subsequent expensive operations (e.g., 3x3 convs).

• Origin: Popularized by ResNet for building very deep networks efficiently.
• Function: The initial 1x1 conv reduces dimensionality; the final 1x1 conv restores it.
• Contrast: In an inverted residual block, this order is reversed (expand → depthwise → project).

A 1x1 convolution that operates across all input channels to combine or project them into a new channel space. It is the workhorse for channel mixing and dimensionality adjustment in efficient architectures.

• Role in Depthwise Separable Convs: Follows the depthwise step to create linear combinations of channels.
• Computational Profile: Much cheaper than a KxK convolution but still accounts for most of the computation in a depthwise separable block.
• Ubiquity: Found in nearly all efficient CNN designs, including Ghost Modules, for efficient feature transformation.