Convolutional Neural Network (CNN)

The fundamental building block of a CNN. A convolutional layer applies a set of learnable kernels (filters) that slide across the input's spatial dimensions, performing element-wise multiplication and summation. This operation produces feature maps that highlight the presence of specific patterns, such as edges, textures, or specific symbol clusters in a constellation diagram.

• Local Connectivity: Each neuron connects only to a small, localized region of the input (the receptive field), drastically reducing parameters compared to fully connected layers.
• Parameter Sharing: The same kernel weights are used across the entire input, making the network equivariant to translations—a shifted signal feature will produce a shifted activation.

Pooling layers perform non-linear downsampling, reducing the spatial dimensionality of feature maps. This creates a more compact representation that is robust to minor translations and distortions in the input.

• Max Pooling: Selects the maximum value from a cluster of neurons, preserving the most salient features while discarding precise positional information.
• Average Pooling: Computes the average value, providing a smoother downsampling effect.
• Dimensionality Reduction: By progressively reducing spatial size, pooling decreases computational load for subsequent layers and helps prevent overfitting.

Activation functions introduce non-linearity into the network, enabling it to learn complex, non-linear relationships in data. Without them, the entire CNN would collapse into a single linear transformation.

• ReLU (Rectified Linear Unit): The standard choice, defined as f(x) = max(0, x). It is computationally efficient and mitigates the vanishing gradient problem, accelerating convergence.
• Leaky ReLU: A variant that allows a small, non-zero gradient when the unit is inactive, preventing 'dying' neurons.
• Sigmoid/Tanh: Historically used but now less common in hidden layers due to saturation issues that slow training.

Positioned at the end of the network, fully connected (dense) layers perform high-level reasoning. They take the high-level, spatially-reduced feature maps from the convolutional base and flatten them into a single feature vector.

• Classification Head: The final fully connected layer typically uses a Softmax activation to output a probability distribution over target classes (e.g., QPSK, 16-QAM, 64-QAM).
• Global Average Pooling (GAP): A modern alternative that reduces each feature map to a single average value before the final layer, significantly cutting parameters and acting as a structural regularizer.

A defining feature of deep architectures like ResNet. Skip connections bypass one or more layers by adding the input of a block directly to its output. This simple mechanism addresses the degradation problem where deeper networks paradoxically perform worse.

• Gradient Highway: They provide an unimpeded path for gradients during backpropagation, effectively mitigating the vanishing gradient problem in very deep networks.
• Identity Mapping: The network learns residual functions (F(x) + x) rather than unreferenced mappings, making optimization easier and enabling the training of models with hundreds of layers for complex signal analysis.

A technique applied to the inputs of a layer for each mini-batch. It normalizes the activations to have a mean of zero and a standard deviation of one, then scales and shifts them using learnable parameters.

• Internal Covariate Shift: BatchNorm reduces the change in the distribution of layer inputs during training, allowing for significantly higher learning rates and making the network less sensitive to weight initialization.
• Regularization Effect: The noise introduced by the mini-batch statistics acts as a mild regularizer, often reducing the need for Dropout and accelerating the training of deep modulation classifiers.