Complex-Valued Neural Network

Unlike real-valued networks that treat I and Q components as separate input channels, CVNNs use complex weights and perform multiplication that mixes phase and magnitude. The fundamental operation is a complex dot product, which inherently models rotation and scaling. This allows a single complex neuron to learn transformations that would require two real-valued neurons, preserving the holomorphic relationship between I and Q components.

Standard activation functions like ReLU cannot be directly applied to complex numbers without destroying phase information. CVNNs employ specialized functions such as:

• modReLU: Applies ReLU to the magnitude while preserving the phase: max(0, |z|) * e^(i*θ)
• Complex tanh: Bounds both real and imaginary parts
• zReLU: Passes the value only if the phase lies within a specific range These functions maintain the analyticity required for complex-valued backpropagation.

CVNNs cannot use standard real-valued backpropagation because a complex function is not differentiable in the Cauchy-Riemann sense unless it is holomorphic. Instead, training relies on Wirtinger calculus, which computes gradients with respect to the complex variable z and its complex conjugate z* independently. This framework treats the real and imaginary parts as separate entities during optimization, enabling gradient descent on non-holomorphic cost functions like the standard mean squared error.

A key advantage of CVNNs is their natural ability to handle phase rotation. A constant phase offset in the input, common due to Carrier Frequency Offset (CFO) or channel effects, simply becomes a global rotation in the complex plane. Because complex multiplication is commutative with rotation, a properly trained CVNN can learn representations that are intrinsically equivariant to phase shifts, eliminating the need for explicit data augmentation with rotated examples that real-valued networks require.

Standard batch normalization assumes real-valued distributions and cannot simply be applied to real and imaginary parts independently due to their correlation. Complex batch normalization whitens the complex data by computing a 2x2 covariance matrix between the real and imaginary components. It then scales the data by the inverse square root of this covariance matrix, effectively decorrelating the components and stabilizing training. This process is mathematically equivalent to transforming an elliptical distribution into a circular one.

A Complex Convolutional Neural Network extends the convolution operation to the complex domain. A complex convolution kernel W = A + iB is convolved with a complex input h = x + iy. The output is computed as W * h = (A*x - B*y) + i(B*x + A*y). This operation explicitly models the cross-coupling between I and Q dimensions, allowing the network to learn complex-valued spatial hierarchies directly from raw I/Q samples for tasks like Automatic Modulation Classification.

Extends standard convolution to operate directly on complex numbers. A complex filter W = A + iB is convolved with a complex input h = x + iy using complex multiplication, preserving the algebraic structure of the signal. This allows the network to learn phase rotation-equivariant features that are invariant to the absolute phase offset of an I/Q signal, a property real-valued networks must learn through expensive data augmentation.

Standard batch normalization treats real and imaginary components independently, which can distort the complex distribution. Complex batch normalization whitens the data using a 2x2 covariance matrix to account for the correlation between real and imaginary parts. This stabilizes training by ensuring the normalized output maintains a standard circular complex distribution, preventing the vanishing gradient problem in deep complex architectures.

Applies non-linearity to both magnitude and phase. Key types include:

• modReLU: Applies ReLU only to the magnitude, preserving the phase angle.
• zReLU: Passes the element only if the phase angle lies in [0, π/2].
• Cardioid: A holomorphic function that is bounded almost everywhere, satisfying Liouville's theorem constraints. These functions are critical because standard real-valued activations applied separately to I and Q destroy the analytic relationship between components.

The training algorithm must respect Wirtinger calculus (CR-calculus) because complex loss functions are non-holomorphic. Gradients are computed with respect to the complex variable z and its conjugate z̄ independently. The update rule for a complex weight W uses the conjugate gradient: ∇_W̄ L = ∂L/∂W_real + i(∂L/∂W_imag). This ensures the optimization descends the true complex gradient surface.

A foundational architecture introduced by Trabelsi et al. (2018) that provides a blueprint for building deep feed-forward complex models. DCNs combine complex convolutions, complex batch normalization, and complex weight initialization strategies. They demonstrated that complex networks achieve richer representational capacity with fewer parameters than their real-valued counterparts, particularly on tasks involving polar coordinate representations like I/Q data and MRI reconstruction.

Extends the self-attention mechanism to the complex domain for architectures like Complex Transformers. The attention score between a query Q and key K is computed using the complex inner product: Attention(Q,K,V) = softmax(|QK^H|/√d_k)V, where K^H is the conjugate transpose. This allows the model to attend to both magnitude similarity and phase alignment simultaneously, capturing temporal dependencies in I/Q sequences that real-valued Transformers miss.