Complex-Valued Neural Network

Extends standard convolution to operate on complex-valued inputs using complex-valued weights and complex activation functions. The convolution operation performs complex multiplication, preserving cross-channel phase relationships.

• Complex weight: ( W = W_R + jW_I )
• Complex input: ( X = X_R + jX_I )
• Output: ( Y = W * X = (W_R X_R - W_I X_I) + j(W_R X_I + W_I X_R) )

This enables the network to learn rotationally equivariant features, meaning a phase shift in the input produces a corresponding phase shift in the feature maps rather than a completely different representation.

Standard activation functions like ReLU cannot be directly applied to complex numbers without destroying phase information. CVNNs employ specialized activations:

• modReLU: Applies ReLU to the magnitude while preserving the phase: ( \text{modReLU}(z) = \text{ReLU}(|z| + b) \cdot e^{j\theta_z} )
• zReLU: Passes only elements whose phase lies in ([0, \pi/2]), enforcing sparsity in the complex domain
• cReLU: Applies separate real-valued activations to real and imaginary parts, though this breaks analyticity
• Complex Cardioid: A fully complex, differentiable function that respects the Cauchy-Riemann equations for true complex differentiability

CVNNs rely on Wirtinger calculus (or CR-calculus) for gradient computation because a complex function ( f(z) ) is not necessarily complex-differentiable in the standard sense unless it is holomorphic.

• Treats ( z ) and its conjugate ( \bar{z} ) as independent variables
• Wirtinger derivatives: ( \frac{\partial f}{\partial z} ) and ( \frac{\partial f}{\partial \bar{z}} )
• Enables gradient descent on non-holomorphic loss functions
• Critical for training networks with real-valued loss functions (e.g., cross-entropy) on complex-valued parameters

This framework ensures stable optimization while preserving the algebraic structure of the complex domain.

Standard batch normalization assumes real-valued distributions. Complex batch normalization whitens complex activations by treating them as 2D vectors in the complex plane.

• Computes a complex mean and a 2×2 covariance matrix for each channel
• Normalizes using the inverse square root of the covariance matrix
• Applies learnable complex scaling ( \gamma ) and complex shift ( \beta )
• Preserves the correlation between real and imaginary components

This stabilization technique is essential for training deep CVNNs and accelerates convergence by decorrelating the real and imaginary feature dimensions.

Residual connections in CVNNs perform complex element-wise addition between the input and output of a layer block, enabling gradient flow through deep architectures.

• Complex skip connection: ( Y = F(X, W) + X ) where all terms are complex-valued
• Preserves both magnitude and phase of the identity mapping
• Enables training of very deep complex networks (50+ layers) without vanishing gradients
• Critical for extracting hierarchical RF features from raw I/Q samples

These connections allow the network to learn residual phase corrections and magnitude adjustments, which directly correspond to hardware impairment signatures.

Proper initialization is critical for CVNN training stability. Complex He initialization or complex Xavier initialization adapts standard schemes to the complex domain.

• Rayleigh distribution for magnitude: ensures appropriate variance scaling
• Uniform distribution for phase: ( \theta \sim U[-\pi, \pi] )
• Variance scaled by ( 1/(n_{in} + n_{out}) ) for Xavier or ( 2/n_{in} ) for He
• Prevents exploding or vanishing complex gradients in early training epochs

Poor initialization can cause the network to collapse to a degenerate state where all phase information is lost, rendering the complex-valued architecture no better than a real-valued equivalent.

The native format for complex-valued networks, representing signals as in-phase (I) and quadrature (Q) components. Unlike real-valued networks that treat I and Q as separate input channels, CVNNs preserve the phase and magnitude relationships critical for capturing hardware impairments.

• I component: Cosine-modulated carrier
• Q component: Sine-modulated carrier
• Complex sample: x = I + jQ
• Preserves instantaneous phase information lost in magnitude-only representations

Extends standard convolution to the complex domain by applying complex-valued filters to complex inputs. The operation performs complex multiplication, mixing real and imaginary parts to capture cross-channel correlations between I and Q.

• Filter: W = A + jB
• Input: x = I + jQ
• Output: (A*I - B*Q) + j(B*I + A*Q)
• Learns joint I/Q representations rather than treating them independently

Non-linearities adapted for complex-valued tensors. Standard functions like ReLU cannot be directly applied because complex numbers lack a natural ordering. Common solutions include:

• modReLU: Applies ReLU to magnitude, preserves phase
• zReLU: Passes elements only if phase is in [0, π/2]
• Complex Cardioid: f(z) = 0.5 * (1 + cos(∠z)) * z
• Split activation: Separate real/imaginary non-linearities (sacrifices holomorphicity)

Gradient-based learning using Wirtinger calculus, which treats the complex variable and its conjugate as independent. This enables optimization of non-holomorphic loss functions common in deep learning.

• Wirtinger derivatives: ∂L/∂z and ∂L/∂z̄
• Enables training with real-valued loss functions
• Critical for phase-sensitive optimization
• Preserves analyticity where possible for stable gradient flow

Normalizes complex activations by treating them as 2D vectors. Instead of scalar mean and variance, it computes a 2×2 covariance matrix to whiten the complex distribution, accounting for correlation between real and imaginary parts.

• Centers by complex mean E[z]
• Scales by inverse square root of covariance matrix
• Preserves circular symmetry of complex distributions
• Reduces internal covariate shift without destroying phase structure

A real-valued network with separate I/Q channels applies independent filters to each component, learning correlations only in deeper layers. A CVNN applies complex multiplication from the first layer, inherently capturing phase rotation and I/Q coupling.

• Real dual-channel: f(I) + g(Q) — additive, no cross-talk
• Complex: (A*I - B*Q) + j(B*I + A*Q) — multiplicative mixing
• CVNNs require fewer parameters for equivalent representational power
• Superior for tasks where relative phase between I and Q is discriminative