Complex-Valued Input

Unlike standard ReLU or sigmoid functions that operate on real numbers, complex-valued networks require holomorphic activation functions that are differentiable in the complex domain. Common choices include modReLU, which applies rectification to the magnitude while preserving phase, and cReLU, which applies separate real-valued activations to the real and imaginary parts. The Cauchy-Riemann equations impose strict constraints on fully complex activations, making the design of effective nonlinearities a central research challenge.

Proper initialization of complex weights is critical to avoid vanishing or exploding gradients. The complex Gaussian distribution with zero mean and variance scaled by the number of input connections is standard. The phase of each weight is typically drawn from a uniform distribution between -π and π. Improper initialization can cause the network to converge to suboptimal local minima where all phase information is lost.

Standard backpropagation relies on real-valued derivatives. Complex-valued networks use Wirtinger calculus, which treats the complex variable and its complex conjugate as independent entities. This yields two partial derivatives: the R-derivative and the conjugate R-derivative. The gradient descent update is then computed using the conjugate derivative, ensuring the optimization moves in the direction of steepest descent in the complex plane.

Complex convolution multiplies and accumulates complex-valued filter weights with complex-valued input patches. This operation inherently preserves cross-channel phase relationships that are destroyed when I and Q are processed as separate real channels. The resulting feature maps retain both magnitude and phase information, allowing deeper layers to learn phase-sensitive features such as rotational symmetries in the constellation diagram.

Standard batch normalization assumes real-valued data. Complex batch normalization must whiten complex-valued activations by treating the real and imaginary parts as a 2D vector. This requires computing a 2x2 covariance matrix for each batch and applying a whitening transformation that decorrelates and scales the real and imaginary components jointly. This stabilizes training and accelerates convergence for deep complex architectures.

A key advantage of complex-valued networks is their natural ability to encode rotational equivariance. A phase shift in the input IQ stream, caused by residual carrier offset, corresponds to multiplication by a unit complex number. Complex-valued weights can learn to represent features that are covariant with rotation, meaning the feature representation rotates in lockstep with the input. This eliminates the need for explicit data augmentation with random phase rotations.