Complex-Valued Neural Networks (CVNN)

CVNNs replace real-valued parameters with complex numbers (a + bi), where each weight simultaneously represents magnitude and phase. This dual representation allows a single complex neuron to encode rotational transformations that would require two real-valued neurons. During training, Wirtinger calculus enables gradient computation for non-holomorphic activation functions, ensuring proper backpropagation through both real and imaginary components.

Standard ReLU and sigmoid functions cannot operate directly on complex values without destroying phase information. CVNNs employ specialized activations:

• modReLU: Applies ReLU to magnitude while preserving phase
• zReLU: Passes values only when phase lies in [0, π/2]
• Complex cardioid: Maps inputs to a cardioid-shaped region These functions maintain the analytic signal properties critical for RF applications.

A defining property of CVNNs is their natural handling of rotational transformations. When input data is multiplied by a unit complex number (a pure phase rotation), the network's internal representations transform predictably. This makes CVNNs exceptionally efficient at learning from IQ constellation diagrams, where modulation schemes like QPSK and 16-QAM exhibit rotational symmetries that real-valued networks must learn as separate patterns.

CVNNs process analytic signals directly without separating real and imaginary components. This preserves the orthogonality between I and Q branches and captures the full second-order statistics of complex random processes. For non-circular signals common in communications, CVNNs naturally model the correlation between a signal and its complex conjugate—a relationship real-valued networks require widely linear filtering to approximate.

By operating in the complex domain, CVNNs achieve comparable representational power with fewer trainable parameters than their real-valued counterparts. A complex weight matrix of size N×N contains 2N² real degrees of freedom but encodes richer geometric structure. Empirical studies on channel estimation and automatic modulation classification tasks show CVNNs converging in fewer epochs with lower generalization error, particularly when training data is limited.

Training CVNNs requires extending gradient descent to the complex plane. Wirtinger calculus provides the mathematical framework by treating the complex variable and its conjugate as independent quantities. The gradient is computed as:

• ∇_z L = ∂L/∂z + i(∂L/∂z̄) This enables end-to-end complex optimization without separating real and imaginary paths, maintaining the network's ability to learn phase-sensitive features throughout all layers.

The mathematical backbone of CVNN training. Standard backpropagation fails on complex functions because they are non-holomorphic. Wirtinger calculus solves this by treating the complex variable and its conjugate as independent, computing separate derivatives. This yields the conjugate gradient for optimization, ensuring the loss function decreases monotonically in the complex plane.

The native data format for CVNNs. A real-valued RF signal is downconverted and centered at zero hertz, represented as a complex-valued stream where the in-phase (I) component is the real part and the quadrature (Q) component is the imaginary part. This preserves both amplitude and phase without doubling the bandwidth, making it the ideal input for phase-sensitive neural processing.

A statistical property critical to CVNN design. A complex signal is proper (circular) if its probability distribution is rotationally invariant and it is uncorrelated with its own complex conjugate. Many communication signals (QPSK, QAM) are circular. Improper (non-circular) signals like BPSK or those with IQ imbalance require widely linear architectures that process both the signal and its conjugate.

Standard ReLU and sigmoid cannot be directly applied in the complex domain without destroying phase information. CVNNs use specialized activations:

• modReLU: Applies ReLU to magnitude, preserves phase
• zReLU: Passes elements with phase in [0, π/2]
• cReLU: Applies ReLU separately to real and imaginary parts
• Cardioid: A holomorphic function learned from data The choice critically impacts gradient flow and phase preservation.

An augmented processing approach essential for non-circular signals. A standard linear filter operates only on the signal z. A widely linear filter processes both z and its complex conjugate z*, effectively doubling the dimensionality. This is the minimum architecture required to achieve optimal mean-square error performance for improper data, and forms the basis for augmented CVNN layers.

Standard batch normalization assumes real-valued distributions. Complex batch normalization must whiten 2D circular distributions, not just 1D. This requires computing a 2x2 covariance matrix to decorrelate the real and imaginary components. The process centers, scales, and rotates the complex data to have zero mean and unit circular variance, stabilizing CVNN training.