Complex-Valued Neural Network

Standard real-valued backpropagation fails in the complex domain because activation functions like the complex ReLU are non-holomorphic. CVNNs rely on Wirtinger calculus (or CR-calculus), which computes gradients with respect to the complex variable and its complex conjugate independently.

• Enables gradient-based optimization for non-analytic complex functions
• The gradient is defined as ∇_z f = ∂f/∂z* (conjugate gradient)
• Implemented in frameworks like PyTorch and TensorFlow with complex tensor support
• Critical for training deep architectures like Complex-Valued CNNs and RNNs

Unlike real-valued activations that operate solely on magnitude, CVNN activation functions must handle both amplitude and phase components of the complex signal. Improper activation design can destroy the phase information essential for coherent processing.

• modReLU: Applies ReLU to the magnitude while preserving the phase: modReLU(z) = ReLU(|z| + b) * e^(iθ_z)
• Complex Cardioid: A non-linear function that squashes the magnitude based on the phase angle
• zReLU: Passes the complex value through only if the phase lies in [0, π/2]
• Complex Tanh: Bounded output applied separately to real and imaginary parts or to magnitude and phase

A single complex multiplication performs a rotation and scaling operation simultaneously. This gives CVNNs a representational advantage over real-valued networks with twice the parameters.

• A complex weight w = |w|e^(iφ) rotates the input by angle φ and scales by |w|
• Enables efficient learning of transformations in the I/Q plane
• Reduces the parameter count compared to equivalent real-valued architectures
• Naturally models physical phenomena like phase shifts and rotations in electromagnetic waves

Standard batch normalization assumes real-valued, uncorrelated features. Complex batch normalization must account for the covariance between real and imaginary components to properly whiten the complex distribution.

• Whitens using the full 2×2 complex covariance matrix, not just variance
• Normalizes both the real-imaginary correlation structure and the magnitude
• Learnable shift and scale parameters are complex matrices
• Stabilizes training of deep CVNNs for channel estimation and beamforming tasks

Complex convolution extends the standard convolution operation by using complex-valued filters. This is particularly powerful for processing raw I/Q samples directly, without separating them into real and imaginary streams.

• A complex filter kernel convolves with a complex input feature map
• Preserves cross-channel phase relationships between I and Q components
• Used in Automatic Modulation Classification (AMC) and RF fingerprinting
• Outperforms dual real-valued CNNs on tasks where phase coherence matters

Deep Complex Residual Networks (Complex ResNets) combine complex convolutions with skip connections to learn the mapping from received pilots to the full channel response matrix.

• Skip connections prevent vanishing gradients in deep complex architectures
• Directly processes complex baseband symbols without I/Q decomposition
• Achieves superior Normalized Mean Square Error (NMSE) compared to MMSE estimators
• Adapts to non-linear hardware impairments and high Doppler scenarios that classical estimators cannot model

The mathematical framework that enables gradient-based optimization of real-valued cost functions with respect to complex-valued parameters. Since a complex function is not holomorphic in the standard sense, Wirtinger calculus defines derivatives with respect to the complex variable and its conjugate independently. This is the backbone of backpropagation in CVNNs, allowing standard stochastic gradient descent to work seamlessly in the complex domain without needing to separate real and imaginary parts.

Unlike real-valued ReLU, activation functions in the complex domain must handle magnitude and phase carefully. Common choices include:

• modReLU: Applies ReLU to the magnitude while preserving phase.
• Complex Cardioid: A holomorphic function bounded in magnitude.
• Split Activation: Applies separate real-valued activations to the real and imaginary components. The choice critically impacts the network's ability to learn phase-sensitive representations without saturation.

The native data format for complex-valued neural networks in wireless systems. A signal is represented as z = I + jQ, where I is the in-phase component and Q is the quadrature component. CVNNs process this as a single atomic entity, inherently preserving the instantaneous phase and amplitude relationships that are destroyed when I and Q are fed as separate real channels into a standard real-valued network.

A core physical-layer task where a CVNN predicts the complex channel response matrix H from received pilot symbols. Because the channel itself is a complex phenomenon (attenuation + phase rotation), a CVNN can learn a more accurate and sample-efficient mapping from pilots to channel state information (CSI) than a real-valued equivalent, directly optimizing for complex-domain mean squared error.

A technique to linearize power amplifiers by applying an inverse nonlinear transformation to the baseband signal. CVNNs are uniquely suited for this because they can model the complex-valued nonlinear distortion (AM/AM and AM/PM) directly. A complex neural network learns the inverse transfer function, preserving the phase fidelity of the predistorted signal, which is critical for meeting spectral emission masks.

A demodulation method that requires precise knowledge of the carrier's absolute phase. CVNNs can be trained to perform joint channel estimation and coherent symbol detection directly from raw I/Q samples. By operating in the complex domain, the network implicitly learns to track and compensate for phase noise and frequency offset, effectively building a neural coherent receiver.