Cascade Forward Neural Network

Unlike a standard multilayer perceptron where each layer connects only to the next, a cascade forward network establishes direct weighted links from every hidden layer—and the input layer—straight to the output neurons. This means the output layer receives a composite signal: the transformed features from the final hidden layer plus the raw or partially processed signals from all preceding layers. For digital predistortion, this allows the network to simultaneously learn both deep nonlinear transformations and shallow corrective terms, effectively modeling the PA's complex AM/AM and AM/PM distortion with fewer layers.

The direct connections create auxiliary gradient paths during backpropagation. When computing the error gradient for a neuron in an early hidden layer, the gradient flows not only through the subsequent hidden layers but also directly from the output layer via the shortcut connection. This mitigates the vanishing gradient problem that plagues deep standard networks, enabling faster and more stable training. For PA linearization, this ensures that even subtle memory effects captured in the first hidden layer receive a strong, unattenuated learning signal.

The architecture inherently performs a form of residual learning without explicit skip connections. The output is a linear combination of nonlinear transformations at different depths:

• Input layer: Provides a linear baseline (the original signal)
• Shallow hidden layers: Model low-order, short-term memory effects
• Deep hidden layers: Capture high-order, long-term nonlinear dynamics This allows the network to focus its deeper layers on learning the deviation from linearity, which is precisely the distortion that must be corrected.

Because every layer contributes directly to the output, a cascade forward network can achieve modeling accuracy comparable to a much deeper standard feedforward network. A 3-hidden-layer cascade network can often match the performance of a 5- or 6-layer standard MLP for PA behavioral modeling. This reduction in sequential depth translates directly to lower inference latency—a critical advantage when implementing real-time DPD on FPGA or ASIC hardware where every nanosecond of computational delay reduces the viable linearization bandwidth.

A standard Real-Valued Time-Delay Neural Network (RVTDNN) for DPD uses a purely sequential feedforward topology. The cascade forward variant enriches this by adding cross-layer connections. Key distinctions:

• RVTDNN: Output = f(W₃ · f(W₂ · f(W₁ · x)))
• Cascade Forward: Output = f(W₃ · h₃ + W₂' · h₂ + W₁' · h₁ + W₀' · x) The cascade formulation provides a more expressive hypothesis space for the same number of parameters, often yielding 1-2 dB better ACLR improvement when trained on identical PA measurement data.

The multiple signal paths converging at the output can create internal covariate shift challenges—the distribution of activations arriving from different layers may have vastly different scales. Integrating batch normalization after each hidden layer normalizes these contributions before they reach the output summation, preventing any single layer's signal from dominating the gradient. This is particularly important when modeling PAs with strong nonlinearities, where the deep layers may produce large activation magnitudes that would otherwise destabilize training.

A deep neural network design where layers learn the difference between the target and the input, implemented via skip connections. In cascade forward networks, the direct connections from each hidden layer to the output naturally create a residual-like structure. This simplifies the optimization landscape, allowing the network to learn the PA's nonlinear distortion as a correction term rather than modeling the full output from scratch. This approach significantly improves gradient flow and convergence speed when training very deep predistorter architectures.

The gradient computation algorithm used to train recurrent neural network predistorters by unrolling the network's temporal operations and propagating the error signal backward through the sequence. While cascade forward networks are feedforward, understanding BPTT is essential because many PA behavioral models incorporate tapped delay lines to capture memory effects. The cascade forward topology's direct output connections help mitigate the vanishing gradient problem that often plagues BPTT in deep recurrent structures, making it a preferred architecture for memory-aware predistorters.

The strategy for setting the initial values of a neural network's parameters before training. For cascade forward networks, proper initialization is critical due to the multiple direct paths to the output layer. Common strategies include:

• Xavier/Glorot initialization: Scales weights based on fan-in and fan-out, ideal for tanh activations
• He initialization: Optimized for ReLU activations, accounting for the asymmetric activation function
• Orthogonal initialization: Preserves gradient norm through deep networks, beneficial for the cascade topology's parallel signal paths Poor initialization can cause exploding gradients in the direct connections, destabilizing early training epochs.

A training technique that randomly deactivates a fraction of neurons during each forward pass to prevent co-adaptation. In cascade forward networks, dropout must be applied carefully because each hidden layer has a direct connection to the output. Standard dropout on hidden layers can disrupt the direct signal path, so practitioners often use:

• DropConnect: Drops individual weights rather than entire neurons
• Spatial dropout: Drops entire feature maps in convolutional variants
• Variational dropout: Applies the same dropout mask across time steps for temporal consistency This prevents the network from overfitting to specific PA measurement conditions while preserving the cascade topology's gradient flow advantages.

The process of reducing the numerical precision of a neural network's weights and activations from 32-bit floating-point to lower bit-width integers (typically 8-bit or 16-bit). For cascade forward networks deployed on FPGA-based DPD systems, quantization is essential to meet real-time inference latency constraints. The direct connections to the output layer require special attention during quantization-aware training because the accumulated error from multiple quantized paths can compound at the summation node. Techniques like per-channel quantization and learned quantization ranges help preserve linearization accuracy after compression.

The systematic process of optimizing the neural network's architectural and training parameters. For cascade forward networks, critical hyperparameters include:

• Number of hidden layers: More layers increase modeling capacity but also the number of direct output connections
• Neurons per layer: Determines the dimensionality of the learned PA nonlinearity representation
• Learning rate: Must balance the gradient contributions from both the sequential and direct paths
• Batch size: Affects the stability of gradient estimates, particularly for the direct connections
• Activation functions: ReLU variants work well with direct connections, while tanh provides bounded outputs suitable for I/Q signal predistortion Grid search and Bayesian optimization are common tuning approaches, with ACLR improvement and EVM reduction as the primary optimization objectives.