Learned ISTA

The core mechanism of Learned ISTA is algorithm unrolling (or deep unfolding), where each iteration of the classical ISTA algorithm is mapped to a distinct layer of a neural network. The shrinkage-thresholding function acts as a non-linear activation, while the gradient descent step becomes a linear transformation. This creates a fixed-depth recurrent computational graph that is fully differentiable, allowing end-to-end training via backpropagation. The number of unrolled iterations, typically 10-20, directly controls the trade-off between computational complexity and recovery accuracy.

Unlike classical ISTA, which relies on hand-tuned parameters derived from the Lipschitz constant of the measurement operator, Learned ISTA treats step sizes and shrinkage thresholds as learnable parameters. During training, these values are optimized via stochastic gradient descent to minimize the reconstruction loss on a dataset of representative signals. Key advantages include:

• Per-layer adaptation: Each unrolled layer learns its own independent step size and threshold, enabling non-uniform convergence dynamics.
• Data-driven regularization: The shrinkage threshold effectively learns the optimal soft-thresholding level for the specific signal distribution, outperforming fixed heuristics like universal thresholding.

A defining characteristic of Learned ISTA is its ability to achieve high-quality sparse recovery in significantly fewer iterations than its classical counterpart. While classical ISTA may require hundreds of iterations to converge, Learned ISTA typically reaches superior accuracy in 10-15 unrolled layers. This acceleration arises because the network learns to take larger, more aggressive steps in early layers and finer, denoising steps in later layers. The learned parameters effectively encode a problem-specific preconditioning strategy that exploits the statistical structure of the training data.

Despite its learned components, Learned ISTA retains provable convergence guarantees under certain conditions. By constraining the learned step sizes to satisfy a bound related to the inverse Lipschitz constant of the measurement matrix, the unrolled network is guaranteed to be a contractive mapping. This ensures that the output converges to a fixed point and that the network's behavior remains stable and interpretable. Techniques like soft-thresholding with tied parameters across layers can further enforce the theoretical properties of the original ISTA algorithm while still benefiting from learned adaptation.

Learned ISTA has found significant application in massive MIMO channel estimation, where the wireless channel exhibits sparsity in the angular or delay domain. The algorithm unrolls the sparse recovery of the channel impulse response from received pilot signals. Key benefits in this domain include:

• Pilot reduction: Achieves accurate channel estimation with fewer pilot symbols than classical Least Squares (LS) or MMSE estimators.
• Robustness to noise: The learned shrinkage function effectively denoises the channel estimate, particularly in low SNR regimes.
• Integration with Model-Driven Unfolding: Serves as a foundational building block for more complex unfolded receivers like DeepRx.

The foundational architecture is often referred to as LISTA (Learned Iterative Shrinkage-Thresholding Algorithm). Several variants have been proposed to enhance performance:

• Tied LISTA: Shares parameters across all unrolled layers, reducing the parameter count and enforcing a recurrent structure that can be iterated until convergence at inference time.
• Analytical LISTA (ALISTA): Incorporates theoretical insights to analytically compute the step size, leaving only the threshold as a learnable parameter, which improves generalization.
• Coupled LISTA: Introduces a learned coupling matrix between layers to model more complex dependencies beyond the standard gradient descent step.

The overarching methodology that unrolls iterative optimization algorithms into neural network layers. Each layer corresponds to one iteration of a classical algorithm, with learnable parameters replacing hand-crafted hyperparameters.

• Preserves the structural priors of the original algorithm
• Drastically reduces the number of trainable parameters compared to black-box networks
• Enables interpretable intermediate outputs at each layer
• Also known as deep unfolding or algorithm unrolling

An unrolled version of the Alternating Direction Method of Multipliers (ADMM), a powerful algorithm for constrained convex optimization. By unfolding ADMM, the penalty parameters and proximal operators become learnable functions.

• Applied in distributed beamforming and resource allocation
• Handles constraints naturally through the augmented Lagrangian structure
• Converges in fewer iterations than classical ADMM due to learned acceleration

Unfolds the Approximate Message Passing (AMP) algorithm, which excels in high-dimensional sparse signal recovery under i.i.d. Gaussian measurement matrices. The learned variant replaces the scalar denoising function with a learned denoiser.

• Achieves state-of-the-art performance in compressive sensing
• Used in massive MIMO channel estimation
• The Onsager correction term is preserved to maintain statistical independence across layers

A related paradigm where the network is defined implicitly as the fixed-point solution of an iterative process, rather than unrolling a fixed number of steps. The forward pass finds the equilibrium, and backpropagation uses the implicit function theorem.

• Memory footprint is constant regardless of effective depth
• Equivalent to an infinite-depth weight-tied unrolled network
• Applied to inverse problems and physical layer tasks where convergence guarantees are desired

An extension of Learned ISTA that incorporates convolutional sparse coding priors. Instead of a fixed dictionary, LISTA-CP learns a convolutional dictionary, making it suitable for signals with shift-invariant structure.

• Applied in image denoising and radio frequency signal separation
• The convolutional structure dramatically reduces the parameter count for high-dimensional signals
• Combines the efficiency of unrolled ISTA with the representational power of convolutional sparse coding

A variant of Learned ISTA where the weight matrices are constrained to be analytically structured based on the measurement matrix, rather than fully learned. This enforces a tighter coupling to the physical sensing model.

• Improves generalization to unseen measurement matrices
• Reduces overfitting when training data is limited
• Maintains the interpretability of the original ISTA structure while still learning optimal step sizes and thresholds