Beamforming Transformer

A beamforming transformer learns a deterministic function that maps raw channel state information (CSI) matrices directly to complex beamforming weight vectors. Unlike traditional algorithms like MMSE or Zero-Forcing that solve an optimization problem for each channel realization, the transformer performs a single inference pass. This is achieved by treating antenna elements or user channels as a sequence of tokens, allowing the self-attention mechanism to model inter-antenna and inter-user interference patterns implicitly. The model is trained end-to-end on a loss function that directly maximizes a communication objective, such as sum-rate or signal-to-interference-plus-noise ratio (SINR).

A critical inductive bias for beamforming is permutation equivariance: if the order of users in the input is permuted, the output beamforming vectors should permute identically. Beamforming transformers achieve this by avoiding positional encodings for the user dimension and using a shared, per-user processing architecture. This property ensures the model generalizes to any number of users without retraining. The architecture is also invariant to the ordering of antennas, meaning the physical geometry of the array is learned purely from the channel coefficients, not hard-coded into the model structure.

Standard transformer attention operates on real numbers, but beamforming weights and CSI are inherently complex-valued. A beamforming transformer uses complex-valued attention, where queries, keys, and values are complex tensors. The dot-product attention score is computed using the Hermitian inner product (conjugate transpose) to preserve phase relationships. Activation functions like modReLU or complex GELU are applied separately to magnitude and phase. This native complex processing prevents the information loss that occurs when naively splitting IQ components into separate real channels.

Wireless channels exhibit structure at multiple scales: fast temporal fading, frequency selectivity across subcarriers, and spatial correlation across antennas. A beamforming transformer processes this hierarchically using multi-head attention with different heads attending to different subspaces. Some heads may focus on wideband spatial covariance, while others capture per-subcarrier phase rotations. This is often combined with a hierarchical architecture where lower layers process per-subcarrier features and upper layers aggregate across the full bandwidth, mimicking the multi-resolution analysis of traditional codebook-based beamforming.

A beamforming transformer trained on a specific antenna array geometry (e.g., a 64-element uniform linear array) can generalize to different array sizes and configurations without retraining. This is enabled by treating each antenna element as an independent token and using relative positional encodings based on the physical antenna spacing in wavelengths. The self-attention mechanism learns to weight interactions based on this relative distance, allowing the model to adapt to arbitrary array layouts. This zero-shot transfer capability is a significant advantage over traditional algorithms that require explicit array manifold calibration.

Practical beamforming systems must satisfy hardware constraints such as total power budgets, per-antenna amplitude limits, and constant-modulus requirements. Beamforming transformers incorporate these constraints directly into the training process. A projection layer at the output normalizes the predicted weights to satisfy the power constraint. For constant-modulus constraints, the final layer outputs only phase angles, with unit magnitude enforced by construction. Alternatively, the constraint can be added as a Lagrangian penalty term in the loss function, allowing the model to learn a near-optimal solution that respects hardware limitations.

A specialized transformer designed to process raw Channel State Information (CSI) matrices as input tokens. It leverages multi-head self-attention to capture complex spatial and frequency correlations across massive antenna arrays. By learning a compressed latent representation of the propagation environment, it enables superior channel estimation and feedback compression, directly feeding a downstream beamforming predictor with highly refined spatial signatures.

An extension of the standard attention mechanism that operates natively in the complex domain. Unlike real-valued networks that process IQ components separately, this preserves the critical magnitude and phase relationships inherent in baseband signals. The attention scores and weighted sums are computed using complex arithmetic, allowing the model to learn more expressive and physically meaningful representations for beamforming weight prediction.

A discrete, learnable token that represents an individual multipath component of a wireless channel. Each token encodes the physical characteristics of a single path: its delay, Doppler shift, and complex gain. By feeding a set of these tokens into a transformer, the model can process the channel as an unordered set of paths, using self-attention to model interactions between scatterers for predictive beamforming.

An attention mechanism that simultaneously models dependencies across both spatial dimensions (antenna elements) and temporal dimensions (symbol periods). This unified processing allows the transformer to learn correlations between how signals evolve over time and how they are distributed across an array. It is critical for beamforming in high-mobility scenarios where the channel changes rapidly.

An end-to-end learned neural receiver architecture that replaces the entire traditional processing chain. It uses a unified deep learning model to perform joint spatial and temporal processing for detection, which includes implicit channel estimation, equalization, and beamforming. This demonstrates the convergence of reception and beamforming into a single, optimized neural network.

The application of Rotary Position Embedding (RoPE) to RF signal tokens. RoPE encodes relative positional information through rotation in the complex plane, which is mathematically elegant for complex-valued signal representations. It allows the beamforming transformer to understand the relative timing or frequency offset between signal samples without requiring absolute position to be explicitly learned.