Joint Spatio-Temporal Attention

Unlike cascaded architectures that apply spatial beamforming followed by temporal equalization, joint attention computes attention weights across both dimensions simultaneously. A received symbol at antenna m and time t can attend directly to a symbol at antenna n and time t-k, capturing cross-dimensional dependencies that sequential processing misses.

• Eliminates information bottlenecks between decoupled spatial and temporal stages
• Learns correlations induced by moving scatterers and time-varying channels
• Particularly effective in high-mobility MIMO scenarios where spatial signatures change rapidly

To manage the quadratic complexity of attending over all space-time positions, factorized attention splits the operation into spatial heads and temporal heads, or uses axial attention that applies spatial and temporal attention sequentially within each block.

• Spatial heads: Attend across antenna elements at a fixed time step to learn instantaneous beamforming patterns
• Temporal heads: Attend across time steps for a fixed antenna to model inter-symbol interference
• Full joint heads: A subset of heads may attend across the full space-time grid for critical global dependencies
• Reduces computational complexity from O((S×T)²) to O(S² + T²) where S is spatial dimension and T is temporal length

Raw IQ samples are tokenized into complex-valued embeddings that preserve both magnitude and phase information. Each token represents a space-time position with a learned projection that encodes the antenna index and temporal position into a unified representation.

• Tokens are formed by concatenating real and imaginary components or using complex linear layers
• Rotary Position Embedding (RoPE) is applied natively in the complex plane to encode relative temporal offsets
• Spatial position encoding uses learned antenna index embeddings that capture array geometry
• Preserves the Hermitian symmetry properties inherent in baseband signals

The architecture employs cross-attention between pilot symbols and data symbols, allowing the model to implicitly perform channel estimation and interpolation without an explicit estimation stage. Pilot tokens serve as queries that gather channel state information from the received signal.

• Pilot positions attend to nearby data symbols to propagate channel estimates
• Data symbols attend to pilot tokens to incorporate CSI context into detection
• Eliminates the need for separate MMSE or LS estimation blocks
• Naturally handles irregular pilot patterns and non-uniform array geometries

For real-time applications, a causal attention mask restricts each space-time token to attend only to past and present positions. This enables the model to operate in a streaming fashion without requiring future samples, critical for low-latency physical-layer processing.

• Triangular masking matrix prevents information leakage from future time steps
• Spatial attention within the same time step remains fully connected
• Enables sample-by-sample or symbol-by-symbol online detection
• Compatible with autoregressive decoding for learned channel coding applications

Joint spatio-temporal attention serves as the core processing block in DeepRx MIMO architectures, replacing the entire traditional receiver chain—channel estimation, equalization, spatial combining, and demapping—with a single neural network trained end-to-end.

• Trained on synthetic channel models (e.g., CDL, TDL) with diverse propagation conditions
• Loss function directly optimizes bit error rate (BER) or mutual information
• Outperforms LMMSE-IRC receivers in interference-limited and high-mobility scenarios
• A single model generalizes across varying MIMO configurations and modulation schemes