ns-3 Gym

Implements the standard OpenAI Gym interface (step(), reset(), observation_space, action_space) directly within the ns-3 simulation environment. This allows researchers to plug in any Gym-compatible RL algorithm—such as Stable-Baselines3, RLlib, or TF-Agents—without writing custom integration code. The agent interacts with the network simulation as if it were a standard RL environment, receiving observations and issuing actions at each discrete time step.

Leverages pybind11 to create a high-performance, low-latency bridge between the Python-based RL agent and the C++ ns-3 simulation core. Key characteristics:

• Zero-copy data sharing where possible to minimize overhead
• Direct access to ns-3's full C++ API from Python scripts
• Custom gym::Ns3GymEnv base class for defining environment logic
• Enables real-time or faster-than-real-time simulation depending on scenario complexity

Provides a flexible API for defining arbitrary observation and action spaces tailored to specific wireless research problems. Observations can include:

• PHY-layer metrics: SINR, RSRP, CQI reports
• MAC-layer state: buffer occupancy, queue length, retransmission counts
• Network-level data: throughput, latency, connected UEs

Actions map to controllable ns-3 parameters such as transmission power, MCS index, or handover thresholds, supporting both discrete and continuous action spaces.

Aligns the RL agent's decision cycle with ns-3's discrete-event simulation engine. Each step() call advances the simulation by a configurable time interval (e.g., one Transmission Time Interval in LTE/5G). During this interval, the full protocol stack executes—packet scheduling, HARQ retransmissions, channel fading updates—before returning the next observation and reward. This ensures that RL agents learn policies grounded in realistic, event-driven network dynamics rather than simplified abstractions.

Supports multi-agent reinforcement learning (MARL) scenarios where multiple RL agents control different network elements simultaneously. Each agent can manage a separate base station, cell, or user equipment, with observations scoped to local information. The framework also integrates with ns-3's distributed simulation capabilities via MPI, enabling large-scale network topologies to be partitioned across compute nodes while maintaining a unified Gym interface for centralized training with decentralized execution (CTDE).

Built-in support for deterministic, reproducible experiments essential for academic research:

• Fixed random seeds for both ns-3's stochastic models (fading, mobility) and the RL algorithm
• Automatic logging of episode rewards, convergence metrics, and network KPIs
• Integration with Weights & Biases and TensorBoard for real-time training visualization
• Standardized scenario definitions (e.g., 3GPP urban macro) for fair algorithm comparison across publications