Bounded Randomization

The first step in bounded randomization is to define the parameter space—the set of all simulation variables that can be varied. This includes physics parameters (mass, friction, actuator strength), visual parameters (textures, lighting, camera noise), and environmental parameters (object positions, terrain properties). Bounds are then applied to each parameter based on empirical measurements or physical laws to ensure sampled values remain within a plausible real-world distribution. For example, a robot arm's joint friction might be bounded between 0.05 N·m·s/rad and 0.15 N·m·s/rad based on datasheet specifications and real-world calibration.

The core principle is to enforce physically plausible limits on randomization. This prevents the policy from learning in unrealistic edge cases that would never occur in reality, which can degrade robust policy performance. Bounds are derived from:

• System Identification: Calibrating simulation models using real-world sensor data to establish baseline parameter values and uncertainty ranges.
• First Principles: Using known physical laws (e.g., gravity = 9.81 m/s², coefficient of friction < 1).
• Hardware Specifications: Manufacturer datasheets for motors, sensors, and materials.
• Empirical Observation: Measuring real-world variability under different conditions (temperature, wear).

Bounded randomization navigates a key trade-off. Unbounded randomization can create a vast training distribution, potentially increasing out-of-distribution (OOD) robustness, but risks including non-physical scenarios that teach incorrect dynamics. Tightly bounded randomization ensures high realism but may not cover the full spectrum of real-world variability, leading to domain shift. The optimal bound is often found iteratively, starting with conservative estimates based on system ID and gradually expanding until sim2real success rate plateaus or degrades.

Automatic Domain Randomization (ADR) is an advanced technique that algorithmically determines and expands these bounds during training. It starts with initial, safe bounds and progressively pushes them outward in directions where the policy's performance is weak. This automates the search for the worst-case domain within a growing but still physically constrained parameter space. ADR ensures the policy is challenged by increasingly difficult, yet plausible, scenarios, leading to more efficient and robust training than static bounded randomization.

In practice, bounded randomization is implemented within physics simulation engines like NVIDIA Isaac Sim, PyBullet, or MuJoCo. Parameters are sampled each episode from defined distributions (e.g., uniform, Gaussian) with hard min/max limits. Example code snippet for randomizing gravity in PyBullet:

pythonCopy
# Define bounds for gravity randomization
gravity_lower = [-0.5, -0.5, -9.81]  # x, y, z
gravity_upper = [ 0.5,  0.5, -11.81]
# Sample within bounds
random_gravity = [np.random.uniform(l, u) for l, u in zip(gravity_lower, gravity_upper)]
p.setGravity(*random_gravity)

This controls the randomization distribution for a key physics parameter.

The ultimate test of effective bounded randomization is real-world validation. A policy trained with bounded randomization is deployed on physical hardware in a zero-shot transfer setting. Its performance is compared against policies trained with unbounded randomization or no randomization. Key metrics include task success rate, stability, and adaptation time. Successful bounded randomization should yield policies that work immediately on real robots, demonstrating high sim-to-real transfer performance without exhibiting behaviors that exploit simulation inaccuracies.

Training a policy to grasp objects with a robotic arm. Bounded randomization is applied to key physics parameters:

• Object mass: Varied between 0.05 kg and 0.5 kg (plausible for small tools).
• Surface friction: Varied between 0.3 and 0.8 (typical for plastic/metal).
• Actuator strength: Varied ±10% around the nominal motor torque. Unbounded randomization could create scenarios with zero-gravity objects or impossibly high friction, leading to policies that fail on real hardware. Bounding ensures the policy learns robust strategies for the real world's physical limits.

Training a vision-based object detector for a self-driving car in simulation. Visual randomization is bounded to plausible real-world conditions:

• Lighting intensity: Varied between 500 lux (dawn) and 100,000 lux (bright sun).
• Rain/snow particle density: Capped at levels that maintain sensor visibility.
• Object texture colors: Varied within the RGB color space, but not to neon or impossible hues. This prevents the model from learning to rely on unrealistic visual artifacts (e.g., pitch-black nights or fluorescent green cars) that don't exist in the target domain, ensuring reliable perception.

Training a quadruped robot to walk on varied terrain. Physics randomization is carefully bounded to the robot's physical capabilities and real-world ground properties:

• Ground friction: Bounded between 0.4 (grass) and 1.2 (rough concrete).
• Payload mass: Varied up to 20% of the robot's weight (its rated capacity).
• Motor latency: Varied between 5ms and 50ms (envelope of real servo delays). If joint damping or motor torque were randomized to extreme values, the policy might learn gaits that over-stress and break real actuators. Bounding protects hardware during deployment.

Training a system to pick randomly oriented parts from a bin. Domain randomization for this contact-rich task is bounded by material science and geometry:

• Part dimensions: Varied within manufacturing tolerances (e.g., ±0.5mm).
• Part material density: Varied within known values for aluminum or steel.
• Bin wall friction: Set to ranges measured from real bin surfaces. Unbounded variation in part size or weight could lead to grasp strategies that are impossible or unsafe for the real gripper, causing jams or damage. Bounds ensure the policy generalizes to real production variance.

Training a drone policy for stable flight in windy conditions. Dynamic randomization is bounded by aerodynamics and sensor models:

• Wind gust magnitude: Capped at 15 m/s (strong storm-level winds).
• IMU noise: Injected noise is bounded by the datasheet specs of a real inertial measurement unit.
• Battery voltage drop: Simulated drop is bounded by the discharge curve of a real LiPo battery. Randomizing wind speeds to hurricane-force levels or simulating IMU noise an order of magnitude too high would train a policy that is overly conservative or unstable in normal conditions. Bounds keep training relevant.

Bounded randomization is explicitly contrasted with its unbounded counterpart to highlight its engineering purpose.

Unbounded Randomization Example:

• Varying object friction from 0 to 10.
• Varying gravity from -5 m/s² to 20 m/s².
• This can create catastrophic forgetting or policies that exploit simulation quirks.

Bounded Randomization Solution:

• Define bounds using system identification data, datasheets, or physical first principles.
• The bounds form a plausible parameter space that encapsulates real-world variability without including physically impossible scenarios.
• This focuses the policy's learning on the challenging, but realistic, part of the domain shift.

Domain Randomization is the foundational sim-to-real transfer learning technique where a policy is trained in a simulation with randomly varied parameters (e.g., physics, visuals, lighting) to force it to learn robust, generalizable behaviors. The core hypothesis is that exposing the model to a sufficiently broad distribution of simulated experiences will cover the real-world distribution, enabling zero-shot transfer. It treats the reality gap as a domain adaptation problem solved through synthetic data diversity.

Automatic Domain Randomization (ADR) is an algorithmic extension that dynamically expands the randomization distribution during training. Instead of using a fixed range, ADR automatically increases the difficulty or breadth of randomized parameters (e.g., object mass, friction coefficients) in response to the policy's performance. This creates a curriculum randomization effect, continuously pushing the policy to adapt to more challenging and diverse scenarios, thereby maximizing out-of-distribution (OOD) robustness.

The randomization distribution is the defined probability distribution (e.g., uniform, Gaussian, log-normal) from which simulation parameters are sampled during domain randomization. It formally specifies the parameter space for variation.

• Bounded distributions (e.g., Uniform[low, high]) are used in bounded randomization to keep parameters within physically plausible limits.
• The choice of distribution and its bounds directly controls the diversity of the training simulation ensemble and is critical for balancing robustness against unrealistic scenarios.

These are the two primary axes of parameter variation in domain randomization.

• Physics Randomization: Varies dynamics parameters like mass, inertia, friction, motor torque limits, and actuator latency. This builds robustness against inaccuracies in the simulated rigid body dynamics and real-world mechanical variation.
• Visual Randomization: Alters appearance parameters like textures, lighting conditions (hue, intensity), camera pose, and background geometry. This improves the robustness of perception pipelines to the domain shift between synthetic and real visual data.

Zero-shot transfer is the primary objective of bounded and domain randomization: deploying a simulation-trained policy directly onto physical hardware without any fine-tuning on real-world data. Success is measured by the Sim2Real success rate. Achieving high zero-shot transfer indicates the policy has learned a robust policy that generalizes across the reality gap. This is distinct from policy transfer and adaptation methods, which involve subsequent real-world fine-tuning.

The reality gap (or simulation-to-reality gap) is the performance drop observed when a model trained in simulation fails on a physical system due to modeling inaccuracies. It is a specific instance of domain shift, a broader machine learning concept where a model's performance degrades due to differences between training (source) and deployment (target) data distributions. Domain randomization techniques, including bounded randomization, are explicitly designed to mitigate this gap by training models to be invariant to a wide range of domain shifts.