Physics Randomization

Physics randomization operates by sampling key dynamic properties from predefined distributions during each training episode. This creates a broad ensemble of simulated physical worlds. Common parameters include:

• Mass and Inertia: Varying the mass and inertial properties of objects and robot links.
• Friction Coefficients: Randomizing static and dynamic friction between surfaces.
• Actuator Dynamics: Altering motor torque limits, damping, and gear ratios.
• Gravity and External Forces: Applying variations in gravitational force or adding random wind disturbances.
• Object Elasticity and Damping: Changing the restitution (bounciness) and damping of collisions.

Unlike visual randomization, which targets perception, physics randomization is fundamentally concerned with robust control. By experiencing varied dynamics, a policy learns to generate stable motor commands that succeed despite uncertainties in the real system's physical response. This is critical for tasks like:

• Precise Manipulation: Gripping objects with unknown weight or surface friction.
• Dynamic Locomotion: Walking or running on surfaces with varying compliance.
• Aerial or Underwater Control: Stabilizing drones or submersibles in turbulent conditions. The policy learns invariant strategies that are effective across the randomized parameter space.

Physics randomization is intrinsically linked to the problem of system identification (SysID). The reality gap often exists because the simulation's nominal physics model does not match the true physical parameters of the real robot. Randomization addresses this by training the policy to be robust to a range of possible parameter values, which ideally brackets the true values. This turns an identification problem into a robustness problem. The policy does not need to know the exact mass or friction; it needs to work given that these values are within the trained distribution.

A critical design choice is defining the randomization bounds. Bounded randomization samples parameters within physically plausible limits (e.g., friction between 0.2 and 1.0). This is standard for most applications. Unbounded or extreme randomization intentionally samples from very wide or unrealistic distributions to force the policy to develop ultra-robust, often conservative, behaviors. The trade-off is that policies trained with overly broad randomization may exhibit cautious or sub-optimal performance in the nominal real-world case. The bounds define the robustness envelope of the final policy.

Effective physics randomization leverages massively parallel simulation. Platforms like NVIDIA Isaac Sim or OpenAI's MuJoCo wrappers allow thousands of simulation instances (environments) to run concurrently, each with a different set of randomized physics parameters sampled per episode or per environment. This parallelization is essential because:

• It provides the policy with diverse experience at scale within a single training batch.
• It makes the expected gradient over the randomization distribution computationally tractable.
• It enables techniques like Automatic Domain Randomization (ADR), which dynamically expands parameter bounds based on policy performance.

It is crucial to distinguish physics randomization from visual randomization, though both are subsets of domain randomization. Visual randomization alters appearance: textures, lighting, colors, and camera properties. It trains perceptual robustness. Physics randomization alters dynamics: mass, friction, forces, and motor models. It trains control robustness. A complete sim-to-real pipeline often employs both simultaneously: the policy must learn to interpret varied visual inputs and execute precise control under varied dynamics, leading to holistic robustness for real-world deployment.

Domain Randomization (DR) is the overarching sim-to-real technique where a wide range of simulation parameters—including physics, visuals, and sensors—are randomly varied during training. The goal is to expose the policy to such a diverse set of virtual experiences that the real world appears as just another variation, enabling zero-shot transfer. It treats the reality gap as a problem of distributional robustness.

• Core Principle: Create a broad, randomized parameter space (e.g., object mass, friction, lighting, textures) so the policy cannot overfit to simulation artifacts.
• Contrast with System ID: Unlike system identification, which seeks to make the simulation more accurate, DR accepts inaccuracy and trains for robustness across inaccuracies.

Automatic Domain Randomization (ADR) is an algorithmic extension of standard DR that dynamically and automatically expands the range of randomized parameters during training. It starts with a narrow parameter distribution and progressively pushes the boundaries whenever the policy achieves mastery, actively searching for the worst-case domain within the expanding range to maximize robustness.

• Key Mechanism: Uses an adversarial process where one component proposes increasingly challenging simulation parameters, and the policy must adapt to succeed.
• Benefit: Reduces the need for manual tuning of randomization distributions and can discover effective randomization ranges that human engineers might not anticipate.

Visual Randomization is a subset of domain randomization focused specifically on varying the perceptual inputs to a policy. It alters non-physical rendering parameters to improve the robustness of vision-based policies, making them invariant to real-world visual noise.

• Randomized Elements: Includes textures, colors, lighting conditions (position, intensity, color), camera parameters (gain, noise, distortion), and background scenes.
• Purpose: Prevents the policy from relying on "cheating" via simulation-specific visual cues (e.g., perfect edges, uniform textures). It forces the learning of semantically relevant features, improving performance under varying real-world lighting and camera conditions.

System Identification (SysID) is the complementary approach to domain randomization for closing the reality gap. It involves empirically measuring a physical system's dynamics (e.g., robot inertia, actuator response, friction coefficients) and using that data to calibrate the simulation model, thereby increasing its simulation fidelity.

• Contrast with DR: While DR embraces inaccuracy, SysID seeks to minimize it. The techniques are often used in tandem: a SysID-calibrated simulation provides a more accurate base, upon which DR adds bounded randomization to account for residual uncertainty and wear.
• Process: Involves collecting real-world input-output data and optimizing simulation parameters to match the observed dynamics.

Domain-Adversarial Training is an alternative, gradient-based technique for achieving domain generalization. Instead of explicitly randomizing the source domain (simulation), it uses an adversarial network to encourage the learning of features that are indistinguishable between the source (sim) and target (real) domains.

• Mechanism: A feature extractor learns to generate representations, while a domain classifier tries to predict if features came from simulation or real data. The feature extractor is trained to fool this classifier.
• Comparison to DR: It learns domain-invariant features implicitly, whereas DR provides explicit, broad exposure. It often requires some real-world data, unlike the purely simulation-based DR approach for zero-shot transfer.

The Reality Gap (or Simulation-to-Reality Gap) is the fundamental problem that physics randomization aims to solve. It is the performance discrepancy between a model trained in simulation and its performance when deployed on a physical system, caused by inevitable inaccuracies in the simulation model.

• Sources of the Gap: Includes unmodeled contact and rigid body dynamics, simplified actuator models, imperfect sensor noise models, and missing environmental disturbances.
• Bridging Strategies: The gap is addressed by a spectrum of techniques, from high-fidelity simulation and system identification on one end, to domain randomization and domain-adversarial training on the other. The choice involves a simulation fidelity trade-off based on compute cost and data availability.