Parameter Distribution

The probability distribution from which parameters are sampled defines the shape and spread of variations.

• Uniform: Samples parameters with equal probability across a defined minimum and maximum range (e.g., light_intensity ~ Uniform(100, 500)). Simple and ensures broad coverage.
• Gaussian/Normal: Samples parameters around a mean with a defined standard deviation (e.g., object_mass ~ Normal(1.0, 0.2)). Concentrates variations near typical values while allowing outliers.
• Log-Normal: Used for parameters that must remain positive and have a natural multiplicative spread (e.g., friction coefficients).
• Categorical: For non-numeric parameters, like selecting a texture from a predefined set of assets.

The minimum and maximum limits of the distribution define the extent of environmental variation the model must handle.

• Narrow Bounds: Create a consistent, easier training environment but risk overfitting to the simulation's specific parameters, leading to poor sim-to-real transfer.
• Wide Bounds: Force the model to learn invariant features and policies that work under highly diverse conditions, promoting robustness. However, excessively wide bounds can lead to over-randomization, where the task becomes impossible to learn.
• Asymmetric Bounds: Used when the real-world parameter is known to vary more in one direction (e.g., lighting is more likely to be too dim than too bright).

Defines how the distribution changes over the course of training, which is critical for stable learning.

• Static: Parameters are re-sampled from a fixed distribution at the start of every training episode. Simple but may not optimize learning efficiency.
• Curriculum (Progressive): The range or complexity of randomization is gradually increased. For example, starting with a narrow uniform distribution for object mass and slowly widening it. This helps the model learn a stable policy before facing extreme variations.
• Adaptive/Dynamic: The distribution is adjusted algorithmically based on the model's performance, as seen in Automatic Domain Randomization (ADR), which seeks the 'hardest' useful variations.

Describes whether parameters are varied independently or jointly.

• Independent Sampling: Each parameter (e.g., lighting hue, floor friction) is sampled from its own distribution without regard to others. This is simpler but can generate physically implausible or extremely challenging combinations.
• Correlated/Joint Sampling: Parameters are varied together according to a joint distribution or rules. For example, increasing sun angle might be correlated with increased shadow length and a warmer color temperature. This creates more realistic, coherent environments but requires more careful design.
• Factorized Randomization: A systematic approach where groups of related parameters are randomized together in a controlled manner to ensure broad but plausible coverage of the parameter space.

Determines when and how often parameters are re-sampled during a single training trial.

• Episodic Randomization: All parameters are fixed at the start of an episode (a complete trial from start to terminal state) and held constant until the episode ends. This is most common, teaching the model to adapt to a consistent, if randomized, environment.
• Within-Episode (Dynamic) Randomization: Parameters can change stochastically during a single episode. For example, lighting could gradually shift from day to night, or wind speed could change suddenly. This trains models for robustness to non-stationary environments, a key requirement for long-duration real-world deployment.

A strategic use of parameter distributions to deliberately compensate for low simulation fidelity.

• Instead of trying to perfectly match real-world parameters (high fidelity), the distribution is designed to be broader and more diverse than expected in reality. This forces the model to rely on task-relevant features rather than simulation-specific artifacts.
• For example, a low-fidelity texture renderer might use wildly varied colors and unrealistic material shaders. The model learns to recognize objects by shape, not by a specific texture pattern, which transfers better to real objects.
• This characteristic is the philosophical core of Domain Randomization: using structured variation to bridge the reality gap caused by imperfect simulators.

Domain Randomization is the overarching simulation-based training technique where a model is exposed to a wide variety of randomized environments during training. The Parameter Distribution defines the statistical range for these randomizations. The goal is to force the model to learn features or policies that are invariant to these superficial changes, thereby improving robustness and enabling sim-to-real transfer.

• Core Mechanism: Systematically varies simulation parameters (e.g., lighting, textures, physics).
• Primary Use Case: Training models in simulation for deployment in the real world, especially in robotics and computer vision.

Parameter Perturbation is the specific act of varying individual simulation parameters. It is the execution mechanism enabled by the Parameter Distribution. While the distribution defines the range and probability of variations, perturbation is the instantiation of a specific sampled value.

• Examples: Changing a light's intensity from 500 to 1500 lumens, altering a surface's friction coefficient from 0.2 to 0.8.
• Relationship to DR: The collective perturbation of many parameters across episodes creates the randomized training domain.

Automatic Domain Randomization is an advanced technique that algorithmically optimizes the Parameter Distribution during training. Instead of using a fixed, manually-tuned distribution, ADR dynamically adjusts the randomization range based on the model's performance, seeking the 'hardest' yet learnable variations.

• Key Benefit: Reduces the need for expert tuning of parameter bounds.
• Process: Starts with a narrow distribution and expands it as the agent learns, effectively creating an adaptive curriculum. It automates the search for a distribution that maximizes robustness.

Curriculum Randomization is a training strategy that defines a Randomization Schedule for the Parameter Distribution. It begins training with a narrow, easy distribution (e.g., small variations in lighting) and progressively expands it to more challenging, wider distributions over time.

• Objective: To prevent over-randomization and learning collapse by allowing the model to master basic tasks before introducing extreme variability.
• Analogy: Similar to teaching a child to walk on flat ground before introducing slopes or obstacles.

Visual Domain Randomization is a specialized application of DR where the Parameter Distribution is applied exclusively to visual and perceptual simulation parameters. This is used to train robust computer vision models.

• Randomized Parameters: Textures, colors, object shapes, lighting conditions (direction, color, intensity), camera properties (noise, blur, position).
• Goal: To learn visual features invariant to appearance, enabling models to recognize objects based on geometry and semantics rather than specific textures or lighting seen in simulation.

Dynamics Randomization is a specialized application of DR where the Parameter Distribution is applied to the physical and dynamical parameters of a simulation. This is critical for training robust robotic control policies.

• Randomized Parameters: Mass, inertia, friction coefficients, motor torque limits, actuator latency, joint damping, and other physics engine properties.
• Goal: To learn control policies that are invariant to the exact physical dynamics of the system, compensating for the reality gap and hardware variations between robots.

Over-Randomization is a critical failure mode in Domain Randomization that occurs when the defined Parameter Distribution is too wide or extreme. This makes the task impossible or the environmental variations so vast that the model cannot discern a coherent policy, leading to degraded or failed learning.

• Symptoms: Training loss fails to converge, policy performance is chaotic.
• Mitigation: Techniques like Curriculum Randomization or Automatic Domain Randomization are used to carefully manage the expansion of the parameter distribution, ensuring it remains within a 'learnable' regime.

A Randomization Pipeline is the software infrastructure that operationalizes Domain Randomization. It automates the process of sampling parameters from the Parameter Distribution, configuring individual simulation instances, and generating the randomized training data or environments for the model.

• Key Components: A Parameter Sampler, a scene/configurator generator, and often a Physics Randomization Engine.
• Function: Ensures that each training episode or batch is instantiated with a unique, valid set of parameters drawn from the specified distributions, enabling scalable and reproducible DR training.