Parameter Space

The dimensionality of a parameter space refers to the number of independent, variable parameters it contains. A high-dimensional space (e.g., varying mass, friction, motor torque, lighting, textures) creates a vast set of possible simulation instances, forcing the policy to learn more generalizable features. However, it also increases the sample complexity of training.

• Example: A simple robotic arm simulation might have a 5D parameter space (link mass, joint damping, actuator strength, gravity, object friction).
• Trade-off: Higher dimensionality can improve robustness but requires more training episodes to adequately explore the space.

The bounds of a parameter space define the minimum and maximum values for each continuous parameter, while its support describes the set of all possible values (including discrete choices). Setting physically plausible bounds is critical; overly broad bounds can generate unrealistic dynamics that hinder learning, while overly narrow bounds fail to cover real-world variation.

• Bounded Randomization: A standard practice where parameters are sampled from distributions (e.g., uniform, Gaussian) within defined intervals.
• Example: Surface friction might be bounded between 0.2 (slippery) and 1.5 (sticky), based on real-world material properties.

Parameters within the space are often interdependent, meaning changing one affects the behavior or valid range of another. Ignoring these correlations can create non-physical simulation states.

• Example: Increasing an object's mass should typically increase its inertia tensor proportionally. Randomizing them independently breaks physical consistency.
• Engineering Implication: Effective domain randomization requires understanding and modeling these relationships, often through grouped parameter sampling or the use of structured randomization distributions.

This is a core design tension. Fidelity refers to how accurately a single simulation instance mimics reality. Coverage refers to the breadth of the parameter space and the diversity of scenarios it generates.

• High-Fidelity, Narrow Coverage: A precise simulation of one specific robot in one lighting condition. Transfers poorly to any change.
• Lower-Fidelity, Broad Coverage: A less physically accurate simulation that randomizes many parameters broadly. Often yields more robust policies that generalize to the messy real world.
• Automatic Domain Randomization (ADR) algorithms dynamically manage this trade-off, expanding the parameter space bounds as the policy masters current challenges.

A parameter space contains both continuous parameters (e.g., lighting intensity, coefficient of friction) and discrete parameters (e.g., number of obstacles, texture asset selection, sunny/cloudy condition).

• Continuous Parameters: Sampled from real-valued distributions. Create smooth variations in the environment.
• Discrete Parameters: Sampled from a set of categorical choices. Introduce structural or qualitative changes.
• Combined Use: A comprehensive domain randomization strategy employs both types. For instance, randomizing the continuous mass of a box and the discrete shape of the box (cube, cylinder, sphere).

Parameters are logically grouped by the aspect of the simulation they affect. Common groups include:

• Physics Parameters: Mass, friction, restitution, motor gains, joint limits, gravity.
• Visual Parameters: Texture, color, lighting (direction, intensity, color), camera parameters (noise, distortion), background.
• Dynamic Parameters: Force/torque disturbance magnitude, object spawn locations, wind velocity.
• Sensor Parameters: Noise models (Gaussian, dropout) for IMU, LiDAR, and camera readings.

Grouping allows for targeted randomization strategies, such as applying aggressive visual randomization for a vision-based policy while keeping physics parameters more stable.

The randomization distribution is the specific probability function (e.g., uniform, Gaussian, log-normal) from which values for a parameter are sampled during training. It defines the likelihood of encountering different values within the parameter space.

• Uniform Distribution: Often used for parameters like color or texture, where all values within a range are equally probable.
• Truncated Normal Distribution: Used for physical parameters like mass or friction, centering variation around a nominal value while excluding physically implausible extremes.
• The choice of distribution directly impacts the policy's robustness and the efficiency of training.

Bounded randomization is the practice of constraining the variation of simulation parameters within physically plausible or operationally safe limits. It prevents the policy from learning in unrealistic or dangerous parts of the parameter space.

• Example: Randomizing a robot's joint friction coefficient between 0.05 and 0.5 N·m·s/rad, based on real motor specifications, rather than from 0 to 10.
• This technique ensures the randomized training distribution remains relevant to the target domain (the real world), improving the likelihood of successful zero-shot transfer.

Physics randomization is a core subset of domain randomization focused on varying the parameters of a simulation's dynamics engine. It targets the parameter space governing physical interactions.

• Key Parameters: Mass, inertia, friction coefficients, restitution (bounciness), actuator strength and latency, motor constants, and gear ratios.
• The goal is to create policies robust to the inevitable inaccuracies in any physics model, making them tolerant of the reality gap in dynamics.

Visual randomization alters the parameters controlling the visual appearance of a simulation to improve the robustness of perception-based policies. It creates a diverse visual parameter space.

• Randomized Attributes: Object textures, colors, material reflectivity, lighting conditions (position, color, intensity), camera parameters (gain, exposure, lens distortion), and background scenery.
• This technique helps models become invariant to visual domain shift, such as changes in lighting between a lab and a warehouse.

Curriculum randomization is a training strategy that dynamically adjusts the parameter space during learning. It starts with a narrow, easy distribution of parameters and progressively expands it to more challenging ranges.

• Process: Begin training with low variance (e.g., small mass changes). As the policy masters these conditions, gradually widen the randomization bounds.
• This approach can lead to faster convergence and more stable learning than immediately training on the full, complex parameter space.

Within a defined parameter space, the worst-case domain refers to the specific combination of randomized parameters that most severely degrades the policy's performance. Identifying it is key for robust optimization.

• In techniques like adversarial domain randomization, the simulator actively searches for these challenging parameter settings to train against.
• Strengthening the policy against its worst-case domain within the training space generally improves its out-of-distribution (OOD) robustness to unseen real-world conditions.