Randomization Schedule

Static Randomization uses a fixed, broad distribution of parameters from the very start of training. Parameters are sampled independently for each training episode or batch from this unchanging distribution.

• Mechanism: A uniform or Gaussian distribution is defined for each parameter (e.g., lighting intensity: 100-1000 lux, object mass: 0.5-2.0 kg). Every new simulation rollout uses a fresh, independent sample.
• Objective: To immediately expose the model to maximal environmental diversity, forcing it to learn a policy or features that are invariant across the entire specified range.
• Use Case: Effective for tasks where the target real-world domain is highly variable and poorly characterized, or when using very high-capacity models that can absorb the diversity from the outset.

Dynamic Randomization adjusts the parameter distribution during training based on the model's current performance or other metrics. The schedule evolves in response to the learning process.

• Mechanism: The range or center of parameter distributions is modified algorithmically. A simple form increases variance as training progresses. More advanced forms, like Automatic Domain Randomization (ADR), actively search for parameters on which the model fails and adds them to the distribution.
• Objective: To optimize training efficiency by focusing computational resources on the most challenging or informative parts of the parameter space, avoiding wasted effort on trivial or impossible variations.
• Example: ADR in robotics might start randomizing arm length only after the policy masters picking with a default length, systematically discovering and training on failure cases.

Curriculum Randomization follows a planned progression from easy to hard, deliberately structuring the schedule to guide the learning process. It is a principled form of dynamic scheduling.

• Mechanism: Training begins with a narrow, deterministic, or minimally randomized environment (e.g., fixed lighting, perfect physics). The randomization breadth or difficulty is gradually increased according to a predefined schedule or performance threshold.
• Objective: To provide a stable learning foundation, preventing early training instability or collapse that can occur under extreme randomization. It mirrors pedagogical concepts of scaffolding.
• Key Benefit: Often leads to faster convergence and more stable training than static randomization, especially for complex tasks, by avoiding the over-randomization pitfall early on.

This approach applies randomization not just across training steps, but within individual training episodes or trajectories according to a temporal schedule. Parameters change over simulated time.

• Mechanism: Instead of sampling parameters once per episode, they are varied according to a function of time (e.g., sinusoidal changes in lighting, linearly increasing friction). This creates non-stationary environments within a single rollout.
• Objective: To train models that are robust to temporal variations and can adapt their behavior in real-time, which is crucial for long-horizon tasks in dynamically changing real-world conditions.
• Application: Essential for training policies for drones or autonomous vehicles that must operate reliably from day to night or in gradually changing weather conditions simulated within a single flight.

This distinction defines the structure of the parameter space exploration, which is a core aspect of the schedule's design.

• Independent Sampling: Each parameter (e.g., texture, lighting, gravity) is varied independently. This is computationally simple and can cover a vast combinatorial space, but may generate many unrealistic or incoherent environments (e.g., zero gravity with high ground friction).
• Systematic Sampling: Parameters are varied in a structured, often factorized way to ensure coverage of all combinations within a defined set. This can involve grid searches over parameter ranges or using Latin Hypercube sampling for better space-filling properties.
• Engineering Trade-off: Systematic schedules guarantee broad coverage and are easier to debug but are more computationally expensive. Independent schedules are cheaper but may leave gaps or produce implausible scenarios.

HIL Randomization is a specialized schedule where the randomized simulation runs in real-time, interfacing directly with physical hardware controllers or sensors. The schedule must account for real-time constraints.

• Mechanism: A physical robot's controller receives commands and sensor feedback from a rapidly randomized simulation (e.g., varying joint friction every few seconds). This creates a closed loop between the real actuator/sensor hardware and a virtual, changing world.
• Objective: To train or adapt policies on real hardware while leveraging the safety and speed of simulation, and to identify hardware-specific failure modes (e.g., latency, calibration errors) under varied conditions.
• Critical Constraint: The randomization schedule must operate at the control frequency of the hardware (often 100-1000 Hz), requiring highly optimized randomization pipelines and physics randomization engines.

A Static Randomization Schedule samples parameters from a fixed, pre-defined distribution throughout the entire training process. This is the most common baseline approach.

• Mechanism: At the start of each training episode or batch, parameters (e.g., lighting hue, object mass, floor friction) are sampled independently from uniform or Gaussian distributions.
• Use Case: Foundational for initial Domain Randomization experiments. Used in OpenAI's Dactyl project to train a robotic hand to manipulate objects by randomizing colors, textures, and lighting in MuJoCo.
• Limitation: The fixed range must be manually tuned; if set too narrow, the model may not generalize, but if set too wide (Over-Randomization), learning can fail.

A Dynamic Randomization Schedule algorithmically adjusts the parameter distribution based on the model's current performance or learning progress.

• Mechanism: The schedule uses metrics like task success rate or policy entropy to decide whether to expand the randomization range, shift its mean, or focus on challenging parameters. Automatic Domain Randomization (ADR) is a prime example, where a secondary controller searches for parameters that maximize the training policy's learning signal.
• Use Case: NVIDIA's DRIVE Sim uses adaptive schedules to vary weather conditions and traffic density for autonomous vehicle training, focusing computational resources on scenarios where the perception model is underperforming.
• Advantage: Reduces manual tuning and can systematically discover a Parameter Distribution that optimally bridges the Reality Gap.

A Curriculum Randomization Schedule progressively increases the difficulty or diversity of randomized parameters, following a "easy-to-hard" learning trajectory.

• Mechanism: Training begins with minimal or no randomization (e.g., canonical simulator settings). The schedule then gradually expands the range of parameters (e.g., increasing max object weight, adding more visual clutter) as the model's proficiency improves.
• Use Case: Training robotic grasping policies. Start with fixed lighting and simple object shapes, then slowly introduce random shadows, varied textures, and deformable objects. This method is central to Curriculum Learning applied to Domain Randomization.
• Benefit: Stabilizes early training and can lead to better final Sim2Real Performance by preventing the model from being overwhelmed initially.

A Systematic Randomization Schedule applies randomization in a controlled, often factorized manner to ensure comprehensive coverage of the parameter space.

• Mechanism: Instead of randomizing all parameters simultaneously every episode, the schedule cycles through them in a structured way. For example, one training run might vary only lighting conditions, the next only textures, and a third only physics properties.
• Use Case: In Sim-to-Real Transfer for drone navigation, a schedule might systematically vary wind models, camera noise profiles, and GPS dropout rates in separate phases to isolate and learn robustness to each disturbance.
• Advantage: Improves interpretability by revealing which parameter variations most impact performance, aiding in Simulation Fidelity prioritization.

Randomization Schedules are often specialized for either Visual Domain Randomization or Dynamics Randomization, as they target different parts of the Domain Gap.

• Visual Schedule: Governs parameters like HDRi skybox textures, object albedo, camera gain, and post-processing effects (blur, noise). Schedules may correlate parameters—e.g., sunset lighting with warmer object tones.
• Dynamics Schedule: Governs physical parameters like actuator latency, joint damping, center-of-mass offsets, and surface friction. Schedules may need to respect physical plausibility (e.g., correlation between mass and inertia).
• Application: A vision-based picking robot might use a heavy visual schedule early in training to achieve Invariant Feature Learning, then introduce a dynamics schedule later to fine-tose motor control.

In production, the schedule is executed by a Randomization Pipeline, a software framework that automates the training loop.

• Components: The pipeline consists of a Parameter Sampler (guided by the schedule), a Simulation Configurator (e.g., a Physics Randomization Engine), and a Data Logger.
• Workflow: 1. The schedule dictates the parameter bounds for the current training iteration. 2. The sampler draws specific values. 3. The configurator applies them to the simulator (e.g., Unity, Isaac Sim). 4. The agent collects experience. 5. Performance metrics are fed back to potentially update the schedule.
• Example: Hardware-in-the-Loop (HIL) Randomization uses a real-time pipeline where a physical robot controller receives commands and sensor feedback from a rapidly re-randomized simulation, enabling safe stress-testing of policies.

Domain Randomization is the overarching simulation-based training technique. A Randomization Schedule is the specific implementation plan for this technique. DR works by:

• Varying simulation parameters (e.g., lighting, textures, physics) across a wide range.
• Forcing the model to learn features or policies that are invariant to these changes.
• The schedule dictates how, when, and to what extent these parameters are randomized.

The Parameter Distribution defines the statistical range from which individual simulation parameters are sampled. It is a core component specified by the Randomization Schedule.

• Common distributions include uniform, Gaussian (normal), or custom bounded ranges.
• For example, a schedule might specify that object mass is sampled from a uniform distribution between 0.5 kg and 2.0 kg.
• The schedule may dynamically adjust the distribution's mean or variance over training time.

Curriculum Randomization is a specific type of schedule that follows a learning progression. Instead of fixed or uniformly random parameters, it systematically increases difficulty.

• Starts Easy: Initial training uses a narrow, easy parameter distribution (e.g., consistent lighting).
• Progressively Hardens: The schedule gradually expands the randomization range or introduces new randomized parameters.
• This scaffolds learning, helping the model master basic skills before facing high variation, often leading to more stable and effective training.

Automatic Domain Randomization is an advanced technique that automates the creation of the Randomization Schedule. Instead of manual tuning, ADR uses an algorithm to:

• Search the parameter space for the most challenging yet learnable variations.
• Adapt the schedule in real-time based on the agent's current performance.
• Concentrate randomization on parameters where the model is overfitting, optimizing for robustness without manual intervention. It represents a shift from a static to a self-adjusting schedule.

The Randomization Pipeline is the software framework that executes the Randomization Schedule. It is the engine that brings the schedule to life during training.

• Automates the workflow: Samples parameters from the defined distributions, configures the simulator, launches training episodes, and collects data.
• Ensures reproducibility by managing random seeds and parameter logs.
• A robust pipeline is essential for implementing complex schedules, especially dynamic or curriculum-based ones, at scale.

Over-Randomization is a critical failure mode that a poorly designed Randomization Schedule can cause. It occurs when parameter variations are too extreme.

• Consequences: The task may become impossible (e.g., randomizing gravity to near-zero), or the model fails to learn any coherent policy due to a lack of consistent signal.
• A key goal of schedule design is to balance diversity with learnability. Techniques like curriculum randomization or ADR are explicitly designed to mitigate this risk by carefully controlling the difficulty progression.