Normalized Score

The core function of a normalized score is to express performance relative to defined baseline policies. A common formulation is:

Normalized Score = (Policy Score - Random Policy Score) / (Expert Policy Score - Random Policy Score)

• A score of 0.0 indicates performance equivalent to a random policy.
• A score of 1.0 indicates performance matching an expert or optimal policy.
• Scores can be negative (worse than random) or exceed 1.0 (surpassing the expert baseline). This scaling neutralizes the arbitrary magnitude of environment-specific reward functions.

Normalized scores are essential for benchmark suites like Meta-World or the DM Control Suite, which contain tasks with incommensurate reward scales. For example, comparing a raw score of +250 from a door-opening task to a score of -15 from a bipedal walking task is meaningless. After normalization, both scores are expressed on the same 0-to-1 scale, allowing researchers to compute an aggregate mean score across the entire suite to evaluate generalist policies. This is critical for assessing out-of-distribution (OOD) generalization.

Normalized scores provide an intuitive, human-interpretable gauge of progress. A score of 0.75 immediately conveys that a policy achieves 75% of the way from random to expert performance. This is more actionable than monitoring raw cumulative reward. It allows engineering managers to track improvement across training epochs in simulation and, crucially, to quantify the sim-to-real gap by comparing normalized scores from simulation evaluation versus real-world episodes. A significant drop indicates a large reality gap or distribution shift.

Proper calculation requires careful handling of baseline scores, which are often stochastic. Best practice involves:

• Running the random policy for a large number of episodes (e.g., 1,000) to establish a stable mean and standard deviation for its score.
• Defining the expert policy score, which could be a scripted controller, human demonstration, or the known maximum achievable reward.
• Reporting confidence intervals alongside the normalized score to account for variance in both the policy under test and the baselines. This rigor supports reproducibility in research publications.

While powerful, normalized scores have limitations and are rarely used in isolation:

• They can mask important details like sample efficiency or catastrophic failure modes if the task is binary (success/failure).
• They rely on the quality and relevance of the chosen baselines. A poor expert baseline inflates scores.
• Therefore, they are typically reported alongside absolute metrics like success rate, Success Weighted by Path Length (SPL) for navigation, or Mean Average Precision (mAP) for vision tasks. Ablation studies also use normalized scores to measure the contribution of individual system components.

Normalized scores are the standard for evaluating policy robustness trained with techniques like domain randomization. The protocol involves:

1. Training a policy in a simulation with randomized parameters (dynamics, visuals).
2. Evaluating it in a held-out set of test environments within simulation, each with different, fixed parameters.
3. Computing a normalized score for each test environment.
4. Reporting the mean and standard deviation of these scores across all test environments. A high mean with low standard deviation indicates a robust, generalizable policy, which is a strong predictor of successful zero-shot transfer to physical hardware.

The most fundamental robotics metric, Success Rate measures the percentage of trials where a policy completes its defined task. It is often the raw performance value that is later normalized. For example, a policy that successfully opens a door in 8 out of 10 trials has an 80% success rate. This binary metric (success/failure) is clear but lacks nuance about the quality of the execution.

In Reinforcement Learning (RL), Cumulative Reward (or return) is the sum of all rewards an agent receives in an episode. It is the primary objective for policy optimization. However, reward functions are arbitrarily scaled, making direct comparison between different tasks impossible. Normalized scores are frequently calculated by scaling the achieved cumulative reward between the performance of a random policy (lower bound) and an expert policy or known maximum (upper bound).

A composite metric for navigation tasks, SPL refines simple success rate by penalizing inefficient paths. It is calculated as:

SPL = (Success) * (Optimal Path Length / Agent's Path Length)

• A successful but meandering agent receives an SPL < 1.0.
• This metric inherently normalizes for task scale (path length) and is a standard in benchmarks like Habitat and AI2-THOR, providing a more nuanced performance measure than success rate alone.

A standardized collection of tasks and environments, a Benchmark Suite (e.g., MetaWorld, RoboSuite, DM Control) provides the common ground for evaluating and comparing algorithms. These suites:

• Define the tasks, initial conditions, and termination criteria.
• Provide the expert and random policy performance baselines required for score normalization.
• Ensure reproducibility by fixing evaluation protocols. Without a benchmark suite, normalized scores lack a consistent frame of reference.

The Evaluation Protocol is the rigorous, predefined procedure for testing a policy. It dictates:

• The number of real-world episodes or simulation rollouts.
• The randomization seeds for environment initialization.
• How sensor noise and domain shifts are introduced during testing.
• The exact formula for calculating the final reported metric (e.g., normalized score). Adherence to a shared protocol is essential for fair comparison and scientific progress in sim-to-real research.

Distribution Shift is the core challenge quantified by sim-to-real benchmarking. It refers to the change in the statistical distribution of inputs (e.g., lighting, textures, friction) between the training simulation and the deployment reality. A normalized score measured on real hardware directly reflects a policy's resilience to this shift. Techniques like Domain Randomization and Domain Adaptation aim to minimize the performance drop caused by distribution shift, which would be reflected in a higher normalized score.