YCB Object Set

The set is designed to cover a wide spectrum of object affordances and physical properties critical for manipulation research. Categories include:

• Food items (e.g., mustard bottle, gelatin box, banana) for deformable and articulated object handling.
• Tools (e.g., hammer, screwdriver, scissors) for testing functional grasps and tool use.
• Containers (e.g., bowl, mug, pitcher) for testing prehensile and non-prehensile (pouring) actions.
• Geometric shapes (e.g., wood block, large clamp) for fundamental grasp stability and pose estimation tests. This diversity ensures algorithms are evaluated on a broad range of grasp challenges, from simple power grasps to complex in-hand manipulation.

Every object in the set is accompanied by a precise, textured 3D mesh model and detailed metadata, including:

• Mass and inertia properties for accurate physics simulation.
• Dimensions and bounding boxes for planning and perception.
• Suggested grasp poses derived from analytical models.
• RGB-D scan data of the physical objects for perception algorithm training. This rich dataset allows for highly realistic simulation-based training (e.g., in PyBullet or MuJoCo) and provides ground truth for evaluating 6D pose estimation and grasp detection algorithms.

The YCB Set established common performance metrics to enable direct comparison between different robotic systems and algorithms. Key metrics include:

• Grasping success rate: The percentage of successful lifts and holds from a predefined set of initial poses.
• Pose estimation accuracy: Measured using the Average Distance (ADD) metric, which calculates the mean distance between model points transformed by the estimated pose versus the ground truth pose.
• Task completion metrics: For structured tasks like stacking or placing objects in a box. These standardized metrics move the field beyond anecdotal demonstrations to quantitative, reproducible benchmarking.

A core purpose of the YCB Set is to facilitate sim-to-real transfer. Researchers can:

1. Train manipulation policy networks or grasp planners like DexNet in simulation using the accurate 3D models.
2. Apply techniques like domain randomization (varying textures, lighting, physics parameters) to the YCB objects to improve robustness.
3. Evaluate the trained policies on the identical physical objects, directly measuring the sim-to-real gap. This creates a controlled pipeline for developing algorithms that work reliably on physical hardware.

The YCB Object Set is the de facto standard for major robotics challenges, most notably the Amazon Robotics Challenge (later the Amazon Picking Challenge). It has been used in thousands of research papers to benchmark:

• Visual servoing and 6D pose estimation pipelines.
• Grasp wrench space analysis and force closure calculations.
• Task and motion planning for complex, multi-step chores.
• Imitation learning and reinforcement learning for control. Its widespread adoption creates a common language and baseline, accelerating progress across the robotics community.

The YCB Set interacts with several advanced concepts in dexterous manipulation:

• In-hand manipulation: Objects like the screwdriver or marker are used to test finger gaiting and re-orientation.
• Tactile servoing: The set's objects provide varied surfaces for testing GelSight or other tactile sensors.
• Non-prehensile manipulation: Items like the bowl and plate are used for pushing and sliding tasks.
• Contact-implicit trajectory optimization: Planners can use the YCB models to discover complex contact sequences for tasks like opening the lid of the mustard bottle. Extensions like the YCB-M set add articulated objects (e.g., laptop, drawer) to increase complexity.

6D Pose Estimation is the computer vision task of determining an object's full three-dimensional position (X, Y, Z) and three-dimensional orientation (roll, pitch, yaw) relative to a camera. It is critical for precise robotic manipulation, such as inserting a peg into a hole or re-grasping an object.

• Inputs: Typically uses RGB-D (color + depth) data or just RGB images.
• Methods: Includes direct regression with CNNs, keypoint detection, and template matching using point pair features.
• Challenge: Must be robust to occlusion, clutter, and varying lighting—common in real-world scenes like those modeled by the YCB Object Set.

Force Closure is a fundamental condition in robotic grasping where the set of contact forces applied by the gripper can generate any resultant wrench (combined force and torque) on the object. This ensures the object can be held securely in equilibrium against arbitrary external disturbances.

• Mathematical Basis: Analyzed using friction cones and linear algebra to determine if contact forces can balance any external wrench.
• Grasp Planning: A primary quality metric for evaluating candidate grasps in both analytic methods (like those in DexNet) and real-world execution.
• Relation to YCB: Objects in the YCB set are often evaluated for the ease with which force-closure grasps can be achieved.

The Sim-to-Real Gap refers to the performance discrepancy between a robot policy trained in a physics simulation and its performance when deployed on physical hardware. This gap arises due to imperfect modeling of dynamics, sensor noise, and actuator delays in simulation.

• Mitigation Strategies: Key techniques include Domain Randomization (varying simulation parameters like friction and lighting) and System Identification (calibrating the simulator to real-world data).
• Benchmark Role: The YCB Object Set is frequently used to evaluate sim-to-real transfer, as policies trained to manipulate YCB models in simulation must work on the real-world counterparts.

Tactile Servoing is a closed-loop control method that uses real-time feedback from tactile sensors (e.g., GelSight) to guide robotic manipulation. Instead of relying solely on vision, it corrects motions based on direct contact sensations, such as maintaining a desired contact force or following an object's contour.

• Applications: Essential for delicate tasks like in-hand manipulation, inserting connectors, or handling fragile YCB objects like the plastic banana.
• Feedback Loop: The controller adjusts the robot's pose to minimize the error between the current tactile reading (e.g., pressure distribution) and a desired tactile signature.

Model Predictive Control (MPC) is an advanced, online optimization-based control method. At each time step, it uses a dynamic model of the robot and its environment to predict future states over a horizon, solves for an optimal sequence of control inputs, and executes only the first step before re-planning.

• Use in Manipulation: Ideal for contact-rich tasks with dynamics, like pushing or non-prehensile manipulation of YCB objects, as it can explicitly handle constraints (e.g., joint limits, force limits).
• Connection to Planning: Contact-Implicit Trajectory Optimization is often formulated as an MPC problem, allowing the solver to discover optimal contact sequences.