GelSight

The sensor's core is a soft, transparent elastomer (often silicone) with a reflective coating. This compliant skin conforms to object surfaces, capturing microscopic geometry. Key properties include:

• High Elastic Modulus: Ensures rapid, accurate return to its original shape after deformation.
• Optical Transparency: Allows internal camera to see the deformation pattern.
• Durability: Engineered to withstand thousands of contact cycles without permanent deformation.

An array of colored LEDs (typically red, green, and blue) is positioned around the camera inside the sensor. When the gel deforms, this structured light reflects at different angles, creating a color-coded height map.

• The chromatic gradient directly encodes surface slope and depth.
• This system enables sub-millimeter spatial resolution and micron-level depth sensitivity, far exceeding binary contact sensors.

A miniature, high-framerate camera captures the deformation of the reflective gel surface. The resulting image is a detailed tactile image or tactile heightmap.

• This provides a dense 3D force field measurement, not just a single force vector.
• Standard resolutions exceed 30x30 pixels per square centimeter, allowing detection of features like fingerprints, textures, and part edges.

Using photometric stereo algorithms, the system converts the colored deformation image into a precise 3D reconstruction of the contact surface.

• Outputs include a 3D mesh and a surface normal map.
• This allows for direct measurement of object geometry, including curvature, edges, and defects, enabling tasks like blind grasping and object identification.

By analyzing the lateral displacement of tracked gel surface features over consecutive frames, GelSight can detect incipient slip and measure shear forces.

• This is critical for grasp stability and dexterous manipulation.
• It provides a direct measurement of the friction cone at each contact point, informing control systems if a grip is about to fail.

GelSight's rich data stream enables advanced robotic capabilities:

• In-Hand Manipulation: Precise feedback for rolling, sliding, or reorienting objects within a grasp.
• Material Identification: Classifying objects based on surface texture and compliance.
• Tactile Servoing: Closed-loop control using real-time contact geometry to guide tasks like insertion or contour following.
• Sim-to-Real Transfer: Providing realistic, high-fidelity tactile feedback in simulation for training manipulation policies.

A closed-loop control method that uses real-time tactile sensor feedback to guide robotic manipulation. Unlike position-based control, it directly uses contact information (e.g., pressure distribution, shear forces) to achieve tasks like maintaining stable contact, following a contour, or inserting a peg.

• Key Mechanism: Continuously compares current tactile readings to a desired tactile state, generating corrective motor commands.
• Role of GelSight: Provides the high-resolution, geometric and textural feedback required for precise servoing on complex surfaces.

The fine-grained control of an object within a robotic hand's grasp, using finger motions to reposition or reorient it without releasing it. This includes rolling, sliding, pivoting, and finger gaiting.

• Core Challenge: Requires precise force modulation and contact state estimation to prevent slip or excessive grip force.
• Role of GelSight: Enables observation of subtle object motions and micro-slip within the grasp, providing the state feedback necessary for dexterous in-hand reorientation policies.

The process of identifying incipient slip—when an object begins to move relative to the gripper—using sensor data. Early detection allows a controller to increase grip force or adjust finger posture to regain stability.

• Traditional Methods: Use binary force/torque thresholds or vibration (accelerometer) signals.
• GelSight's Advantage: Detects micro-slip through high-frame-rate observation of shear deformation patterns on the gel surface before full macro-slip occurs, enabling preemptive correction.

A geometric and force condition in robotic grasping where the set of contact forces can generate any resultant wrench (combined force and torque) on an object. This ensures the object can be held securely against arbitrary external disturbances.

• Mathematical Basis: Analyzed via the Grasp Wrench Space (GWS). Force closure is achieved if the origin of the wrench space lies within the convex hull of the possible contact wrenches.
• Role of Tactile Sensing: GelSight helps verify contact locations and local surface normals, which are critical inputs for computing and maintaining a force-closure grasp in real-time on unknown objects.

A robot control strategy that regulates the dynamic relationship between force and motion at the end-effector. Instead of directly controlling position, it makes the robot behave as a programmable mass-spring-damper system.

• Key Parameters: Stiffness (K), damping (B), and inertia (M) matrices define the "mechanical impedance."
• Integration with Tactile Sensing: GelSight provides the contact geometry and interaction forces needed to tune impedance parameters online. For example, lowering stiffness upon initial contact to prevent bouncing, or increasing damping when sliding along a surface.

The computer vision task of determining an object's three-dimensional position and three-dimensional orientation (rotation) relative to a sensor frame. The "6D" refers to 3 translational and 3 rotational degrees of freedom.

• Visual vs. Tactile: Typically performed with external cameras (exteroceptive). Tactile pose estimation uses contact information to refine or infer pose when the object is occluded in the hand.
• GelSight's Role: The high-resolution deformation image can be matched against an object model to estimate local contact patch pose, which can be integrated over multiple contacts to estimate the global object pose in-hand.