In-Hand Manipulation

Finger gaits are sequences of finger repositioning steps, analogous to a human walking their fingers around an object. This technique allows for large-scale reorientation by breaking and re-establishing contacts in a controlled manner.

• Purpose: To overcome the limited workspace of individual fingers.
• Mechanism: Involves planning stable intermediate grasps during the gait cycle to prevent dropping.
• Challenge: Requires solving complex contact sequencing and force closure problems in real-time.

Rolling manipulation exploits the non-holonomic constraints of rolling contact between curved finger surfaces and an object. By controlling finger motions, the object can be rotated as if on bearings.

• Principle: Based on differential geometry and the kinematics of rolling without slipping.
• Application: Ideal for fine, continuous reorientation of spherical or cylindrical objects.
• Implementation: Often uses tactile servoing to maintain desired contact conditions and prevent slip.

Sliding manipulation involves controlled frictional slipping between the fingertips and the object. By modulating grip forces and finger velocities, the object can be translated or rotated within the palm.

• Control Paradigm: Typically implemented using hybrid force-velocity control.
• Key Requirement: Accurate slip detection and prediction using tactile sensors to avoid uncontrolled drops.
• Use Case: Useful for small adjustments where breaking contact is inefficient.

Specialized hardware is foundational for in-hand manipulation. Key designs include:

• Underactuated Hands: Use fewer motors than degrees of freedom (e.g., via tendons), enabling adaptive grasping that conforms to object shape.
• High-DOF Hands: Like the Shadow Hand or Allegro Hand, which offer independent finger control for precise motions.
• Tactile Sensing: Sensors like GelSight or BioTac provide high-resolution contact geometry, pressure, and slip data essential for closed-loop control.

This approach uses physics simulators and known object properties to plan manipulation sequences offline or online.

• Process: The planner simulates candidate finger motions, evaluating outcomes based on quasi-static or dynamic models.
• Methods: Includes trajectory optimization and contact-implicit trajectory optimization to discover when and where contacts should occur.
• Limitation: Performance degrades with model inaccuracy, leading to the sim-to-real gap.

Deep reinforcement learning (RL) and imitation learning train neural network policies that map sensory input (vision, touch) directly to motor commands.

• Advantage: Can discover robust strategies that compensate for complex dynamics and uncertainty.
• Training Paradigms: Reinforcement Learning for Control in simulation, often with domain randomization for robustness.
• Architecture: Policies are often visuomotor control policies or use recurrent networks to maintain state over the manipulation sequence.

In-hand manipulation is critical for small-parts assembly where components must be precisely aligned before insertion. This includes:

• Inserting a USB connector into a port by rotating it to the correct orientation.
• Threading a nut onto a bolt by using finger motions to spin it.
• Kitting operations where a robot picks multiple different parts and reorients them for presentation to a human worker or another machine. This capability reduces the need for complex, custom-designed fixtures and feeders, increasing production line flexibility.

In warehouses, robots must handle a vast array of package shapes and sizes. In-hand manipulation allows a single gripper to:

• Reorient parcels for optimal placement on a conveyor belt or into a shipping container.
• Perform singulation, separating one item from a bin of jumbled objects by manipulating it into an isolated grasp.
• Adjust grip on deformable objects like polybags without dropping them. This reduces damage rates and enables mixed-SKU palletizing and depalletizing with greater autonomy.

In medical settings, fine dexterity is paramount. Robotic systems employing in-hand manipulation can:

• Pass surgical instruments to a surgeon, presenting the handle in the correct orientation.
• Manipulate soft tissues or sutures during minimally invasive procedures.
• Handle delicate lab equipment like slides and vials for automated testing. These applications demand sub-millimeter precision and the ability to react to compliant, non-rigid objects, pushing the boundaries of force-sensitive control.

For robots to operate in human environments, they must interact with everyday objects designed for human hands. Key use cases include:

• Opening a jar by unscrewing the lid while holding the base.
• Using kitchen tools like a spatula to flip a pancake.
• Reorienting a smartphone to plug in a charger.
• Turning a doorknob or key. These tasks require combining dexterous manipulation with robust perception to handle objects with varying mass, friction, and compliance.

Research labs require repetitive, precise handling of sensitive materials. In-hand manipulation enables:

• Pipetting and liquid handling by fine-tuning the angle and position of vials.
• Manipulating petri dishes to access specific sectors under a microscope.
• Handling microplates and slides without contamination.
• Cap opening and closing for sample tubes. Automation here increases throughput and reproducibility while minimizing human exposure to hazardous substances.

In unstructured, hazardous environments like disaster zones or Explosive Ordnance Disposal (EOD), robots must manipulate unknown objects with extreme care. Applications include:

• Turning a valve to shut off a gas line.
• Carefully rotating an unstable object to inspect it from all sides.
• Disarming procedures that require unscrewing casings or manipulating wires. These scenarios highlight the need for robust, contact-rich manipulation under high uncertainty and the critical importance of tactile feedback and force control to prevent catastrophic failures.

A closed-loop control method that uses real-time feedback from tactile sensors to guide manipulation. Unlike vision-based servoing, it relies on direct contact measurements.

• Key Mechanism: Continuously adjusts motor commands to minimize the error between a desired and actual tactile signal (e.g., contact force distribution, gel deformation).
• Primary Use: Maintaining stable contact, following contours, or performing delicate insertion tasks where visual occlusion is common.
• Sensor Types: Often employs high-resolution sensors like GelSight or BioTac arrays.

A geometric and force condition determining if a robotic grasp can resist arbitrary external wrenches (combined forces and torques). It is a foundational concept for secure in-hand manipulation.

• Formal Definition: A grasp achieves force closure if the set of possible contact forces can generate any resultant wrench on the object.
• Implication for In-Hand Manipulation: A grasp must maintain force closure throughout the manipulation trajectory to prevent accidental drops, even as finger positions change.
• Analysis Tool: Often evaluated using the Grasp Wrench Space, which visualizes all achievable wrenches.

Manipulation without enclosing or firmly grasping an object. It is a complementary strategy to in-hand (prehensile) manipulation.

• Common Techniques: Pushing, pivoting, tumbling, or sliding objects using surfaces, gravity, or dynamic motions.
• Strategic Advantage: Can reorient or reposition objects that are too large, heavy, or fragile for a single hand's grasp.
• Integration with In-Hand Manipulation: Advanced systems may sequence non-prehensile pushes with prehensile regrasps to achieve complex object reorientation.

A mechanical design principle where a robotic hand has fewer actuators than degrees of freedom. This is common in anthropomorphic hands to reduce complexity, weight, and cost.

• How It Works: Uses tendons, linkages, or compliant mechanisms to couple the motion of multiple joints to a single motor.
• Impact on In-Hand Manipulation: Enables adaptive grasping, where the hand conforms to object shape passively. However, it complicates precise, independent finger control required for sophisticated in-hand maneuvers.
• Example: The Shadow Dexterous Hand uses tendon-driven underactuation for 24 degrees of freedom controlled by 20 actuators.

The process of releasing and reacquiring an object within a hand or between hands to achieve a new, more functional grasp configuration. It is often a necessary component of long-horizon in-hand manipulation.

• Trigger Conditions: Required when the current grasp lacks the dexterity or force closure needed for the next manipulation step (e.g., switching from a power grasp to a precision grasp).
• Planning Challenge: Involves finding intermediate stable placements (possibly using a supporting surface) and collision-free paths for the fingers.
• Example: Picking up a pen with a fist, then adjusting fingers to a writing grip involves a regrasp sequence.

A robot control strategy that regulates the dynamic relationship between force and motion at the end-effector or finger, making the robot behave as a programmable mass-spring-damper system.

• Core Equation: (\tau = M(\ddot{q}) + B(\dot{q}) + K(q)), where (M), (B), and (K) are inertia, damping, and stiffness matrices.
• Critical for In-Hand Manipulation: Allows fingers to exhibit compliant contact, essential for maintaining stable grip on uncertain objects and preventing damage. It enables the hand to "yield" appropriately during finger gaiting or rolling manipulations.
• Counterpart: Admittance Control, which computes motion from measured force.