Non-Prehensile Manipulation

The most fundamental non-prehensile technique, where a robot uses its end-effector to apply a force to an object, causing it to slide across a surface. This requires modeling friction cones and predicting the object's motion under quasistatic assumptions. Key challenges include:

• Sticking vs. Sliding: Determining if the applied force overcomes static friction.
• Pivot Points: Objects can rotate about a corner or edge during a push.
• Planning: Algorithms must plan sequences of pushes to achieve a goal pose, often using sampling-based planners in the object's configuration space.

This technique involves using a fixed edge or corner of an object as a fulcrum to rotate it, often to reorient it or move it over small obstacles. It exploits gravity and momentum. Implementation requires:

• Center of Mass Estimation: Critical for predicting if a tumbling motion will succeed or cause the object to fall.
• Impulsive Control: Applying a sharp, controlled force to initiate rotation.
• Dynamic Stability: Ensuring the object comes to rest in a stable configuration after the maneuver. This is common in logistics for moving large boxes.

A dynamic extension of tumbling where an object is intentionally knocked over from one stable pose to another. This is useful for objects with a low static stability margin. The robot must:

• Compute Energy Thresholds: Apply just enough kinetic energy to overcome the potential energy barrier of the current stable pose.
• Predict Final Pose: Model the object's inertia tensor and contact geometry to predict its resting state after toppling.
• Use Environmental Constraints: Walls or other objects can be used to limit and guide the toppling motion, making it more predictable.

Many real-world tasks combine non-prehensile actions with brief grasping phases. For example, a robot might push an object into a position where it can be grasped, or use a finger gaiting motion that involves controlled pushing to adjust an in-hand object's pose. This highlights that non-prehensile and prehensile manipulation exist on a continuum. Effective hybrid strategies require:

• Unified Planning Frameworks: That can sequence contact-rich actions of different types.
• Contact Mode Selection: Deciding when to make or break a secure grasp.

Non-prehensile planning is computationally challenging due to hybrid dynamics (making/breaking contact) and underactuation. Core approaches include:

• Quasistatic Models: Assume motions are slow enough that inertial forces are negligible, simplifying physics to friction-based pushing models.
• Contact-Implicit Trajectory Optimization: Solves for robot motions without pre-specifying when/where contacts occur, allowing the optimizer to discover useful non-prehensile interactions.
• Sampling-Based Planners: Like RRT or PRM, explore the combined configuration space of the robot and the manipulable object, generating paths that may include pushes or pivots.

Due to the complexity of analytical modeling, reinforcement learning (RL) and imitation learning are powerful tools for discovering non-prehensile policies. Key methodologies are:

• Model-Free RL: Agents learn control policies through trial-and-error in simulation, often using domain randomization to bridge the sim-to-real gap. The policy network learns to output pushes or tumbling actions.
• Visual Foresight: Models learn to predict the outcome of actions on pixel space, allowing planning via model predictive control (MPC) in the latent space.
• Demonstration Learning: Human operators provide examples of complex non-prehensile maneuvers, which are then encoded into dynamic movement primitives (DMPs) or used for behavior cloning.

The most common form of non-prehensile manipulation, where an object is translated across a surface using a controlled pushing contact. This is essential for tasks like:

• Table clearing: Sweeping items into a collection bin.
• Object reorientation: Adjusting an object's pose for a subsequent grasp.
• Singulation: Separating one item from a clustered group.

Key challenges involve predicting the quasistatic motion of the object, which depends on friction geometry and the center of friction.

Involves using a sequence of pushes or pivots to rotate an object about an edge or corner. This is critical for manipulating objects too large, heavy, or awkward for the gripper to lift fully.

Applications include:

• Repositioning large boxes in warehouses.
• In-grasp manipulation where an object is partially held and tumbled within the hand to achieve a desired orientation.
• Manipulating objects on constrained surfaces where sliding is not possible.

A hybrid technique where an object is lifted slightly and rotated about a contact point with the environment (like a table edge). This allows for precise reorientation with minimal lifting force.

Mechanics: The robot controls the tip-over point by managing the line of action of the applied force relative to the object's center of mass. This is more energy-efficient than full regrasping and is often used as a pre-grasp manipulation step.

Extends beyond quasistatic pushing into the dynamic regime, where inertial forces dominate. The robot imparts momentum to cause an object to fall or roll into a desired configuration.

Examples:

• Knocking over a bottle into a hand.
• Flicking a switch or pressing a button.
• Dynamic rolling of cylinders or spheres.

This requires sophisticated contact mechanics models and is often addressed with reinforcement learning in simulation.

Enabling robots to operate in human environments, which are designed for non-prehensile actions by people.

Key tasks include:

• Surface cleaning: Wiping, sweeping, and mopping.
• Rearranging items on cluttered countertops and tables.
• Opening doors and drawers by pushing or pulling handles.
• Bed-making by smoothing sheets.

These tasks require compliant control (like impedance control) to manage uncertain contact forces and prevent damage to the environment.

In-hand manipulation is the fine-grained, contact-rich control of an object within a robotic hand's grasp. Unlike non-prehensile methods, it maintains a firm grip while using finger gaits, rolling, or sliding to reposition or reorient the object. Key techniques include:

• Finger gaiting: Sequentially breaking and re-establishing contacts to "walk" the object.
• Pivoting: Rotating an object about a contact point.
• Sliding: Translating the object against the fingers' surfaces. This is essential for tasks like tool reorientation or assembling components without putting the object down.

Tactile servoing is a closed-loop control method that uses real-time tactile sensor feedback—not vision—to guide manipulation. It is critical for non-prehensile tasks where contact state is paramount. The controller adjusts the robot's motion to achieve a desired tactile pattern, such as:

• Maintaining a specific contact force while pushing.
• Following the contour of an object via shear sensor readings.
• Achieving uniform pressure distribution. This enables robust execution in the presence of visual occlusion and object compliance, directly complementing vision-based non-prehensile planning.

These are foundational force-motion interaction control paradigms essential for safe and effective non-prehensile manipulation.

Impedance Control regulates the dynamic relationship between force and motion, making the robot's end-effector behave like a programmable mass-spring-damper system. It is ideal for maintaining stable contact during pushing or pivoting.

Admittance Control inverts this relationship: measured forces are used to compute a desired motion. This makes the robot compliant, allowing it to yield to contact forces, which is useful for tasks like aligning an object against a surface by feel.

Contact-implicit trajectory optimization is a planning method that discovers when and where contacts should occur without a human pre-specifying the contact sequence. For non-prehensile tasks like tumbling or batting an object, the solver simultaneously optimizes the robot's state trajectory, control inputs, and contact forces. This is formulated as a mathematical program with complementarity constraints (MPCC). It allows the robot to autonomously synthesize complex behaviors—such as using a series of glancing impacts to reposition an object—that would be difficult to script manually.

Visual servoing is a robot control technique that uses direct visual feedback from a camera to drive the end-effector's motion. It is highly relevant for dynamic non-prehensile tasks. There are two primary types:

• Position-Based Visual Servoing (PBVS): A 3D pose of the target object is estimated, and the error is computed in Cartesian space.
• Image-Based Visual Servoing (IBVS): The error is computed directly in the 2D image plane (e.g., reducing the pixel distance to target features). IBVS is often more robust for non-prehensile tasks like batting a ball or tracking a sliding object, as it avoids errors from inaccurate 3D reconstruction.

Model Predictive Control is an advanced, receding-horizon optimal control method critical for executing dynamic non-prehensile maneuvers. At each control cycle, MPC:

1. Uses a dynamic model (e.g., of the robot, object, and their interaction) to predict future states over a short time horizon.
2. Solves an optimization problem to find the best sequence of control inputs that minimizes a cost (e.g., error in object position) and satisfies constraints (e.g., joint limits, friction cones).
3. Executes only the first control input before re-planning. This allows real-time adjustment to disturbances, making it ideal for tasks like keeping a ball in the air or pushing an object on a slippery surface.