Human-Robot Interaction (HRI)

The robot's ability to perceive and interpret the human and the environment. This multimodal input layer is the foundation for all interaction.

• Computer Vision: Cameras and depth sensors (e.g., RGB-D, LiDAR) for tasks like human pose estimation, gesture recognition, and gaze estimation.
• Audio Processing: Microphone arrays for speech recognition, emotion recognition from vocal tone, and sound localization.
• Force/Torque Sensing: Sensors in joints or end-effectors to measure contact forces, critical for physical human-robot interaction (pHRI) and Power and Force Limiting (PFL) safety.
• Proximity & Tactile Sensors: To detect nearby humans or direct touch, enabling reactive safety behaviors.

The software layer that processes sensory data to understand context, infer intent, and plan actions. This is where AI models and control logic reside.

• Intent Recognition: Inferring a human's goals from behavior and context.
• Action Anticipation & Human Motion Forecasting: Predicting future human actions for proactive robot behavior.
• Task and Motion Planning (TAMP): Decomposing high-level instructions into feasible movement sequences.
• Theory of Mind (ToM) Modeling: Advanced systems that attribute beliefs and knowledge to the human to predict behavior.
• World Models: Maintaining an internal representation of the environment's state for planning and prediction.

The bidirectional channels through which information and commands are exchanged between the human and robot.

• Natural Language Processing (NLP): For understanding spoken or typed commands and generating verbal responses.
• Social Cues: Robot expression via lights, sounds, screen-based faces, or simple body movements (e.g., nodding).
• Explainable AI (XAI) Interfaces: Visualizations or natural language explanations of the robot's decisions to build trust.
• Tangible Interfaces: Kinesthetic teaching, where a human physically guides the robot, or control panels.
• Brain-Computer Interfaces (BCI): Direct neural control pathways for specialized applications.

The physical execution layer where planned actions are translated into safe, fluent movement. This defines the robot's physical interactivity.

• Collaborative Robot (Cobot) Actuators: Feature force-limited joints and back-drivable motors for safe physical human-robot interaction (pHRI).
• Control Paradigms:
  • Shared Autonomy: Blends human input with autonomous control.
  • Impedance/Admittance Control: Allows compliant, spring-like response to external forces.
  • Visuomotor Policies: Neural networks that map visual input directly to motor commands.
• Safety-Rated Control Systems: Hardware and software that enforce standards like ISO/TS 15066, implementing Speed and Separation Monitoring (SSM) and Power and Force Limiting (PFL).

The integrated hardware and software features that ensure human physical safety, a non-negotiable requirement for co-located work.

• Inherent Design: Rounded edges, smooth surfaces, and minimized pinch points.
• Functional Safety Systems: Safety-rated monitored stop, hand-guiding, and speed and separation monitoring (SSM).
• Standards Compliance: Adherence to ISO 10218 (robots) and ISO/TS 15066 (collaborative applications), which define test methods for pain thresholds.
• Real-Time Monitoring: Light curtains, safety scanners, and onboard sensors that trigger protective stops if a human enters a hazardous zone or contact limits are exceeded.

The frameworks and quantitative measures used to assess the performance, usability, and social acceptability of an HRI system.

• Task Performance Metrics: Success rate, time to completion, and number of human interventions.
• Human Factors Metrics: NASA-TLX for workload, trust scales, and usability questionnaires (e.g., SUS).
• Fluency Metrics: Measures of team coordination like idle time, concurrent activity, and smooth turn-taking.
• Safety Metrics: Recorded force during incidental contact, frequency of safety stops, and separation distance violations.
• Social Metrics: Adherence to proxemics norms and user perceptions of robot personality or likeability.

This application focuses on robots working alongside humans in warehouses, factories, and on assembly lines. Collaborative robots (cobots) are central, designed for power and force limiting (PFL) and speed and separation monitoring (SSM) to ensure safety. Key tasks include:

• Parts presentation and kitting: A cobot holds a part in an ergonomic position for a human to assemble.
• Machine tending: The robot loads/unloads a CNC machine while the human performs quality inspection.
• Co-packaging: Human and robot work on the same packaging line, with the robot handling heavy or repetitive items. Safety standards like ISO/TS 15066 govern these interactions, ensuring force and speed limits prevent injury during incidental contact.

HRI in healthcare spans physical assistance, surgery, and cognitive therapy. Robots in this domain require high reliability, precision, and often empathetic interaction.

• Surgical assistance: Systems like the da Vinci provide surgeons with enhanced dexterity and 3D visualization via teleoperation.
• Physical rehabilitation: Robotic exoskeletons and end-effector devices use impedance control to provide adaptive support for gait or arm movement therapy.
• Socially Assistive Robotics (SAR): Robots like PARO (a therapeutic seal) or more advanced humanoids provide cognitive stimulation, companionship, and adherence prompts for patients with dementia or autism, leveraging principles from affective computing and emotion recognition.

This broad category involves robots interacting with the public in unstructured environments, requiring robust social navigation, proxemics, and natural communication.

• Front-of-house services: Robots in hotels deliver items, in airports provide directions, and in restaurants may seat guests. They must navigate dynamically around people using human-aware motion planning.
• Education and tutoring: Robots act as engaging peers or tutors for children, using gesture recognition and turn-taking to maintain engagement and personalize lessons.
• Public safety and inspection: Robots equipped with sensors perform routine inspections in hazardous areas (e.g., nuclear facilities, construction sites) or assist first responders, often operated via shared autonomy where the robot handles low-level navigation while the human focuses on high-level task assessment.

Applications here bring robots into the home to aid with daily tasks, demanding a high degree of autonomy, safety, and intuitive intent recognition.

• Mobility and fetch-and-carry: Robots assist individuals with motor impairments by retrieving objects, opening doors, or providing stability as a mobile walker.
• Home maintenance and cleaning: Autonomous vacuum cleaners and lawnmowers are early examples, with newer systems aiming for more complex tasks like loading dishwashers or sorting laundry, which require advanced 3D scene understanding and dexterous manipulation.
• Companionship and wellness monitoring: For elderly individuals living alone, robots can provide social interaction, medication reminders, and detect falls or changes in routine, requiring sensitive human pose estimation and action anticipation to be effective.

HRI is critical in extreme environments where direct human presence is dangerous or impossible. Here, teleoperation and human-in-the-loop (HITL) control are paramount.

• Space robotics: Robotic arms on the International Space Station (like Canadarm2) are teleoperated by astronauts for maintenance, or from Earth for unmanned missions. Shared autonomy allows the robot to execute pre-planned trajectories while the human supervises.
• Deep-sea and disaster response: Remotely Operated Vehicles (ROVs) explore ocean depths or search rubble after disasters. Operators rely on sensor feeds and often use kinesthetic teaching or learning from observation to train the robot for specific manipulation tasks in these novel environments.
• Field science: Robots accompany researchers in archaeology or geology, carrying equipment, taking samples, or creating 3D maps, acting as a semi-autonomous field assistant.

HRI principles are used to create realistic human-robot interaction scenarios for training both robots and humans in virtual environments.

• Sim-to-real transfer learning: Robots are trained in high-fidelity physics simulators where they interact with simulated humans, learning safe and effective policies before physical deployment. This is essential for complex physical human-robot interaction (pHRI) tasks.
• Human operator training: Pilots, surgeons, or technicians train using robotic simulators that provide haptic feedback and realistic dynamics, improving their teleoperation skills in a risk-free setting.
• Protocol development and testing: New collaborative workflows, like human-robot teaming on an assembly line, are designed and stress-tested in simulation to evaluate fluency, safety, and efficiency before real-world implementation.

Shared Autonomy is a control paradigm where task execution is dynamically shared between a human operator and the robot's autonomous controller. The system blends human high-level intent and judgment with machine precision and low-level control.

• Dynamic Allocation: Control authority can shift based on context, user input, or task complexity.
• Implementation: Often uses Human-in-the-Loop (HITL) architectures for supervision. Techniques include mixing autonomous and teleoperated commands or using the human to resolve ambiguities the robot cannot.
• Example: A surgical robot where the surgeon defines the cutting path, and the robot autonomously stabilizes the tool and filters out hand tremor.

Intent Recognition is the computational process by which a robot infers a human's immediate goals or planned actions. It is a cornerstone for proactive and fluid HRI, moving beyond reactive command-following.

• Input Modalities: Uses Human Pose Estimation, Gaze Estimation, Gesture Recognition, contextual scene understanding, and interaction history.
• Methods: Ranges from rule-based systems to deep learning models like transformers that sequence observed actions.
• Application: In a warehouse, a robot observes a worker reaching for a heavy box and infers the intent to lift, moving closer to provide support before an explicit command is given.

Social Navigation is the planning and execution of robot movement paths that adhere to human social norms and proxemics. The goal is for the robot to be predictable, courteous, and non-disruptive in shared spaces.

• Core Principles: Maintaining comfortable personal space, passing on the culturally appropriate side, giving right-of-way, and avoiding cutting between interacting people.
• Techniques: Algorithms often integrate social force models or learn cost maps from human trajectory data.
• Challenge: Norms vary by culture and context, requiring adaptive models. A robot in a busy hospital corridor must navigate differently than one in a quiet office.

Explainable AI (XAI) for Robotics encompasses techniques and interfaces that make a robot's decisions, plans, and failures understandable to human users. This transparency is critical for trust calibration, debugging, and safe collaboration.

• Methods: Visualizing planned paths, highlighting sensor data that triggered a decision (e.g., "I stopped because I detected motion in my blind spot"), using natural language to state goals and constraints.
• Importance: In high-stakes settings like manufacturing or healthcare, an unexplained robot failure or sudden movement can break trust and halt collaboration.
• Outputs: Can be visual overlays, auditory explanations, or textual logs accessible via a maintenance interface.

Learning from Observation (LfO) is a machine learning paradigm where a robot acquires a skill by watching a human (or another agent) perform a task, without receiving explicit action labels or reward signals. It is a key method for intuitive robot programming.

• Contrast with Imitation Learning: LfO is more challenging as the learner must infer the actions taken from state transitions alone (the correspondence problem).
• Related to: Kinesthetic Teaching, where the robot is physically guided. LfO is observation-only.
• Application: A domestic robot watches a person prepare a simple meal and learns the sequence of object interactions (open fridge, grasp carton, pour) to replicate the task.