Explainable AI (XAI) for HRI

A counterfactual explanation answers the question, "What would need to be different for the robot to have made a different decision?" Instead of describing the model's internal weights, it provides actionable, contrastive scenarios a human can understand.

• Example: A delivery robot chooses a longer hallway route. A counterfactual explanation might be: "I did not take the shorter path through the kitchen because a human was detected there. If the kitchen had been empty, I would have taken that route, saving 45 seconds."
• Key Benefit: Directly links robot decisions to observable world states, making explanations intuitive and tied to the shared environment.

This technique generates heatmaps that highlight which regions of a robot's visual input (e.g., camera image) most influenced its decision. It translates the abstract concept of "feature importance" into a spatially grounded visual explanation.

• Implementation: Common methods include Grad-CAM (Gradient-weighted Class Activation Mapping) for convolutional neural networks used in vision-based navigation or object recognition.
• HRI Application: A robot grasping an object can show which parts (the handle vs. the body) it deemed most critical for a successful grip. This allows a human to verify the robot's perceptual understanding and correct misalignments (e.g., "No, grasp the lid, not the side").

The robot generates a concise, plain-language summary of the reasoning behind its chosen action or plan, often using a dedicated language model conditioned on its internal state.

• Components: This typically involves a two-stage process: 1) Extracting key decision factors from the planner or policy (e.g., goal, perceived obstacles, battery level). 2) Formulating these factors into a coherent sentence using templates or a generative model.
• Example Output: "I am stopping because my path is blocked by an unidentified object. My battery is at 85%, so I can wait for 5 minutes or attempt a detour if you instruct me to."
• Critical for: Trust Calibration and facilitating Verbal Repair when misunderstandings occur.

This method exposes the robot's task decomposition and planning hierarchy. It shows the human the high-level goal, the sub-tasks, their sequence or dependencies, and the current execution state.

• Representation: Often uses node-and-edge graphs, Gantt charts, or hierarchical lists in a user interface.
• HRI Value: Enables Shared Mental Models. A human can see if the robot is stuck on a specific sub-task (e.g., "locate the valve") and provide targeted help. It also allows for Adjustable Autonomy, where a human can approve, modify, or prune parts of the plan.
• Related Concept: Explainable Planning (XAIP), which focuses on making automated planner outputs interpretable.

The robot explicitly communicates its confidence level in its perceptions, predictions, and the likely success of its planned actions. This is a foundational form of transparency.

• Methods: Can be presented as probability distributions, confidence intervals, or simple categorical labels (High/Medium/Low). Techniques include Bayesian neural networks, Monte Carlo dropout, or ensemble methods to estimate predictive uncertainty.
• HRI Impact: Directly informs Trust Calibration. A robot saying, "I am 95% confident this is a door" versus "I am 45% confident this is a door" prompts vastly different human responses. It signals when the robot requires human oversight or verification, enabling smooth Dynamic Role Allocation.

This user-centric approach tailors explanations by answering the specific question a human is likely asking, rather than providing a full model dump. A contrastive explanation addresses "Why action A, and not action B?" A selective explanation provides the minimal, most relevant information based on context.

• Mechanism: The system infers the human's query type from context, dialogue, or user role (e.g., a safety engineer vs. a casual user).
• Example: If a robot moves left, a novice might ask "Why did you go left?" (answered with a simple rationale). An expert might ask "Why did you go left instead of right?" (triggering a contrastive explanation comparing the cost maps for both paths).
• Principle: Follows the cognitive science of explanation, making communication more efficient and reducing human cognitive load.

This XAI method explains a robot's failed action by presenting a minimal, actionable change to the situation that would have led to success. Instead of stating "grasp failed," the system might generate: "The grasp would have succeeded if the object were rotated 30 degrees clockwise.** This is particularly valuable for on-the-job training and collaborative assembly, where a human can quickly rectify the state of the world. The explanation is generated by querying the robot's internal world model or policy to find the nearest successful state in its feature space.

Used primarily in vision-based HRI, this technique produces a heatmap overlay on the robot's camera feed, highlighting the image regions (pixels) that most influenced its decision. For example, a heatmap might show the robot focused on a tool's handle when planning a grasp, or on a traffic light when deciding to stop. This visual explanation helps humans understand what the robot "sees" as relevant, which is crucial for diagnosing perceptual errors, aligning mental models, and ensuring the robot is attending to safety-critical features in a shared workspace.

The robot articulates its reasoning process using generated natural language statements alongside its actions. For instance, a fetch-and-carry robot might say: "I am moving to the kitchen counter because I detected a coffee cup there, and your calendar indicates a meeting in 5 minutes.** This transforms opaque autonomous behavior into a narrative a human can follow and interrogate. It relies on multimodal models that can ground perceptual data and task plans into coherent language, often using templates or large language models fine-tuned on task domains.

This interface exposes the robot's hierarchical task network (HTN) or behavior tree, showing the decomposed plan, current step, future steps, and alternative branches. It makes the high-level intent and contingency planning transparent. In a manufacturing setting, a screen might show a flowchart where the current node is "Insert Component A" with a failed branch to "Recover from misalignment.** This allows human supervisors to understand the robot's overarching strategy, anticipate its next moves, and manually steer it to a different branch if needed, supporting adjustable autonomy.

The robot communicates its internal confidence level in its perception, classification, or planned action. This is often presented as a simple numeric score, progress bar, or color-coded indicator (e.g., green/high, yellow/medium, red/low). For example, a robot handing a surgeon a tool might display "Scalpel ID Confidence: 92%** on a screen. This quantifies uncertainty and signals when the robot is operating on shaky inferences, prompting the human to provide verification or take over. It is a foundational XAI output for trust calibration and safe shared autonomy.

The robot explains why it chose action A over a plausible alternative action B. For example, a mobile robot might explain: "I chose path around the left side of the table instead of the right because my LiDAR detected less clutter variance on the left.** This type of explanation addresses the human's natural "why not?" question. It requires the system to have access to a contrastive reasoning model that can evaluate and justify trade-offs between different options, making the decision-making process more transparent and debatable.

Theory of Mind (ToM) is a robot's computational ability to attribute mental states—such as beliefs, intents, desires, and knowledge—to its human partner. For XAI, this is foundational. A robot with ToM can tailor its explanations by inferring what the human already knows, what they might be confused about, or what their current goal is. This enables context-aware explanations that are neither overly simplistic nor unnecessarily technical.

• Key Mechanism: Models of human belief states are maintained and updated based on interaction history and perceptual cues.
• XAI Application: The robot can answer "Why did you do that?" not just with the factual reason, but with a reason relevant to the human's inferred task understanding.

Intent Recognition is the process by which a robotic system infers a human's goals or planned actions from observed signals (e.g., gaze, gesture, motion, physiological data). In XAI, this relationship is bidirectional. While intent recognition lets the robot anticipate human needs, the robot must also make its own intent recognizable. Explainable planning involves projecting future actions or highlighting immediate goals in a way a human can parse.

• Key Mechanism: Probabilistic models (e.g., Bayesian networks, hidden Markov models) or deep learning sequences map multi-modal observations to a library of possible intents.
• XAI Link: A robot that can recognize human intent can provide proactive explanations ("I'm moving this out of your way because I see you reaching for the tool behind it").

Trust Calibration is the process of aligning a human user's level of trust in a robot's capabilities with the robot's actual performance. Over-trust can lead to safety risks, while under-trust leads to disuse. XAI is the primary tool for achieving this calibration. Explanations directly manipulate the human's mental model of the robot's reliability and decision-making process.

• Key Mechanism: Explanations are tuned based on real-time performance and difficulty metrics. After a failure, a clear explanation of the cause (e.g., sensor occlusion, planning uncertainty) can prevent a catastrophic loss of trust.
• XAI Application: Interfaces might visually represent confidence scores or uncertainty estimates alongside decisions, or verbally explain trade-offs made during a task.

Shared Autonomy dynamically allocates control authority between human and robot. Adjustable Autonomy allows the level of robot self-governance to be modified. In both paradigms, XAI is critical for smooth control hand-offs. The human must understand what the robot is doing, what it plans to do next, and why it may be requesting or ceding control.

• Key Mechanism: Explanation becomes part of the communication protocol for negotiation. For example, a robot might say, "I am taking over the precise alignment because my force sensors detect slippage you may not feel."
• XAI Application: Explanations justify transitions between autonomy modes, making the robot's behavior predictable and its requests for human intervention understandable.

Natural Language Grounding is the process by which a robot maps words and phrases to perceptual entities, spatial relationships, and actions in its physical environment. For XAI, this capability must work in reverse: the robot must unground its internal symbolic plans and perceptual features back into natural language that a human can understand. This is the core technical challenge of generating verbal explanations.

• Key Mechanism: Vision-Language-Action (VLA) models or semantic mapping systems create a shared vocabulary between perception and language.
• XAI Application: Enables a robot to answer "What is that?" or "Why are you going there?" by referring to objects ("the red valve"), locations ("left of the table"), and actions ("turning clockwise") using common terms.

Multimodal Fusion integrates data from multiple sensors and communication channels (speech, gesture, gaze, force) to understand human intent. For XAI, this fusion is also used to generate explanations across multiple output channels. An effective explanation might combine a verbal utterance ("I can't grasp this"), a highlighted point in a camera feed (showing occlusion), and a haptic pulse in a shared controller.

• Key Mechanism: A central blackboard architecture or attention-based neural model aligns and weights inputs from different modalities to form a coherent context.
• XAI Application: Creates redundant and robust explanations. If the human isn't looking at the screen, a verbal explanation is given. If the environment is loud, a visual highlight on an AR headset is used.