Interpretable Robot Failure Attribution for Remote Robot Supervisors

Abstract

Robot failures are inevitable and often require human review so that underlying issues can be resolved. This is a challenging task for human supervisors, who must interpret data split across multiple modalities, with limited context for the situation at hand. Performing accurate and efficient failure attribution typically requires purpose-designed interfaces and procedures. We propose an approach that integrates models learned from limited data with feature-level interpretability tools to accelerate the review of multimodal data, highlighting relevant times and modalities in a review interface. We implement this method in a warehouse shelf-picking workcell setting using community-standard visualization tools. In user study with twelve participants, we observe that the assistance improves overall labeling recall and shows a trend toward reducing cognitive workload. We discuss participants’ strategies for using the assistance, and suggest future directions for developing context-aware robotics visualization tools.

BibTeX Entry

@inproceedings{walker2026interpretable,
  author = {Walker, Nick and Wang, Zihan and Grotz, Markus and Cakmak, Maya},
  title = {Interpretable Robot Failure Attribution for Remote Robot Supervisors},
  year = {2026},
  month = aug,
  location = {Kitakyushu, Japan},
  booktitle = {IEEE International Conference on Robot and Human Interactive Communication},
}