A User Interface for Sense-making of the Reasoning Process while Interacting with Robots
Authors:
- Chao WangCWChao Wang
- Joerg DeigmoellerJDJoerg Deigmoeller
Preprint
Abstract
As knowledge graph has the potential to bridge the gap between commonsense knowledge and reasoning over actionable capabilities of mobile robotic platforms, incorporating knowledge graph into robotic system attracted increasing attention in recent years. Previously, graph visualization has been used wildly by developers to make sense of knowledge representations. However, due to lacking the link between abstract knowledge of the real-world environment and the robot's actions, transitional visualization tools are incompatible for expert-user to understand, test, supervise and modify the graph-based reasoning system with the embodiment of the robots. Therefore, we developed an interface which enables robotic experts to send commands to the robot in natural language, then interface visualizes the procedures of the robot mapping the command to the functions for querying in the commonsense knowledge database, links the result to the real world instances in a 3D map and demonstrate the execution of the robot from the first-person perspective of the robot. After 3 weeks of usage of the system by robotic experts in their daily development, some feedback was collected, which provides insight for designing such systems.Keywords
All Related Versions
- Version 1, 2022-10-15, ArXiv
- Version 2, 2023-01-27, ArXiv
- Published version: , 1.
This publication has 12 references indexed in Scilit:
- Knowledge graphs as tools for explainable machine learning: A surveyArtificial Intelligence, 2022
- Connecting Human-Robot Interaction and Data Visualization2021
- Designing for Mobile and Immersive Visual Analytics in the FieldIEEE Transactions on Visualization and Computer Graphics, 2019
- Reinforcement Knowledge Graph Reasoning for Explainable Recommendation2019
- Jointly Learning Explainable Rules for Recommendation with Knowledge Graph2019
- Seq2seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence ModelsIEEE Transactions on Visualization and Computer Graphics, 2018
- Force‐Directed Edge Bundling for Graph VisualizationComputer Graphics Forum, 2009
- Evolving interface design for robot search tasksJournal of Field Robotics, 2007
- Comparing the usefulness of video and map information in navigation tasks2006
- Arc diagrams: visualizing structure in strings2003