An Augmented Reality Debugging System for Mobile Robot Software Engineers

Robotics presents a unique set of challenges, which change the way that we must approach the debugging of robotic software. Augmented reality (AR) provides many opportunities for enhancing debugging, allowing the developer to see the real world as the robot does, superimposed in situ on the real world view of the human, intuitively displaying the limitations and discontinuities in the robot's real world view. This paper contributes a systematic analysis of the challenges faced by robotic software engineers, and identifies the recurrent concepts for AR based visualisation of robotic data. This in turns leads to a conceptual design for an AR enhanced intelligent debugging space. Both an open source reference implementation of the conceptual design and an initial evaluation of the implementation's efficacy are described. The AR system provides an opportunity to understand the types of errors that are encountered during debugging. The AR system analysis and design provide a reusable conceptual framework for future designers of robotic debugging systems, and guidelines for designing visualisations. In concert with common, standard robotics interfaces provided by Player/Stage, the AR system design supplies a set of common visualisations, so that many data visualisations can be provided to developers with no additional effort.