We introduce ROS Commander (ROSCo), an open source system
that enables expert users to construct, share, and deploy
robot behaviors for home robots. A user builds a behavior
in the form of a Hierarchical Finite State Machine (HFSM)
out of generic, parameterized building blocks, with a real
robot in the develop and test loop. Once constructed, users
save behaviors in an open format for direct use with
robots, or for use as parts of new behaviors. When the
system is deployed, a user can show the robot where to
apply behaviors relative to fiducial markers (AR Tags),
which allows the robot to quickly become operational in a
new environment. We show evidence that the underlying state
machine representation and current building blocks are
capable of spanning a variety of desirable behaviors for
home robots, such as
opening a refrigerator door with two arms (video),
flipping a light switch (video),
opening a drawer (video),
unlocking a door, and
handing an object to someone (video). Our experiments show that
sensor-driven behaviors constructed with ROSCo can be
executed in realistic home environments with success rates
between 80% and 100%.
ROSCo—as a system for using, deploying, and creating behaviors—has interfaces at three different levels. The first level, shown in the first figure, is intended for expert users that are willing to spend time learning the interface (perhaps through video online tutorials and how-to guides) but are not necessarily expert roboticists. Behaviors, represented as HFSMs, are constructed at this level using parameterized building blocks where there each block is matched an appropriate graphical interface. Once constructed, they can be saved to disk, reused, and shared.
The second tier interface, shown above, is designed for users with an intermediate level of expertise and conceptualized as a process of users giving a tour of their home to the robot, telling it how to operate what and where. This "touring" interface works by asking users to attach an ARToolKit marker, dragging a 3D frame relative to that marker to a behavior specific spot (e.g. the middle of a drawer handle), then selecting the desired behavior.
Designed for users that just wants to use the robot with its potentially numerous capabilities, the third interface allow users to start and control ROSCo behaviors. Users start by selecting the desired behavior from hierarchically organized menus, similar to music play lists in an MP3 player, then telling the behavior to start.
Shown above are ROSCo behaviors for using the PR2 to give a back rub and use a hand fan. Although uncommon in research robotics, we believe such uses might be commonplace once general purpose robots are in homes and users empowered by expert interfaces such as ROSCo to create new robot capabilities. Uniquely, general purpose robots has the potential to displace many smart and motorized devices in homes as well as create new opportunities for automation.
In the BackRub demo (named after a famous search engine), users first activate the behavior through ROSCo's web interface then lean into the PR2's grippers for a "back rub." Stopping the behavior can also be carried out through the web interface (which runs on mobile phones).
With the fanning behavior, users first activate the behavior through ROSCo's web interface then place the fan in the robot's gripper, after which the behavior plays a looping motion.
Home environments are complex and varied presenting significant and unsolved challenges to robotics. Tools such as ROSCo can help address these difficulties by producing autonomous capabilities that can be used during shared teleoperation. In such scenarios, robot autonomy reduces the mental demand on users and teleoperator control enable mobile manipulators to operate more robustly in home environments.
We tested our system with Henry Evans, a man with quadriplegia, in his home as part of a teleoperation interface where repeatable actions such drawer opening are performed using ROSCo behaviors, and harder tasks are performed by human teleoperator. Using such systems based on shared autonomy, we hope to restore the ability to live independently to similarly motor impaired individuals.
For questions, contact either Hai Nguyen or Charles C. Kemp, the authors of this work. The code is available in the packages rcommander_core and rcommander_pr2 at ros.org. We also refer readers to our publications for more information: