IROS2019国际学术会议论文集1936

Design of a Semi-Humanoid Telepresence Robot for Plant Disaster Response and Prevention Irvin Steve Cardenas and Jong-Hoon Kim1 AbstractWe introduce the end-to-end design of WRS- Telebot, a semi-humanoid telepresence robot platform for disaster response. Its software design focuses on enabling high- fi delity immersive telepresence control of its manipulators and proactive multilateral control enabled by anomaly detection of the operators physical state. The robots key mechanical design features include a transformable mobile base that allows the robot to switch between two drive systems: caterpillar-track and differential drive, depending on the encountered terrain. The upper body is composed of a retractable dynamically balanced 20-DoF humanoid torso, that can fold entirely, allowing the robot to navigate through space-constrained environments. We evaluate its performance during off-site tests to show its usability. I. INTRODUCTION Most traditional plant disaster response and recovery robot platforms are mechanically composed of a track-mobile base and a single manipulator as noted by the NIST 6. Both the locomotion system and manipulators of such robots incorporate control algorithms that are not reactive to en- vironmental constraints, e.g. terrain with rubble, man-made structures such as stairs. Neither are they reactive to operator constraints that range from operator discomfort, to inade- quate operation. Commonly, they are controlled via direct- teleoperation through interfaces that often require domain expertise or dedicated training - highlighted by initiatives to develop robot training facilities and programs. Overall, disaster prevention, response and recovery is a fi ght against time. Ineffective locomotion or manipulation can have a negative impact in the overall time it takes a robot to complete a mission. Similarly, inadequate collab- oration between the robot and the operator, or inadequate multilateral operation and collaboration amongst a team of human operators, can produce unexpected results and lead to unnecessary actions taken by the robot operator(s). Most obvious, mechanical, software or network failures, and performance degradation inadequately handled, can render the robot useless. In this paper, we present the design of our semi-humanoid telepresence robot platform used for plant disaster prevention tasks - WRS-Telebot (Fig. 1). The key considerations in the design of the platform are: (1) intuitive control, (2) effi cient locomotion, and (3) enhanced manipulation and navigation. The main contributions of our work are a system design that leverages telepresence features to enhance operational control, the design of a proactive multilateral control (PMC) system, a set of open-source software control interfaces, and Authors are with the Computer Science Department, Kent State Univer- sity, Kent, OH 44242, USA.1 Fig. 1: WRS-Telebot a series of lessons learned that describe the application of the robot and the control modalities, along with practical guidance for system engineers, designers and developers. II. RELATEDWORK Traditional consumer telepresence robots can be regarded as mobile video conferencing platforms on wheels 18 10. As such, they disregard the fundamental goal of telepresence described by early pioneers such as Marvin Minsky - the goal of achieving the feeling of ”being there” 15. In the context of humanoids and disaster response robots, efforts to integrate telepresence technologies has also been made. In 8, the authors present a bi-manual dexterous robotic system that makes use of CyberGloves, an IMU-based motion cap- ture system and a head-mounted display (HMD). A similar approach is presented by 4. Others like 5 use an optical tracking system and a virtual reality (VR) HMD to control a full-scale bipedal humanoid. While 16 applies similar technologies to control a bi-manual robot with a two-wheeled balancing system. Overall, these common approaches to telepresence control rely on the interplay between motion capture and VR technologies. Furthermore, they do not fully consider the role of, what we refer to as - the user-control experience (UCX), the negative impact of extended-use of such control systems, or how data generated throughout the control experience can be leveraged to enhance the operation of the robot. As noted in the literature, the use of immersive technolo- gies (e.g. HMDs) and interaction with virtual experiences 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Macau, China, November 4-8, 2019 978-1-7281-4003-2/19/$31.00 2019 IEEE2748 can have short-term and extended negative health impacts, ranging from fatigue or dizziness to postural disequilibrium 12. But, to our benefi t, such effects can be identifi ed and even rectifi ed during an ongoing virtual experience, by fi rst monitoring and assessing the physiological state of the user 12. To this end, and unlike other telepresence-based systems, our system places emphasis on understanding the operator and leveraging collected data to enhance the oper- ation of the robot. Most notably, it incorporates a physical- state monitoring system into a telepresence control garment (Fig. 2). The data is continuously processed and facilitates transition in and out of shared-control, and can trigger a control handover as part of a multilateral control scheme 2. Fig. 2: Telesuit Construction Details Transformable and multi-mode locomotion robots for disaster response have also been presented. For example, Aero employs multi-legged walking to negotiate complex terrains and wheel-drive to navigate through simple or fl at terrains 19. Other robots like 9 3 employ transformable wheels that extend into legs. While, DRC-HUBO+, win- ner of the DARPA Robotics Challenge (DRC), is a hu- manoid transformer robot that can switch between wheel and bipedal walking mode 13. Similar to previous literature, its dual mode provides a practical advantage during navigation through different types of environments. The walking mode allows the robot to traverse terrains where the ground level varies, e.g. areas of rubble. Whereas its wheel mode allows the robot to traverse through fl at terrain quickly and with high stability. Intrinsically, walking mode allows for extended arm reach. Although, the DRC presented a series of humanoid robots for disaster response, the adoption and, most impor- tant, reliable use of bipedal robots for such tasks has not been achieved 11. Factors such as the control complexity and development costs of bipedal systems play a role in this. Thus, our robots lower body is a non-bipedal vehicle whose design is stable and relative simple to implement, but sophisticated enough to allow more intelligent locomotion strategies than traditional vehicle disaster robots. The design of our telepresence control system and the control strategies applied are infl uenced by the contribu- tions of Coactive Design and its supporting contributions (e.g. Joint Activity Theory, Human-centered Computing and Collaborative Control) 7. As well as previous work on multilateral teleoperated systems in 17. We extend such work to introduce and exploit the features of immersive telepresence, and tackle the design of the robot platform with a user-centric mindset - aiming to understand the user experience (UX) behind the control of our robot and leveraging the data produced. III. DESIGNOVERVIEW The robot platform, WRS-Telebot, was designed with the purpose of allowing immersive telepresence control as the primary control modality. Key design goals include: (1) intuitive control, (2) effi cient locomotion, and (3) enhanced manipulation and navigation. Details on the telepresence garment and its health monitoring system are presented in 2. In this section, we elaborate on how the garment and our task-level shared autonomy mechanism play a role in meeting our design goals. Figure 3 shows a simplifi ed system diagram of the platform. IV. SOFTWAREDESIGN ANDMIDDLEWARE The software is designed with the following fi ve consid- erations: (1) support for telepresence under optimal and de- graded network, (2) support collaborative operation - human and artifi cial, (3) provide control transparency, (4) allow for a scalable user control experience (UCX), (5) optimize for desirable system attributes - modularity, redundancy, fault- tolerance, fl exibility and integrity. The communication layer is a mixture of the ROS middleware and the RTI Data Distribution Service (DDS) middleware, with a DDS-ROS communication bridge. We employ these technologies to minimize the risk of a central point of failure presented by the ROS broker-based pub/sub architecture. Software processes on the operator control station (OCS) leverage ROS middleware, custom software was implemented as nodes, services and actions for the control and monitoring interfaces. Processes on the robots onboard computer(s) employ a mixture of ROS and DDS communication. DDS automatic peer discovery brings a great advantage to our telepresence control by decentralizing how communi- cation is established between operator(s) (master) and robot (slave) nodes. It also simplifi es how we achieve liveliness and maintain application-level Quality-of-Service (QoS) for different processes, e.g. robot video stream or the garments high-throughput motion output. Implementing a system that leverages DDS is similar to ROS, where a set of publisher and subscribers are written along with a set a of topics. The key differences are that publishers and subscribers must be part of a ”Domain”, and must explicitly implement ”Data Writers” and ”Data Readers” respectively. Additionally, DDS allows users to defi ne unique QoS policies or make use of off-the-shelf policies. Further details can be found in our repository 1. During telepresence control, high-resolution low-latency video stream is necessary to maintain the operator immersed in the environment. To mitigate the impact of degraded network condition on the video feed, the robot platform can switch between a video stream shared over a traditional wireless network and a video stream captured by a USB Radio Frequency (RF) device in the OCS. Our current USB RF video capture device has 1-mile range and less than 10 2749 Fig. 3: WRS-Telebot System Diagram and Multilateral Control Framework ms latency. The far right graphic in Fig. 5, shows the main component of our telepresence control interface (TCI). The RF video stream is rendered in the large boxed UI container. While, the lower boxed UI container displays the local video feed of the front-facing camera of an HTC Vive VR system we employ. Additional elements that provide operator-state feedback can be noted in the interface. This interface was developed as desktop Unity VR application. A video feed from either the robots RGB cameras or the video feed captured by the USB RF device are displayed on the VR HMD. Simultaneously the head motions captured by the HMD are sent to WRS-Telebot to replicate. A websocket peer-to-peer service monitored by daemon was implemented that allows continuous access to the feed. A ROS video server was disregarded as an option due to the added complexity and the risk of depending on ROS broker- messaging. Concerns over the throughput of head motion data were ignored due to (1) the high publishing rate of the HMD data and (2) the observed ability of an operator to effortlessly adjust to slight misalignment in the robot head confi guration. The 2018 WRS Disaster Prevention Challenge, did not present a degraded network. But, we still chose the USB RF video capturing device as our primary video source due to the high-resolution and low-latency video achieved when using a 4K camera (e.g. GoPro HERO7). This was not only practical and effi cient, but is also no different to using other video sources - allowing us to process the video stream as if it were from a plain USB camera and superimpose graphical markers, as well as perform online analysis of the video. Fig. 3 shows a simplifi ed system diagram of the robot platform. The left side presents the components of the robot, while the right side presents the OCS components. V. MECHANICAL ANDHARDWAREDESIGN Similar to the literature, our robot can employ two modes of locomotion: wheeled differential-drive mode to move quickly through normal terrain, and track-based mobility mode to traverse through terrains where a larger support polygon can bring more stability and traction. To employ differential-drive, two motors located in the mid area of the base are actuated simultaneously to bring the tail downwards and raise the rear of base upwards. Motors 18 and 17 in Fig. 4 are dynamically actuate to maintain the robots center of mass during that step. Table I shows the specifi cations of the robot. Component Specifi cationComponent Specifi cation Height170 cmDoF Arm (2)7 Wingspan180 cmDoF Gripper (2)2 Weight62 kgMobile Base7 DoF Total30Power 11.1 V3 Cell LiPo 24 Ah DoF Upper27Power 21.1 V6 Cell LiPo 20 Ah DoF Head3 TABLE I: Specifi cation of WRS-Telebot Fig. 1 shows two of the four possible confi gurations of the robot - differential-drive extended-reach (DDER) and track-drive terra-reach (TDTR) respectively. Two variations in the position of the body lead to track-drive extended- reach (TDER) and differential-drive terra-reach (DDTR). TDER, DDER and TDTR are mostly employed due to the observed stability during locomotion. In DDER confi guration the robot gains an extended reach of 0.4 meters, but requires dynamic actuation of torso when the manipulators are fully extended. We apply a dampening mechanism using a PID controller on the 3-DoF waist during the latter, and also to suppress disturbances during TDER locomotion across complex terrains, including climbing stairs. The robot is equipped with two 7 DoF manipulator arms. This allows for correlation between the operators 7-DoF arm movements captured by the Telesuit. Fig. 4 shows the upper body confi guration. Two modular end-effectors can be attached to the manipulator arms. Commonly the robot is equipped with 3-DoF grippers controlled through interfaces such haptic- gloves or VR-joysticks. The base speed of track-drive locomotion is 1 mph with a maximum 5 mph depending on the applied gear ration. Differential drive has a base speed of 1 mph and a maximum of 15 mph. The robots head is equipped with a pair off-the- shelf RGB cameras. The 3-DoF head pans (+,-) 180 degrees, tilts (+,-) 60 degrees, and is able to yaw (+,-) 90 degrees, this allows the operators head motions to be replicated adequately. The mobile base is equipped with an Orbbec 2750 Fig. 4: Lower Body CAD Rendering and Upper Body Joint Confi guration Astra Pro 3D camera which captures both RGB and depth images at low latency and has a 0.6m - 8m range. The Astra Pro is used for SLAM. A Hokuyo UTM-30LX-EW laser rangefi nder is mounted in the robots torso and used during manipulation tasks and for 3-dimensional reconstruction of the environment. It allows the robot to scan a 270 fi eld with an angular step of 0.25, and distance of 30m. VI. CONTROLSTRATEGY ANDINTERFACES Telepresence control through the garment is established as the primary means of operation 2. Consequently, within a team, this may seem to establish a strict control hierarchy in which the operator wearing the garment holds central power over controlling the robot. Within our system, this is quite the opposite. Our control system is both architecturally decentralized and logically decentralized. Architecturally in the sense that the technologies such as DDS allow for indi- vidual operators using individual computers to interact with the system and control the robot. It is logically decentralized in the sense that any operator at any point in time can be replaced - the state of command preserved and picked up by a willing and able operator. What the system does not provide is a recipe for decentralized governance, or decentralizing a human social construct like command hierarchies. Hence, during multilateral operation, it is highly likely that a team leader is selected and assumes most, or a high level of, control of the robot. A. Proactive Multilateral Control and Handovers Bilateral systems have been extensively discussed 17, but limited literature exists on exploring high-dimensions of multi-lateral control; Calling for approaches such as extensive team training and other activities to synchronize operators and acquaint them with collaborating with each other. Within our work, we lightly touch upon the problem of coordinating human operators, and developed a simple scheme for collaboration beyond trilateral control. It consid- ers scenarios where the set of human operators may include good, malicious or faulty actors 1 To the best of our knowl- edge, work in the fi eld of multilateral teleoperation does not assume a threat model that includes such actors. This could be, in particular, due to the application of multilateral teleoperation to highly-focused used cases (i.e. telesurgery and robot-assisted surgery); or due primarily to the fact that up to this point in time complex robot agents have not been fully deployed into the wild. During the operation of our humanoid telepresence robot we assume a modest threat model. We do not assume, as of this work, that operators may collude to cause harm to the robot or purposely misuse the robot; rather, we assume that an operator in immersive telepresence may inadvertently cause injuries to his or herself due to the effects of telepres- ence, or may become unaware of his or her own performance. Hence, also leading to inadequate operation. To this end, the telepresence control garment monitors the physical state of the operator and is able to provide an online report - operator state monitoring. The data is further used to detect anomalies in heart-rate patterns and sp