IROS2019国际学术会议论文集 0370

Teleoperating Robots from the International Space Station Microgravity Effects on Performance with Force Feedback Bernhard Weber Ribin Balachandran Cornelia Riecke Freek Stulp and Martin Stelzer Abstract Sending humans to Mars surface to build habitats is for now prohibitively dangerous and costly An alternative is to have humans in orbiters teleoperating robots on Mars to construct habitats deferring human arrival until these habitats are fi nished This paper describes the Kontur 2 experiments in which the feasibility of this scenario was tested with the International Space Station as an orbiter a cosmonaut operating a force feedback joystick as an input device for teleoperation and Earth as the planet where the teleoperated robot is located In particular we focus on human teleoperation performance which is known to deteriorate under conditions of spacefl ight We investigate whether the provision of force feedback at the joystick is as benefi cial as under terrestrial conditions Our results show that to support humans operating in weightlessness haptic assistance needs to be adjusted to the altered environmental condition I INTRODUCTION When humanity takes the big step of building habitats on Mars the small steps required for the actual construction will probably not be taken by humans First of all the Martian environment is too dangerous and adverse for humans hence the habitats And for now transporting humans to Mars surface and back to Earth is too challenging and costly These small steps will therefore most likely be taken by robots 1 The following arguments speak in favor of the scenario where robots on Mars surface are teleoperated from humans in Mars orbit First the autonomy of robots is not yet such that they can be sent to Mars to build habitats without intervention so partial teleoperation is inevitable Second teleoperating robots from Earth is not possible as signal return times to Mars are between 4 and 24 minutes Third the costs of sending and returning humans to and from Mars orbit is far more feasible than to and from its surface As illustrated in Fig 1 the Kontur 2 experiments consti tute a step towards this vision by testing the teleoperation of robots on Earth from International Space Station ISS 2 In this paper we describe these experiments which were the fi rst to enable real time robotic teleoperation performed with a 2 DoF force feedback joystick onboard the ISS A central aspect of these experiments which we focus on in this paper1 is the impact of using force feedback to perform such tasks Besides natural interaction with the remote environment force feedback improves the sense of being physically present at the remote site i e telepres ence 4 and furthermore physically hinders the operator All authors are with the German Aerospace Center DLR Robotics and Mechatronics Center RMC M unchner Str 20 82234 We ling Germany corresponding author is Bernhard Weber phone 49 8153 28 2194 fax 49 8153 28 1134 e mail Bernhard Weber dlr de 1 3 rather focused on weightlessness effects without haptic effects Fig 1 Teleoperation Setup with a Robot on Planetary Surface controlled by an Astronaut located in an Orbital Spacecraft to apply exaggerated forces Especially for the assembly and construction tasks where parts must be put in contact with each other it is important to feel the forces that are being exerted Compared to alternative feedback modalities such as visual or vibrotactile feedback of collisions assembly precision mental workload and spatial orientation are im proved by force feedback as reported in 5 Two quantitative reviews also reported substantial performance gains when providing force feedback in teleoperation 6 7 Force feedback signifi cantly improves task success and accuracy force regulation and completion times The results described in this paper aim at verifying whether these improvements also hold when teleoperating under microgravity conditions in an orbital vehicle Before describing the methods and results we fi rst provide an overview of the known challenges in force feedback teleoperation as well as the specifi c research questions we address in this paper II FORCEFEEDBACKTELEOPERATION KNOWNCHALLENGES ANDOPENQUESTIONS Force feedback teleoperation of robots e g for planetary exploration controlled from an orbital spacecraft brings two major challenges in terms of overall system control First teleoperation stability i e the human machine interface does not generate any vibrations or oscillations and transparency i e the operators have the impression of interacting directly with the remote environment 8 have to be guaranteed despite telecommunication delays between the operator s site and the remote robotic site Several bilateral control archi tectures have been developed to meet these requirements 9 10 However there is a trade off between both criteria when 2019 IEEE RSJ International Conference on Intelligent Robots and Systems IROS Macau China November 4 8 2019 978 1 7281 4003 2 19 31 00 2019 IEEE8138 communication delays occur Since stability is crucial in terms of safety and operability perfect transparency cannot be achieved 11 Hard contacts can only be displayed introducing stabilizing effects such as motion damping 10 12 13 Apart from technical system control a second main challenge of space teleoperation is human control during spacefl ight Basic perceptual and sensorimotor skills which are crucial for human performance are potentially affected in microgravity Prior research investigated the impact of microgravity on human perception and sensorimotor per formance e g hand eye coordination under conditions of microgravity induced by parabolic fl ight or spacefl ight One consistently observed reaction on the altered grav itational context is a general slowing down effect Motion velocity decreases and motion times increase accordingly while performing aiming movements This response pattern is evident during parabolic fl ight 14 15 as well as during spacefl ight 16 10 140 and 172 days in space 17 1 7 and 14 days 18 5 398 19 8 days in space 20 4 18 days in space Also relevant for space teleoperation performance is the precise regulation of interaction forces Yet there are fi ndings that the production of fi nely graded forces is deteriorated in microgravity Compared to terrestrial conditions peak forces during surgical tasks such as suturing or knotting have been reported to be substantially increased during 0g episodes of parabolic fl ight 21 22 Furthermore studies on force production with isometric joysticks showed that peak force level is elevated during parabolic fl ight 23 24 and in simulated weightlessness induced by water immersion 25 The mechanisms underlying the reported impairments of human perception and performance in microgravity are still not fully understood Most of the cited studies investi gating microgravity during short term exposition attributed the impairments to disturbed proprioception e g 25 26 27 28 According to this approach muscle spindle ac tivity which is crucial for proprioception is altered by the weightlessness of the body and limbs 29 However it is known that sensorimotor programs typically adapt to altered afferent information within four weeks 30 Interestingly researchers also documented performance losses for later mission phases 16 18 They explained the observed slowing of aimed arm motions as a strategy to avoid high reactive forces which are diffi cult to compensate by the weightless body and limbs In sum the fi ndings above clearly suggest that one main obstacle for future force feedback teleoperation missions is the potential degradation of human skills during spacefl ight even beyond the initial phase of adaptation However so far there has been no empirical investigation of a real teleoperation scenario comparing human performance under terrestrial and spacefl ight conditions In the present work the general rationale of planetary exploration with a teleoperated robot on a planetary surface here Earth and the human operator being located in an orbital spacecraft here the ISS was realized In practice demanding operations such as extravehicular activities EVAs are not scheduled in the initial mission phase due to the ongoing physiological adaptation to micro gravity conditions 30 Therefore teleoperation performance after six weeks in space is investigated in the present study As one major research question it is explored whether the slowing down pattern still emerges during a teleoperated free motion task Introducing a mechanical counterforce i e damping as a measure to stabilize teleoperation could further moderate a potential slowing effect On the one hand fi ndings on impaired force perception and regulation in microgravity suggest the effect could be further intensifi ed when motions are damped On the other hand it is also conceivable that damping not only stabilizes in a technical sense but also stabilizes the operator s motions which could be particularly helpful in space Therefore the following open research question was formulated 1 Is there a slowing down effect during teleoperated free motion tasks after having completed adaptation to micro gravity and is this effect moderated by motion damping generated by the force feedback controller Regarding teleoperation contact tasks the main question is whether and to which degree force regulation is deterio rated in microgravity after having completed adaptation By comparing performance with and without force feedback we are able to investigate whether the expected positive effect of haptic contact information is comparable to the terrestrial effects Thus the second research question was 2 How large is the positive effect of providing force feed back during teleoperated robot environment contact tasks in microgravity compared to terrestrial conditions The present work delivers two central contributions by an swering the above research questions 1 the extent of human performance degradation after adaptation to microgravity during a real telerobotic mission from ISS and 2 the extent to which haptic feedback i e damping during free motion and force feedback during contact tasks is benefi cial under conditions of spacefl ight compared to terrestrial conditions III METHODS A Subject The participant was an experienced 42 year old male cosmonaut during his third space mission with a total of 410 days spent in space He was educated as a fi ghter pilot and is a colonel of the Russian Air Force The study protocol was approved in advance by the ethics committee of the Federal Russian Space Agency Roscosmos and the cosmonaut provided written informed consent before participating B Equipment The ISS experiment was conducted in the Russian Zvezda module of the ISS The experimental software ran on a Lenovo T61P 6457 laptop 15 4 display which was at tached to a module rail with a multi articulated arm 8139 a Force feedback joystick The Kontur 2 force feed back joystick developed at our institute 31 see Fig 2 left was also mounted on a rail and was connected to the laptop and the telecommunication system via Ethernet cables The joystick has a 2 DoF 20oworkspace an angular resolu tion of 3 18 10 3degrees a maximum force of 15N and an update rate of 1kHz The cosmonaut worked in an upright posture stabilizing his body with a handhold for the left hand and module rails for the feet see Fig 2 right The pre and post mission setups were similar using terrestrial versions of the equipment with identical specifi cations Contrary to the ISS sessions however no extra handle was required and the experiment was performed in a seated position Fig 2 DLR s Kontur 2 Force Feedback Joystick Left Cosmonaut at the Experimental Workstation in the Zvezda Module of the ISS Right b Teleoperated robot In all settings the cosmonaut controlled a two 2 DoF robot ROKVISS see Fig 3 located at the DLR in Oberpfaffenhofen with the joystick At the outermost segment of this robot a 21cm long pointer was installed A camera right behind the robot focused on a spherical screen with LEDs distributed across four lines for free motion tasks Fig 3 and 5 Tasks including robot environment contacts were performed at a metal structure For these tasks the camera system integrated at the front end of the robot was used An experimental GUI including the respective video streams from the robotic site on earth was displayed on the experimental laptop see Fig 2 right Fig 3 The ROKVISS robot and experimental board with contact structure left and aiming LED screen right The pre mission sessions were performed in the Gagarin Cosmonaut Training Center GCTC in Moscow Here the cosmonaut controlled the robot at DLR via Internet UDP link with a bandwidth of 2 Mbps Data transfer for the ISS sessions was realized using an S Band space link with a channel bandwidth of 256 Kbps for the uplink and 4Mbps for the downlink Stable radio contact windows during ISS overfl ight typically range between 8 10 min The post mission sessions were performed at DLR Here a direct UDP link was used c Controller The bilateral controller for the imple mented master slave teleoperation system Kontur 2 joy stick being the master and the ROKVISS robot being the controlled slave employed a 4 channel architecture The velocity and the measured force from the master interface were sent to the slave side controller and the computed force of the slave controller and the measured force from the slave contacts were sent to the master side to provide force feedback to the operator The signal fl ow diagram of the 4 channel architecture bilateral control is illustrated in Fig 4 Fig 4 Signal Flow Diagram of the 4 Channel Bilateral Controller The H M block represents the human operator and the master device and M Ctrl is the controller at the master side of the teleoperation The force commanded to the master device is the weighted sum of the master controller force fm0 computed force from the slave controller fs2 weighted by gain Gs and the measured forces fefrom the slave experiment board S Env interaction with gain Ge For moving the slave robot the delayed velocity of the master vsdis commanded as the desired value to the slave side proportional integral PI controller S Ctrl which tries to match the measured slave velocity vsto this desired value The force produced by the PI controller fs2is augmented with the force measured from the human master device interface fh amplifi ed by a gain Gh The four blocks T represent the communication time delays between the master and slave sides More details on the bilateral controller its difference from the conventional 4 channel architecture and the passivity based method to stabilize the system despite these delays can be found in 2 C Experimental Tasks Design and Procedure a Experimental Tasks In the whole experimental cam paign fi ve different tasks two free motion tasks aiming and tracking as well as three contact tasks contour following line tracing and spring pulling were performed Here we will concentrate on the aiming and contour following tasks 8140 as the most relevant tasks to answer the research questions During aiming the LED screen was utilized the robot automatically moved onto the center LED as starting position see Fig 5 bottom left Then one of the target LEDs outermost left right top and bottom was activated and had to be reached as quickly as possible with the pointer The contour following task started at the bottom of a curved vertical structure see Fig 5 bottom right with the robot s pointer slightly touching the surface The subject was instructed to slide the pointer along the structure with minimal but permanent contact until reaching the end point Contact had to be re established immediately when deviating from the structure b Design The longitudinal case study was split up in three experimental phases pre mission mission and post mission Pre and post mission data served as a terrestrial baseline for the ISS mission sessions In each phase the fi rst trials were completed with force feedback FF activated and then FF was deactivated in one trial and re activated in the fi nal trial see Table I A potential confounding of gravity effects with time effects e g learning fatigue was avoided by utilizing the very same condition order in all mission phases Still in the pre mission session only four trials could be completed due to the cosmonaut s tight training schedule c Procedure The pre mission session at GCTC in Moscow was conducted 29 days prior to launch After gen eral experimental instruction by GCTC staff the experiment was started On the GUI the video stream the current task task objective and status were displayed see Fig 5 top Fig 5 Experimental GUI and the two tasks Task 1 Aiming pointer at starting LED Task 2 Following a structure contour ideal path overlaid During the pre mission session the cosmonaut completed an initial warm up trial with both tasks received feedback on task performance and then proceeded with three exper imental trials see Table I In each trial the aiming task was completed fi rst and contour following second The ISS sessions were scheduled for the cosmonaut s day 42 43 and 45 after launch Here the individual trials had to be distributed across different mission orbits see Table I One day before each session the cosmonauts received detailed information about the experimental schedule planned tasks and times and detailed task instructions All information for installing the hardware system startup initial joystick test and experimental procedure were provided by the onboard documentation displayed on a tablet computer at module wall One year after launch the terrestrial post mission sessions could be conducted at DLR in Oberpfaffenhofen replicating the experimental mission sequence Although the same bilateral controller was used in all the scenarios the gains Gs Geand Ghwere tuned differently to maximize the controller performance based on the correspondin