IROS2019国际学术会议论文集 1567

Physical Fatigue Analysis of Assistive Robot Teleoperation via Whole body Motion Mapping Tsung Chi Lin1 Achyuthan Unni Krishnan2and Zhi Li1 Abstract Robot teleoperation via motion mapping has been demonstrated to be an effi cient and intuitive approach for controlling and teaching the whole body motion coordination of humanoid robots However the physical fatigue in the usage of such robot teleoperation interfaces may prevent this approach to be widely used in large scale by diverse workforce populations As a result this paper conducts a user study to in vestigate the physical fatigue of teleoperators in the whole body motion mapping teleoperation of a mobile humanoid assistive robot Through a Vicon motion capture system participants teleoperated the robot to perform general purpose assistive tasks that involve reaching to grasp bimanual manipulation loco manipulation and human robot interaction We assess the physical fatigue based on surface electromyography sEMG measurement and compare it between different tasks and muscles Our analysis results indicate that 1 Fatigue happens more in the tasks that involve more precise manipulation and steady posture maintenance 2 Deltoids Biceps and Trapezius are used more for such tasks and thus have more fatigue than others These fi ndings imply that automating the fatigue causing task components may reduce the physical fatigue in motion mapping teleoperation I INTRODUCTION Tele presence tele action TPTA robots have great po tential to support remote healthcare in contagious disease treatment infection control and to provide in home assistance to the elderly and disabled 1 Such robots may need to respond to a wide variety of patient caring and assistive tasks in a cluttered environment and therefore requires the involvement of human experts via teleoperation However even after training with high fi delity robots and input devices it is generally diffi cult for humans to teleoperate complex nursing robots with multiple manipulation navigation and active perception components Many novel interfaces have been developed to control and teach a complex robot platform including mapping whole body human motion to the teleoperated robots 2 This approach is particularly suitable for a humanoid robot for controlling the motion co ordination such as reaching to grasp bimanual manipulation loco manipulation 3 and active perception 4 Although intuitive and effi cient teleoperation via motion mapping may cause more physical fatigue than conventional desktop robot controllers The physical fatigue assessment for various robot teleoperation tasks will enhance the understanding of the characteristics of such robot teleoperation interfaces It 1 Tsung Chi Lin and Zhi Li are with the Robotics Engineering Program Worcester Polytechnic Institute Worcester MA 01609 USA tlin2 zli11 wpi edu 2 Achyuthan Unni Krishnan is with the Mechanical Engineering De partment Worcester Polytechnic Institute Worcester MA 01609 USA aunnikrishnan wpi edu also facilitates the fatigue adaptive interface design that may remove the barrier for such teleoperation interfaces to be used in a large scale and by diverse healthcare workers This paper conducted an experimental study to assess the physical fatigue in assistive robot teleoperation via whole body motion mapping Through a Vicon motion capture system a teleoperator s motion was mapped to control a mobile humanoid nursing robot to perform several general purpose assistive tasks We recruited eight participants and collected their motion and sEMG data during the robot teleoperation tasks We compared the fatigue level between muscles and tasks Our results show that 1 Deltoids Biceps and Trapezius tend to be more used and fatigued for all the robot teleoperation tasks 2 The physical fatigue was mostly caused by tasks that involve precise manipulation and camera control resulting in increased stationary poses and errors This implies that automating such fatigue causing tasks may reduce teleoperation effort Fig 1 Tele robotic Intelligent Nursing Assistant TRINA system II BACKGROUND ANDRELATEDWORK A Assistive Robots for Complex Motion Coordination Tasks Recently developed assistive robots are equipped with multiple action and perception components to perform nursing tasks that involve complex motion coordination and intimate physical human robot interactions 5 The complexity of humanoid robot platform demands more effi cient and intuitive teleoperation interfaces particularly for coordinating the robot manipulation and navigation actions with their active perceptions For instance one recently developed nursing and assistive robot Tele Robotic Intelligent Nursing Assistant TRINA consists of dual armed humanoid torso Rethink 2019 IEEE RSJ International Conference on Intelligent Robots and Systems IROS Macau China November 4 8 2019 978 1 7281 4003 2 19 31 00 2019 IEEE2240 Robotics Baxter an omnidirectional mobile base HStar AMP I and two three fi ngered grippers Righthand Robotics ReFlex grippers see Fig 1 The visual sensor suite of this nursing robot includes a180 panoramic camera Panacast that can turn around with robot head a Microsoft Kinect 2 fi xed to the robot s chest and two Intel RealSense F200 3D cameras wrist cameras that can move with the robot end effectors When safety is guaranteed some nursing tasks can be performed autonomously following pre defi ned task and motion plans 6 However the majority of nursing tasks involve coordinated precise and dexterous manipulation and navigation in cluttered human environment and therefore can only be performed reliably via teleoperation Our prior study evaluated the physical capabilities of TRINA over 26 frequently performed nursing tasks The challenging motion coordination performed frequently in various nursing tasks include reaching to grasp bimanual manipulation and loco manipulation 1 For instance in the task of moving a patient transfer bed a teleoperator needs to control the robot to reach to grasp the bed sides use bimanual manipulation to adjust the moving direction of the bed in a cluttered patient room and use locomotion to push the bed forward In the task of food preparation precise reaching to grasp motion is necessary for picking up the straw and inserting it into the beverage cup while bimanual manipulation is used to lift the food tray from the preparation counter to a food serving cart and then to the patient s bed table Similarly in the barcode scanning task the robot needs to reach to grasp a barcode scanner and align it precisely with the barcode on medical supplies In the patient bed cleaning task the robot needs to use both hands alternatively to grasp and remove the blanket off the patient Manipulating this large heavy and deformable object requires skillful bimanual coordination Among all the available teleoperation interfaces whole body motion mapping seems to be the most intuitive and effi cient interface for teleoperating complex robot motion coordination B Robot Teleoperation via Motion Mapping Human motion mapping systems have been recently used for controlling and teaching the motion coordination of complex robot systems While data gloves are mostly used for multi fi ngered robotic hands 7 motion mapping devices such as motion capture systems e g Vicon 2 portable motion capture devices e g Microsoft Kinect 8 Xsens MVN 2 and exoskeletons 9 are most natural to control humanoid robots to perform bimanual manipulation locomo tion and whole body coordination Compared to conventional teleoperation interface motion mapping have been favored for providing more intuitive control and a more immersive teleoperation experience The outcomes of tele action control can be easily predicted which signifi cantly reduces the learning effort associated with many widely used teleoperation interfaces e g Gamepad controller 10 Despite these advantages the non trivial physical fatigue of motion mapping may prevent such interfaces being widely used for robot teleoperation particularly for long time and on daily basis Prior research has investigated the physical fatigue associated with conventional tele robotic interfaces For instance the physical fatigue during tele robotic surgery causes muscle tremors and may result in dangerous situations in critical surgical steps 11 Besides fatigue level also negatively affects the Quality of Teleoperation QoT which indicates a teleoperator s confi dence in commands and decisions 12 Beyond the teleoperation of medical robots increased fatigue results in reduced performance during the teleoperation of Unmanned Ground Vehicles UGV 13 Based on these results it is necessary to evaluate the physical fatigue in teleoperation via motion mapping interfaces for a better understanding of the interface characteristics and for the development of teleoperation assistive technologies C Approaches for Physical Fatigues Assessment Muscle fatigue has been defi ned as any exercise induced reduction in the maximum capacity to generate force or power output 14 Assessment of physical fatigue can be based on the measurements of force power and torque Besides heart rate can also be used to detect muscle contraction and infer overall physical fatigue level 15 Among all the approaches sEMG measurement have proven to be more effective as it measures muscle activity in a non invasive and real time environment and provides the ability to monitor physical fatigue of a particular muscle 16 Earlier approaches for physical fatigue assessment tended to be more subjective where the sEMG trace was visually evaluated without clearly identifying the threshold for muscle contraction onset 17 More recent computational approaches used signal processing methods to determine the muscle fi ring time and identifi ed the appropriate threshold for muscle contraction onset based on a trial and error approach 18 Indices for assessing sEMG based fatigue generally fall into two categories traditional and novel approaches Traditional approaches like amplitude based parameters and time frequency distributions to non linear parameters are suitable for evaluating isometric fatigue On the other hand novel approaches are more suitable for assessing dynamic muscle fatigues These approaches may utilize dynamic muscle fatigue model differential equations mechanomyography 19 inertial measurement unit IMU measurements 20 power spectral indices and kinetics and kinematics 21 from motion data Because motion mapping teleoperation involves both isometric and dynamic fatigue it is important to distinguish the fatigue types and choose the right indices for accurate assessment of fatigue level III METHODOLOGY A Experimental Setup Fig 1 shows the robot platform used in our experiment The operator console provides whole body human motion mapping interface which can teleoperate the mobile humanoid robot to perform manipulation navigation and active perception Table I describes how the robot can be controlled using human motion Since the operator has to move the lower limbs to control the base of the robot for locomotion the mapping scheme requires the operator to be standing During 2241 the experiment the Vicon motion capture system record human motion at 100 Hz and stream human motion for robot control at 50 Hz Wireless sEMG sensors TrignoTM from Delsys Inc are used to record the EMG signals at 1 000 Hz for 14 individual muscles Anterior Medial and Posterior Deltoids Biceps Brachioradialis Trapezius and Erector Spinae Muscles of the left and right sides of the body which are most involved in controlling the motion of the upper body Teleoperation InputRobot Function Robot s Upper Body Hand position 2 Stacking stack food containers in the instructed order 3 Laundry collect towels and blankets into a laundry basket and take them out Each participant performs each task for three repetitions For each iteration of each task the items were replaced in the same positions to ensure that the tasks were executed in largely the same manner Before the experiment participants were allowed to become familiar with the TRINA system through a formal training session Then we perform a maximum voluntary contraction MVC test for each participant in order to normalize the EMG signal with respect to the maximum force generated by each muscle 23 Participants took a one minute break after fi nishing each task iteration to take quick survey of robot teleoperation experience The operators moved on to the next task only if they felt completely rested and they felt no fatigue in any area of their body C Data Analysis EMG Raw Signal Motion Mapping InterfaceTask Performance High Pass Filter 10 Hz Full Wave Rectification Low Pass Filter 6 order elliptic Band Pass Filter 25 Hz 25 in muscle efforts As shown in Fig 4 the users were grouped based on their utilization of these 2 muscles 2 people in the low 40 muscle usage categories Fig 5 compares the efforts among the muscle groups and user groups for each tasks The ANOVA analysis indicates that the signifi cant differences in the usage of the Middle Deltoid Biceps and Trapezius between the user groups of high and low efforts Overall there was no signifi cant difference in muscle usage across the tasks Across the tasks and participants the Anterior Deltoid was largely used while the Posterior Deltoid was least used Compared to the groups of low and medium muscle effort the user group of high muscle effort has signifi cantly increased usage of Biceps and Trapezius These muscle groups were largely used in robot teleoperation tasks to maintain steady arm postures for performing precise manipulation and for adjusting and holding the wrist cameras We observed several factors that may be related to the muscle effort difference among users through post study interview We noticed there is no signifi cant difference of task completion time between the groups of users of high and low muscle usage see Fig 6 However the group of low muscle efforts tend to be more familiar with robot motion range interface control function teleoperation delay and task steps B Physical Fatigue Analysis We compared the physical fatigue level for the muscle groups actively used based on muscle effort analysis Shown in Fig 7 we compared the physical fatigue level averaged among the tasks and muscle groups The muscle groups we compared include a Trapezius b Biceps c Middle Deltoid and d Anterior Deltoid Across participants we computed the averaged fatigue index for dynamic contrac tion for each percentage of each robot teleoperation task including the three repetitions We determine the averaged fatigue index to be above red color or under green color the threshold we experimentally determined in Section III C 2 The averaged fatigue level across participants shows 2243 020406080100 Low L Low R High L High R Low L Low R High L High R Low L Low R High L High R Muscle GroupsMuscle Usage Deltoid Middle Biceps Trapezius Fig 5 The left section discusses about the muscle effort comparison among tasks and among muscle groups including the a Anterior Deltoid b Middle Deltoid c Posterior Deltoid d Biceps e Trapezius f Lower back and g Forearm The muscle effort is represented as the percentage of time the muscle is contracted The muscle effort analysis distinguishes the users of low medium and high muscle usage The right hand section displays the result from ANOVA analysis The blue lines represent the muscle usage of low muscle usage group and the red line represents the high muscle usage group CollectingStackingLaundry 300 600 900 0 LowMediumHigh Time s Fig 6 The task completion time across usage groups and tasks Laundry 1230 Number of Repetition Stacking 1230 Number of Repetition a b c d Collecting 1230 Number of Repetition Fig 7 Fatigue comparison across tasks that the Trapezius and Biceps tend to be more fatigued than Middle Deltoid and Anterior Deltoid This may be because the Trapezius and Biceps are smaller than the Middle and Anterior Deltoid and therefore have higher force generating capacities 26 In addition the collecting and stacking task caused more fatigue than the laundry task which is consistent with our survey result in Section IV C This may be because the laundry task involves more gross manipulation of soft and non fragile objects and less movements that hold the arm for the careful adjustments in the control of precise manipulation and active perception TasksSubtasks Stacking Collecting Laundry Fig 8 Survey results a k are the potential fatigue causing teleoperation actions see Section IV C C Survey Analysis We surveyed the level of physical demand for each task and the impacts of several possible fatigue causing factors Shown in Fig 8 the reported physical demand levels of the three tasks is ordered as Stacking Collecting Laundry corresponding to the amount of precise manipulation and active perception each task requires The teleoperation actions that cause more fatigue above 3 in rating include a holding steady pose of the wrist camera for observation b aligning objects c raising arm up for long time during teleoperation d grasping small object and e adjusting camera view for best perspective The less fatigue causing actions below 3 in rating include f picking objects from the top g grasping large object h picking up object from the side i placing object j carrying grasped objects and k lifting left to change camera The results confi rm the fatigue causing task characteristics and teleoperation actions implied in the muscle effort and fatigue analysis 2244 V CONCLUSION ANDFUTUREWORK As far as we know this paper may be the fi rst effort to analyze the physical fatigue in robot teleoperation via motion mapping Through the analysis of muscle efforts and fatigue we identify the task characteristics and teleoperation actions that tend to cause physical fatigue Automating such teleoperation tasks may effectively reduce the physical fatigue particularly for novice users Our study is limited to a specifi c robot