A Soft Exoglove Equipped with a Wearable Muscle-Machine Interface Based on Forcemyography and Electromyography IROS2019国际学术会议论文集 2234

A Soft Exoglove Equipped with a Wearable Muscle Machine Interface Based on Forcemyography and Electromyography Anany Dwivedi Lucas Gerez Waris Hasan Chi Hung Yang and Minas Liarokapis Abstract Soft lightweight underactuated assistive gloves exogloves can be useful for enhancing the capabilities of a healthy individual or to assist the rehabilitation of patients who suffer from conditions that limit the mobility of their fi ngers However most solutions found in the literature do not offer individual control of the fi ngers hindering the execution of different types of grasps In this paper we focus on the development of a soft underactuated tendon driven exo glove that is equipped with a muscle machine interface combining Electromyography and Forcemyography sensors to decode the user intent and allow the execution of specifi c grasp types The device is experimentally tested and evaluated using different types of experiments i grasp experiments to assess the capability of the proposed muscle machine interface to discriminate between different grasp types and ii force exertion capability experiments which evaluate the maximum forces that can be applied for different grasp types The proposed device weighs 1150 g and costs 1000 USD in parts The exoglove is capable of considerably improving the grasping capabilities of the user facilitating the execution of different types of grasps and exerting forces up to 20 N I INTRODUCTION According to the World Health Organization WHO in many countries less than 15 of people who require assistive devices and technologies have access to them 1 Impairment of hand function is one of the most common consequences of neurological and musculoskeletal diseases such as arthritis Cerebral Palsy Parkinsons Disease and stroke 2 In order to accelerate the rehabilitation process of impaired people it is important to execute repetitive movements and to try to perform daily tasks 3 Many robotic devices have been developed to assist patients with limited mobility of the hand during physical therapy or to augment the capabilities of able bodied users 4 Although soft underactuated robotic exogloves have be come very popular over the last years they still have several limitations One of these limitations is their inability to ex ecute different types of grasps without requiring mechanical interaction between the user and the device e g pressing buttons or activating differential mechanisms Many studies describe the use of surface electromyography EMG sensors fl ex sensors or other mechanical methods to control the motion of each fi nger of an exo glove in a simplifi ed and in tuitive manner 2 In 5 the authors propose a cable driven portable exoskeleton glove that uses an infrared and a fl ex These authors contributed equally to this work Anany Dwivedi Lucas Gerez Waris Hasan Chi Hung Yang and Minas Liarokapis are with the New Dexterity research group Department of Mechanical Engineering The University of Auckland New Zealand E mails adwi592 lger871 whas626 cyan609 aucklanduni ac nz minas liarokapis auckland ac nz Fig 1 The muscle machine interface consists of a soft wearable sleeve that accommodates multiple Forcemyography FMG and Electromyography EMG sensors The muscle computer interface is connected to the control box that houses the four actuators which control the motion of the soft robotic exoglove sensor to actuate the system Although the device can exert up to 16 N during a pinch grasp it cannot execute different grasping postures and gestures In 6 the authors propose a tendon driven soft robotic glove made out of silicone which can exert up to 20 N of pinch force using an analog switch to trigger the device In 7 the authors describe a soft assistive glove that can exert more than 14 N of force during power grasps by employing hydraulic actuators The device uses EMG signals to control the closing motion of the device but the user has to select the grasp type by pressing mechanical buttons on a control box In 8 the authors propose a soft robotic glove with integrated EMG sensing for disabled people The EMG signals are used to discriminate between the actions of opening closing and holding an object In 9 the authors propose a fabric regulated soft robotic glove that uses EMG sensors combined with RFID Radio Frequency Identifi cation tags in order to control the hand motion RFID tags are attached on objects to help the glove to identify the type of grasp that must be executed while the EMG signals are used to control the motion of the device In previous works 10 we have proposed an underactuated lightweight assistive exo glove that is capable of exerting more than 16 N IEEE Robotics and Automation Letters RAL paper presented at the 2019 IEEE RSJ International Conference on Intelligent Robots and Systems IROS Macau China November 4 8 2019 Copyright 2019 IEEE Fig 2 Artifi cial tendons made out of a low friction braided fi ber connect the motor pulleys to the tendon termination structures that are stitched at the fi ngertips of the soft robotic exo glove Four polyurethane tubes offer a low friction tendon routing solution connecting the control box to the soft exo glove Soft anchor points are stitched onto the exoglove in order to implement rerouting at each fi nger of force using a single actuator and a differential mechanism Although the device can be effi ciently controlled with EMG signals it does not allow the execution of multiple grasping postures and gestures Regardingmuscle computerinterfacesandmuscle machine interfaces many studies have used EMG signals to decode reach to grasp motions 11 the object motion during the execution of dexterous in hand manipulation tasks 12 and the motion of each fi nger independently Such approaches can be used for the EMG based control of prosthetic orthotic and assistive mechanisms In 13 the authors describe an offl ine process for classifi cation of fi nger movements for hand prosthesis using EMG signals They obtained an accuracy of more than 90 for 12 classes of individual fi nger movements using 11 EMG channels In 14 the authors propose an online method for predicting individual fi nger movements for the control of a prosthetic hand using EMG signals The data was recorded using 16 EMG channels and the accuracy was 80 In 15 the authors discriminate between six different hand postures using signals from 5 EMG channels by employing a Sup port Vector Machines classifi er The classifi cation accuracy ranged between 83 99 for the different hand postures In 16 the authors proposed a forcemyography FMG based approach for decoding the fi nger motions during the execution of different grasping tasks More precisely they developed a wearable wrist band that consists of an array of 8 Force Sensitive Resistors FSRs In 17 authors com pared FSR based FMG sensors with commercially available EMG sensors They concluded that FMG sensors performed better in decoding the grasp motion accuracy of 91 2 as compared to the EMG sensors accuracy of 84 6 In 18 the authors developed two different mechanical sensors to detect the muscle movements of the forearm for four different hand postures The fi rst sensor used two FSR sensors to detect the muscle movements whereas the second sensor used a conductive force sensing fabric that was wrapped around the forearm for the same purpose In 19 the authors conducted experiments using eight FSR sensors embedded into a fl exible strap The data was processed using non kernel based extreme learning machine and the method was able to successfully detect several grasp gestures with 92 33 real time classifi cation accuracy A similar strap with eight FSR sensors was used by 20 to detect eleven different hand gestures using Linear Discriminant Analysis LDA The authors reported a classifi cation accuracy of 89 and that the number and positions of FSR sensors have a considerable effect on the accuracy of the system In 21 the authors use an array of tactile sensors to detect fi ve different grasping motions with 98 9 classifi cation accuracy In this paper we propose an assistive glove that is equipped with a muscle machine interface which combines EMG and FSR sensors to decode the user s intentions and discriminate between different grasp types see Fig 1 The device is experimentally tested and its performance is validated through two different experiments i classifi ca tion experiments to validate the capability of the proposed muscle machine interface to discriminate between fi ve differ ent grasp types and ii force exertion capability tests which focus on the maximum forces that the exoglove can apply for different types of grasps The rest of the paper is organized as follows Section II presents the designs of the device and the classifi cation methods Section III details the experimental setup used and presents the results while Section IV concludes the paper and discusses future directions II DESIGNS ANDMETHODS In this section the designs of the assistive exoglove and its components are described and the classifi cation methods used are presented A Exoglove The exoglove was designed to increase the capabilities or to restore the lost dexterity of the human hand and it is composed of three main parts a soft glove a control box and a muscle machine interface based on a sensorized sleeve The soft glove weighs 49 g The control box is composed of four motors Dynamixel XM430 W350 a microcontroller ATmega328P a Raspberry Pi Zero a U2D2 converter a USB communication converter that enables to control and operate the Dynamixel motors through the Raspberry Pi and a Li Po battery The control box can actuate four digits index middle ring and thumb as the fi fth digit pinky plays a supplementary role while grasping objects 22 The sleeve based muscle machine interface is used to decode the human intention based on EMG and FMG signals collected from the human forearm and will be discussed in detail in the following section The entire robotic exoglove see Fig 2 weighs 1150 g including the glove the sensorized sleeve and the control box less than the devices analyzed in 6 23 25 The fi nal prototype costs 1000 USD in parts to be manufactured The operation of the proposed assistive exo glove is straightforward When the user tries to execute a grasp the muscle machine interface detects and captures the activity of the muscles three different EMG processing PCBs fi l ter rectify and derive the envelopes of the EMG signals by integrating them and a microcontroller ATmega328P collects and sends the processed EMG and FMG data to a single board computer Raspberry Pi Zero Once the data have been collected by the single board computer an appropriately trained classifi er identifi es the grasp type that is being executed and triggers the required motors Artifi cial tendons made out of a low friction braided fi ber of high performance UHMWPE Ultra High Molecular Weight Polyethylene connect the motor pulleys to tendon termina tion structures that are stitched onto the fi ngertips of the soft glove Four Polyurethane tubes offer a low friction tendon routing solution connecting the control box to the soft glove The tendons run inside these tubes and inside the glove not only to be rerouted but also to guarantee that their relative motions will not hurt the skin of the user Soft anchor points are stitched onto the fi nger joints in order to reroute the tendon The positions of the anchor points are chosen so as to maximize torque as described in 10 When the motors are triggered the tendons are tensioned and the fi ngers are bent The speed of execution of the grasping task can be set according to user s preference For the experiments conducted for this work we have selected a slow closing speed for the glove to guarantee safety of operation B Sensorized Sleeve The sensorized sleeve was designed to decode the user intention based on EMG and FMG signals collected from the user s forearm The particular sensor positions correspond to the sites of the muscles that are responsible for moving the fi ngers Several factors were taken into consideration Fig 3 The muscle machine interface is composed of a sleeve made out of a stretchable fabric that can be easily worn by the user The inner surface of the sleeve accommodates three Electromyography EMG sensors and fi ve Forcemyography FMG sensors based on Force Sensitive Resistors FSR The EMG sensors are connected to three different PCBs that were designed for signal amplifi cation fi ltering rectifi cation and envelope calculation purposes while designing the sleeve like cost size weight and in tuitiveness of operation The developed wearable prototype is equipped with 3 bipolar EMG channels and 5 FMG channels All the sensors and electronics were integrated on the internal surface of the elastic sleeve The sleeve was made out of a breathable and stretchable fabric and it can be easily worn using a zipper The FMG sensors are implemented using Force Sensitive Resistors FSR and silicone based supporting pads while the EMG sensors were developed using reusable wet silver electrodes supported by thick silicon blocks to maintain a tight contact with the human skin The EMG electronics include four stages i the differential amplifi cation ii band pass fi ltering iii full wave rectifi cation and iv calculation of the envelope of the signal The inner surface of the sleeve is shown in Fig 3 The FSR sensors used in this paper were the 402 Round sensors Interlink Electronics Camarillo CA USA and have a force sensitivity range of 0 2N 20N which is enough to detect even the slightest muscle movements The reusable electrodes were manufactured by printing conductive silver ink on poly ethylene terephthalate PET sheets using an inkjet printer Previous studies describe the inkjet printing process for multiple applications including EMG 26 28 The advantage of using these electrodes over commonly used gel electrodes is that they do not need to be discarded after every use and can be developed in any shape and size to suit the requirements of the application and to improve the effi ciency of the system Reusable electrodes highly improve the practicality of the interface because the user does not have to go through the time consuming procedure of replacing the used electrodes and the sensors can be permanently attached to the interface The main drawback of these electrodes is that in order to maintain the conductivity conductive gel needs to be applied between the electrodes and the skin surface before every use Fig 4 shows the placement of the FMG and EMG sensors on the human forearm when the sleeve is worn The sensors E1 and F1 were placed on the extensor digitorum superfi cialis muscle site to capture the fi nger extensions sensor F2 was placed on extensor pollicis brevis muscle to capture the thumb extensions sensor E2 E3 F3 and F4 were placed on the fl exor digitorum superfi cialis muscle site to capture fi nger fl exsions and sensor F5 was placed on fl exor digitorum profundus muscle site to capture the fl exion of the distal joints when a fi st is made 29 30 The optimal sensor placement depends on the anatomical characteristics of each user but specifi c muscle groups are highly important across different people These muscle sites are used for a proper positioning of the sensors This has been studied in our previous work for a variety of tasks 31 and the fi ndings of this study have been used for positioning the sensors of the proposed sleeve The EMG recording requires amplifi cation and fi ltering of the signals to obtain useful information For this reason custom printed circuit boards PCB were developed to acquire and process the raw data from the EMG electrodes The collected EMG signal is fi ltered on board using a bandpass fi lter that has cut off frequencies of 20 Hz and 480 Hz 32 33 The fi ltered signals are then rectifi ed and enveloped before the classifi er is trained C Classifi cation Methods Three different classifi cation algorithms were used to dis criminate between the examined grasp types based on EMG and FMG data i a Linear Discriminant Analysis LDA classifi er ii a Support Vector Machine SVM classifi er and iii a Random Forest RF classifi er a ensemble classifi er based on decision trees The output of the Random Forest classifi er is the most popular class among the individual trees Regarding features selection the amplitudes of the EMG and FMG signals were used as input to the classifi cation algorithms The classifi ers were trained and tested using the 5 fold cross validation method III EXPERIMENTS ANDRESULTS The experiments that were conducted to assess the per formance of the assistive exo glove were divided into two parts The fi rst part focused on evaluating the ability of the muscle machine interface to discriminate between different grasp types using FMG and EMG signals collected from the user s forearm The second part focused on force exertion capability tests in order to measure the maximum forces that the exoglove can apply for different types of grasps The study has received the approval of the University of Auckland Human Participants Ethics Committee UAHPEC with the reference number 019043 Prior to the study the participating subjects provided written and informed consent to the experimental procedures Fig 4 Electrode placement positions for EMG data collection from the right human arm The blue dots represent the FSR sensors the single yellow dot represents the EMG ground electrode while the black double dots represent the bipolar EMG electrodes The letter E refers to the EMG sensors and the letter F to the FSR sensors The number followed by each letter represents the channel number E1 and F1 are placed at the extensor digitorum superfi cialis muscle site F2 is placed at the extensor pollicis brevis muscle site E2 E3 F3 and F4 are placed on the fl exor digitorum superfi cialis muscle site and F5 is placed at the fl exor digitorum profundus musle site The EMG ground electro