IROS2019国际学术会议论文集 1999

Inverse Dynamics Modeling of Robotic Manipulator with Hierarchical Recurrent Network Pengfei Sun1 Zhenzhou Shao1 Ying Qu2 Yong Guan1 Jindong Tan3 Abstract Inverse dynamics modeling is a critical problem for the computed torque control of robotic manipulator This paper presents a novel recurrent network based on the modifi ed Simple Recurrent Unit SRU with hierarchical memory SRU HM which is achieved by the nested SRU structure In this way it enables the capability to retain the long term information in the distant past compared with the conventional stacked structure The hidden state of SRU is able to provide more complete information relevant to current prediction Experimental results demonstrate that the proposed method can improve the accuracy of dynamics model greatly and outperforms the state of the art methods I INTRODUCTION Inverse dynamics modeling is one of the most essential tasks in the fi eld of robotics 1 In particular during the physical human robot interaction pHRI an accurate dy namics of robot is required for the motion control following the desired trajectory 2 Classical methods approximate the nonlinear inverse dynamics model based on the analytical derivation to establish the relation between the trajectory and torque of joint 3 However it adequately depends on the precise determination of physical characteristics as a prior such as center of mass friction etc Moreover considering the manufacturing uncertainties and dynamics change due to wear and tire the analytical model becomes inaccurate which could plausibly leads to the unsafe behavior of robots especially in the scenario alongside humans An alternative way to model the dynamics of robot is the machine learning based approach using the data from sensors directly such as position velocity acceleration and torque or current as shown in Fig 1 Compared with classical methods the prior knowledge of physical properties is not required Yeung et al convert the inverse dynamics modeling to a regression problem with Gaussian Processes GPs 4 Similarly Gaussian process regression is employed to learn the inverse dynamics online in 5 In practice the trajectory at current time is related to the historical positions velocities and accelerations for inverse dynamics modeling GPs do not consider the context yet In recent years Recurrent Neural Corresponding author 1Pengfei Sun Zhenzhou Shao and Yong Guan are with the College of Information Engineering Beijing Advanced Innovation Center for Imaging Technology and Beijing Key Laboratory of Light Industrial Robot and Safety Verifi cation Capital Normal University Beijing 100048 China 2171002023 zshao guanyong 2Ying Qu is with the Department of Electrical Engineering and Com puter Science The University of Tennessee Knoxville TN 37996 USA yqu3 vols utk edu 3Jindong Tan is with the Department of Mechanical Aerospace and Biomedical Engineering The University of Tennessee Knoxville TN 37996 USAtan utk edu Computed Torque Control Inverse Dynamics Physical Human Robot Interaction Machine Learning Sensory Data Fig 1 Machine learning based inverse dynamics modeling for robot motion control in pHRI Network RNN based approaches make signifi cant progress to learn the long term dependencies in the context using memory mechanism 6 It allows to remember the long term historical information while the short term information rel evant to current inference is extracted Several RNN variants have been applied to inverse dynamics modeling Polydoros et al propose the Principal Components Echo State Network PC ESN to solve the dynamics modeling issue 7 8 ESN maintains the short term memory due to the fading mechanism of the reservoir In 9 a Long Short Term Mem ory LSTM network based inverse dynamics model learning approach is proposed with the time complexity of O n while the time complexity of GPs is O n3 The fundamental concept of LSTM is to remember enough information over long term time since the information in the distant past may plausibly be responsible for current prediction However conventional stacked LSTMs suffer from the fading memory over time Considering the computational burden of LSTMs we adobt a new RNN variant named Simple Recurrent Unit SRU as the basic memory unit for its faster implementation with comparable performance to LSTM 10 This paper proposes a novel SRU structure with hierarchical memory to solve the inverse dynamics modeling problem of robotic manipulator It is referred to as SRU HM for abbreviation The hierarchical memory strengths the ability to process the information of long time series Different from the traditional stacked network we make the inner and outer SRU hierarchical memory by nesting two layers of SRU In this way cells in the inner layer selectively access the outer layer of memory information to improve the completeness of information dependence in a longer period The inner memory cell is returned to the outer SRU and enables the extraction of more valid information to predict the accurate 2019 IEEE RSJ International Conference on Intelligent Robots and Systems IROS Macau China November 4 8 2019 978 1 7281 4003 2 19 31 00 2019 IEEE751 torque as the output of inverse dynamics model The rest of the paper is organized as follows In Section II we describe the problem of inverse dynamics modeling for robotic manipulator briefl y We present the novel SRU structure with hierarchical memory for details in Section III Section IV demonstrates the experimental results Conclusion is drawn in Section V II PROBLEM DESCRIPTION As aforementioned this paper focuses on the inverse dynamics modeling using SRU to seek a functional mapping f between the desired trajectory of end effector and torques of joints as formulated in 1 t f qt qt qt 1 where Rd 1is the torque and q q q R3d 1denotes the tuple of position velocity and acceleration of joint respectively d is the joint number of robotic manipulator SRU should remember long time scale information so that more representative short term memory in the hidden state can be extracted to predict the torque for motion control However the stacked SRU has the same drawback of fading short term memory That is if the torque at current time is related the information in the distant past in the memory cell the accuracy of dynamics model would be reduced Thus the problem of inverse dynamics modeling based on SRU is transferred to capture enough useful long term memory Thus Eq 1 can be reformulated as t f qt qt qt c qi qi qi i 1 2 t 1 2 where c qi qi qi is the memory at time i It can save the historical information from the beginning This is an assumption that the past motion state is related to the current movement as the motion of robotic manipulator is a continuous process Mean squared error MSE is used to measure the accuracy of trained dynamics model in this paper MSE 1 dn d X j 1 n X t 1 t j j t 2 3 where trepresents the actual torque over time t tdenotes the predicted value and n is the length of input data MSE is minimized to obtain the accurate inverse dynamics model through the more effective long term information Adam and AdaGrad are employed to optimize the SRU based network III INVERSEDYNAMICSMODELING OFROBOTIC MANIPULATOR WITHSRU HM In order to retain more long term memory and obtain the accurate inverse dynamics model the structure of SRU with hierarchical memory is proposed As illustrated in Fig 2 the network consists of input layer SRU HM and output layer The desired trajectory at time t including position velocity and acceleration feeds to the input layer to the hidden layer through a feed forward connection As the Forget GateReset Gate Forget GateReset Gate 1t c t h Inner Layer Outer Layer SRU HM Input Output 1y 2y 3y dy t t q t q t q t c t c Fig 2 Structure of the network with SRU HM tanh 1 1 tanh 1 1 1 f r t f f 1t c r t r t x t c t c t x 1t c t f t r t h t h Fig 3 The internal structure of SRU HM hidden layer SRU HM is composed of 2 layers of SRU that are nested together Note that the proposed SRU HM can be nested with arbitrary layers of SRU in theory Here c denotes the information of the outer memory cell and h is the outer hidden state ctrepresents the information of the inner memory cell and htdenotes the inner hidden state The output layer is responsible for the torque prediction based on the hidden state generated by SRU HM Fig 3 shows the internal structure of SRU HM We take 2 layers of SRU in SRU HM as an example in this section The proposed SRU HM is divided into two parts including outer layer and inner layer The inner layer SRU is nested in the structure of outer layer instead of directly stacking the 2 layers of SRU The memory cell in the inner layer can freely select the long term information of the outer memory cell through the gate mechanism In this way it not only obtains more memory in SRU HM but also avoids the vanishing gradient problem A Outer Layer in SRU HM The outer layer in SRU HM is defi ned as follows Xt Wxt 4 ft Wfxt bf 5 rt Wrxt br 6 ct ft ct 1 1 ft Xt 7 752 ht rt g ct 1 r Xt 8 where xt q q q Trepresents the input to outer layer at time t f and r represent the forget gate and reset gate respectively The symbol represents the sigmoid function and g represents the tanh function denotes the element wise multiplication The outer layer only inputs the current motion state of joints of robotic manipulator which ensures that the equa tions 4 6 can be calculated in parallel thus improving the calculation speed 10 The outer memory cell stores the long term data information which is combined with the current input through the fi ltering of forget gate in the outer layer to become the input of the inner layer B Inner Layer of SRU HM The inner layer in SRU HM is formulated as follows xt ft ct 1 1 ft Xt 9 Xt Wt xt 10 ft Wfxt bf 11 rt Wrxt br 12 ct ft ct 1 1 ft Xt 13 ht rt g ct 14 where ftand rtrepresent the forget and reset gates of the inner SRU The symbol represents the sigmoid function and g represents the tanh of the inner SRU We take xtas the input to the inner layer to further select the long term memory information and the inner layer has the same gating mechanism as the outer layer After the selected information in the inner memory cell is activated by the tanh function it is multiplied by the output of reset gate to obtain the hidden state of the inner layer The memory cell of outer SRU is updated which can be obtained by the following formula ct ht 15 t Woutht bout 16 The cells from the inner layer to the outer layer obtain the fi nal hidden state through Eq 8 And the torque vector as output is calculated by Eq 16 Where Woutare the weights linking to the output layer and boutis the bias in the output layer The proposed method is summarized in Algorithm 1 Algorithm 1 Training of SRU HM Input Input data xtand ground truth tat time t length of input data n maximum iterations m Output Predicted torques t 1 for all t 1 n do 2 Calculate the outputs ftand rtthrough forget gate and reset gate in the outer layer respectively 3 Prepare the input to inner layer xt which is equivalent to ctin Eq 7 4 Similarly calculate ftand rtin the inner layer ac cording to equations 11 and 12 5 Update ct htwhere htis calculated according to equations 13 and 14 6 Calculate tusing htthrough output layer according to Eq 16 7 Calculate MSE value according to Eq 3 8 if k m then 9 Update the weights Wk 1in the whole network denotes the learning rate Wk 1 Wk MSE Wk 17 10 end if 11 end for 12 return t t 1 c t c t 1 h t h a SRU t 1 c t c t 1 h t h l t 1 c l t c l t 1 h l t h l 1 t 1 c l 1 t c l 1 t 1 h l 1 t h t 1 c t c t c t 1 h t h t 1 c b Stacked SRU t 1 c t c t c t 1 h t h t 1 c c SRU HM Fig 4 The internal calculation diagrams of SRU Stacked SRU and SRU HM are shown in the fi gures The hidden state outer memory cell and inner memory cell are repre sented by h c and c respectively 11 C Difference with the Stacked SRU Inverse dynamics modeling requires a lot of nonlinear transformation of the motion data of the manipulator to map to the torque of the corresponding joint which requires us to use the multi layer networks for modeling The differences between our proposed method and the multi layer stacked SRU are shown as Fig 4 The stacked SRU takes the output of the previous layer as the input of the later layer for secondary fi ltering That means that the input passed to the next layer is the short term memory selected by the previous layer related to the current moment information but does not transfer long term dependence on the previous layer Therefore it does not guarantee a longer term dependency 753 on information and it may leave out some deep relevant information The proposed SRU HM that the inner SRU memory cell can freely access the outer SRU memory cell This frees the inner memories to remember and process events on longer time scales even when these events are not relevant to the immediate present 11 Compared to stacked SRU SRU HM guarantees more memory information that can be selected and more accurate information at the current moment can be obtained after the second selection IV EXPERIMENTS AND RESULTS In this section three sets of experiments were conducted to verify the performance of proposed SRU HM for inverse dynamics model of the manipulator First the accuracy of proposed method is compared with the state of the art algorithms including GPs 12 ESN 13 and LSTM 14 Second as the RNN variants Gated Recurrent Unit GRU 15 and SRU 10 are applied to inverse dynamics model in the stacked manner to compare with the proposed method Third hyperparameters and temporal correlation are evalu ated in SRU HM The experimental data were obtained using KUKA 9 and Baxter 16 manipulators In particular only the data for fi ve joints of KUKA manipulator are used in the dataset including position velocity acceleration and corresponding torque information The computational confi guration used in the experiments is summarized in Table I TABLE I Confi guration used in the experiments Category Specifi cation Operating SystemWindows CPU64 Intel Core i7 4710MQ v4 2 40GHz RAM12GB Programming LanguagePython A Comparison With the State of the Art Methods The training datasets of KUKA with 1 000 and 10 000 samples are used in this experiment and mean squared errors MSEs on the testing sets are compared among GPs ESN LSTM and proposed SRU HM to measure the prediction accuracy The batch size is set to 64 and cell size is 40 As shown in Table II our trained model outperforms the state of the art methods The prediction accuracy is improved by 14 1 compared with LSTM and the error of dynamics model is largest using ESN model The result of comparison between LSTM and GPs is coincident with the statement in 9 We also investigate the torque error for each joint is illustrated in Fig 5 The torques predicted by SRU HM fi t the ground truth well and achieves the best performance SRU HM and LSTM have the comparable performance with the exception on joint 1 as shown in Fig 5f As shown Table II our training model can reach com parable performance with LSTM but the training time is decreased by 17 5 compared with LSTM Furthermore the MSE of SRU HM is greatly less than that of GPs and ESN which are also popular in inverse dynamics modeling and ESN shows the worst prediction performance TABLE II MSE comparison with existing methods MethodEpochTime s MSE GPs 0 013 1 000 samplesESN500151 1710 0178 LSTM300211 5540 01194 SRU HM300174 9140 01025 GPs 0 0499 10 000 samplesESN5001041 7140 1354 LSTM100776 9140 0138 SRU HM100592 7890 01726 TABLE III MSE comparison with other RNN variants MethodEpochTime s MSE LSTM300211 5540 01194 1 000 samplesGRU300186 5690 02040 SRU300191 2990 01652 SRU HM300174 9140 01025 LSTM100776 9140 0138 10 000 samplesGRU100686 6140 04333 SRU100723 1260 11676 SRU HM100592 780 01726 B Comparison with Other RNN Variants for Inverse Dy namics Modeling Besides we also compare the proposed method with other RNN variants for the inverse dynamics modeling including LSTM GRU and SRU 1 000 samples and 10 000 samples of KUKA datasets are used for the comparison with respect to the MSE training time and prediction accuracy Fig 6 illustrates the convergence of different methods using 1 000 samples for training The proposed SRU HM outperforms the RNN Variants especially for GRU and SRU In partic ular compared to SRU the proposed hierarchical memory mechanism enables the faster convergence and is capable of building an more accurate inverse dynamics model As shown in Fig 7 we also investigate the average torque error for each joint using different methods Most predicted torques based on the proposed SRU HM is the best compared to others For joint 1 the predicted error using SRU HM is acceptable although it is a little larger than LSTM and SRU The comparisons of training time and MSE are summa rized in Table III To compare the time and MSE during the training we set the maximum epoch to 300 and 100 for 1 000 and 10 000 samples respectively The proposed SRU HM can achieve the most accurate model in the shortest training time Especially for the SRU HM and LSTM using 10 000 samples SRU HM is faster by nearly 23 than LSTM in terms of training time We also evaluate the performance of proposed SRU HM using 1 500 samples in the dataset from Baxter manipulator The batch size is set to 64 the cell size is 21 and the maximum epoch equals to 100 in the experiment The same conclusion can be drawn that SRU HM can generate the best inverse dynamics model effi ciently C Evaluation of Hyperparameters and temporal correlation in SRU HM The signifi cant hyperparameters that effect the perfor mance of proposed method SRU HM are batch size cell 754 SRU HM a Joint 1 SRU HM b Joint 2 SRU HM c Joint 3 SRU HM d Joint 4 SRU HM e Joint 5 0 0 005 0 01 0 015 0 02 0 025 0 03 Joint 1Joint 2Joint 3Joint 4Joint 5 MSE ESN LSTM SRU HM f Average MSE of every Joint Fig 5 Comparisons of predicted torques of 5 joints among ESN LSTM and our proposed SRU HM The x