IROS2019国际学术会议论文集 0587

Smart coaching platforms are emerging which combine Body Sensor Networks with AI based training software to monitor and analyze body motions of athletes workers or medical pa tients This allows for new opportunities to explore algorithms to interpret body sensor data and provide analytical feedback for learning a physical task refining body motions or to protect from work related injuries This paper presents a solution to non invasively equip a person with sensors of a Smart Training System STS to improve training efficiency during sport activi ties Our system calculates the significance of each body part during physical activities and provides targeted feedback on which body locations are under performing In experiments the system collected data from 13 inertial sensors attached to the en tire body of inexperienced golf learners Using an indoor golf training net with a central target with 3 concentric zones 1 080 real world golf swings of 11 participants were analyzed During the first 30 swings of each participant the system learned distri butions of motions from each sensor conditioned on swing per formance reported by users from their hitting location on the target In the later 70 swings feedback was provided to a sub group of 8 participants by computing for an optimal set of fea tures determined during training the largest discrepancy The remaining 3 control participants received no feedback From only 100 golf swings for each participant our system led to sig nificantly improved scores by on average 3 7x t test p Body part plays an important role in dis tinguishing between target hits and misses because its varia bility from optimal was higher for missed shots than for hits Feedback provided for body part will likely affect the perfor mance by reducing the variability of body part from optimal The 13 values of the rotational speed ratio dimension of are presented for all 3 swing phases in Figure 13 Figure 13 The higher the value the more important the body part for swings that hit the target The displayed val ues represent Vcritical for the rotational speed feature Feedback Creator and Trigger Feedback is created by combining a time element a motion type element a body part and a motion direction A total of 208 feedback combinations are possible Fig 14 Figure 14 The pool of generic instructions shows all pos sible feedback combinations The information about critical body parts acquired in the pre vious step plays an important role in selecting effective feed back First the distance of the current swing features and the optimal features are calculated for each sensor and stored in a matrix for each swing i The maximum feedback score is calculated by adding up the ma trices of the last 3 swings with n as the latest swing This rep resents the body part that had the highest deviation from the optimal feature set over the last 3 swings If indeed the critical body parts classifier identified this body part as important the feedback will be passed on to the feedback trigger module Eq 5 If not the second highest that was classified as important is selected The element location of in the matrix of the last 3 swings defines time motion type and body part when the feedback is generated Motion direction is determined by the sign of subtracting the optimal motion fea tures from the latest swing features max The feedback trigger only provides feedback if the latest swing did not perfectly hit the highest target the red circle in our ex periments The Feedback is provided as an audio output All 26 elements in the pool of generic instructions have been voice recorded The elements are appended to an audio stream according to the outcome of the C Experimental Protocol Test Setup To record real world performance data with and without sys tem feedback 8 human golf novices were equipped with 13 IMU Sensor Modules For reference 3 additional subjects performed golf swings without the sensor equipment attached to their bodies The sensors were glued into 3D printed hold ers epoxy glued on anti slipping sport bands and strapped around 13 body parts Placement of the sensor did not need to be accurate or reproducible thanks to our algorithmic ap proach described above However it was important to limit slippage during a particular session hence the use of sports bands Further no calibration was necessary Participants could start practicing as soon as the sensors and computers were on None of the subjects had previous golfing experience at the time of the recording All subjects were right handed In our experiments we focus on learning and refining swing body motions for short distance golf swings A golf practice net was setup with three target circles The hitting location in these circles represent the level of accuracy of the golf swing The red circle 3 points the white inner circle 2 points and the white outer circle 1 point Fig 7 If the ball did not hit any circle no points were given The subject was standing ap proximately 4 meters in distance to the target Dataset and T Tests Each subject performed 80 100 golf swings 8 of 11 subjects received feedback after the 30th golf swing The audio feed back was provided via a loudspeaker next to the subject To statistically evaluate the recorded datasets we performed two sample t tests on the performance data Fig 18 We 4032 aimed to test two hypotheses a Did the system increase score overall for our population of participants b Did the system increase score for every participant who used it In addition to these two baseline tests and because some im provement of score may occur just by virtue of practicing even without our coaching system we compared the slopes of cumulative score plots an indication of expected score on each shot between the first 30 and the later swings This al lowed us to evaluate whether participants experienced a higher score expectation after our system was turned on at the 30th swing The overall 2 sample t test Test Description The overall 2 sample t test was performed on two groups The first group includes subject 1 to 8 re ceived feedback after swing 30 while the second group in cludes subjects 9 to 11 received no feedback at all For each group the performance results were concatenated in two sam ples Sample A contained all scores of the first 30 swings Sample B contained all scores after the first 30 swings The two sample t test was performed on Sample A and B Our null hypothesis claims that there is no statistically significant dif ference between Sample A and B The alternative hypothesis suggests the opposite The individual 2 sample t test Test Description The individual 2 sample t test splits the per formance results for each subject in two samples Sample A contains the first 30 swings All following swings are as signed to Sample B None of the subjects received feedback on the first 30 swings However subjects 1 to 8 received feed back during the following swings while subjects 9 to 11 con tinued swings without feedback IV TEST RESULTS The overall 2 sample t test Test Results The results show a very low p Value of Group 1 Sub jects 1 to 8 Fig 15 This demonstrates that there is a statistically significant difference in score between swings that received no feed back Sample A and swings that received feedback Sample B Ultimately the null hypothesis was rejected and the alter native hypothesis was suggested In comparison Group 2 Subjects 9 to 11 showed a high p Value which aligns with the expectation that there is no statistically significant differ ence in score within the Samples A and B This aligns with the experiments in which neither Sample A nor B received feedback In summary the overall t test confirms that swing perfor mance was improved by feedback from STS Even though the control participants of Group 2 improved slightly after the 30th swing that improvement was too small to be statistically significant in contrast the participants receiving feedback from STS in Group 1 improved much more dramatically and significantly Fig 16 The individual 2 sample t test Test Results For 7 of 8 subjects that received feedback the null hypothesis was rejected Fig 17 For 3 of 3 subjects that received no feedback the null hypothesis was suggested The results show that the feedback yielded a statistically signifi cant performance difference in 87 5 of cases In summary the individual t test shows that not every individual subject improves performance from receiving feedback during swings Hence it is expected that although useful for most people this system may not work for all people Note that Bonferroni correction is not warranted here since we perform 11 unrelated tests Possible reasons for the performance out lier participant 4 are that feedback was not understood properly by the subject Another explanation could be that the subject was not able to put the feedback into practice As a result we claim that providing feedback during golf swings might not guarantee for every individual to experience a change in their swing performance Learning analysis Every subject with or without receiving feedback during their experiments showed a performance increase though it was small and not significant for the control Group 2 and subject 4 The score curve for each subject however varies as shown by fitting a line to the cumulative score results for the first 30 swings another for the later swings and comparing their slopes Fig 18 We made multiple observations from the Fig 15 The overall t test results confirm a swing performance change when receiving feedback Figure 16 Average score per swing shows a significant im provement in the second period for subjects who received feedback vs subjects who did not receive feedback On av erage score of participants who received feedback im proved 3 7x Figure 17 The individual t test shows that there can be a p Value outlier Not all subjects who receive feedback experience a swing performance change 4033 collected data 1 The cumulative score curve among subjects receiving feedback is significantly steeper than the subjects not receiving feedback Fig 18 2 During the first 30 golf swings subjects 1 to 8 performed on average lower than the subjects 9 to 11 Subjects 1 to 8 had to get used to performing golf swings with sensors and cables attached to their body which may have lowered their baseline performance Subjects 9 to 11 did not wear additional equipment 3 If the average point gain is too small for the first 30 swings the task might be too difficult Fig 16 Subject 5 4 Training improvements sometimes come with slight delay Subject 2 6 8 did not show instant improvement results when feedback was provided starting at swing 31 V DISCUSSION In this work we demonstrated that body motions for a specific task can be successfully refined by providing body part spe cific feedback The feedback in this work is based on analyt ical data measured on all moving body parts Although the feedback represents the highest variation from an optimal set of body motion features it is not evaluated on how it is re ceived by a person In addition the experiments were per formed with sensors and cables attached to a person For an optimized test setup an unobtrusive body suit with integrated sensor hardware could minimize possible minor bias effects on the baseline performance The feedback system has also limitations STS requires that a task is not too difficult no target hits will not enable the calculation of optimal features or too simple to achieve distance to optimal features is not significant enough to provide meaningful feedback VI CONCLUSION AND FUTURE WORK To evaluate the efficiency of feedback e g do we need to provide this feedback for every other swing only does certain feedback show no improvement etc we need to create a feedback evaluation module This could be achieved by using a shallow recurrent neural network This network quickly learns from the previous scored performances and provided feedback and decides if a future feedback indeed will lead to an improvement Fig 19 Since this feedback module is only dependent on performance score and feedback given and does not rely on the same exact sensor placement locations it could be used throughout multiple training sessions By providing a valuable approach on improving the swing performance of golf players using a full body sensor network we hope that STS made a contribution to the development of Smart Teach ing Systems with wearable technology ACKNOWLEDGMENT This work was supported by the National Science Foundation grant number CCF 1317433 C BRIC one of six centers in JUMP a Semiconductor Research Corporation SRC pro gram sponsored by DARPA and the Intel Corporation The authors affirm that the views expressed herein are solely their own and do not represent the views of the United States gov ernment or any agency thereof REFERENCES 1 CCS Insight Market Forecast Wearables Worldwide 2018 2022 2 H Uematsu et al Halux Projection based Interactive Skin for Dig ital Sports ACM SIGGRAPH Emerging Tech Article No 10 2016 3 Y Konishi Synesthesia suit the full body immersive experience SIGGRAPH 16 ACM SIGGRAPH VR Village Article No 20 2016 4 Teslasuit https teslasuit io Accessed