IROS2019国际学术会议论文集0238

AbstractTo maintain a high level of motivation is a vital issue during rehabilitation because rehabilitation involves much pain, depression, and discomfort. To study the mental state of patients is necessary. A new assistive approach using emotion evaluation combined with an assistive walking device to maintain motivation during the rehabilitation is proposed. To realize it, the walking-emotion relationship within personal walking limitation (PWL) was explored at first. The results showed that when people walked by following the personal preferred walking ratio (PPWR), they were evoked the arousal emotion and kept the current valence emotion simultaneously; however, when people were not following PPWR to walk, the negative emotion was elicited. After that, subject was required to wear the assistive walking device to conduct the test, which aims to observe the emotion variation when subject walked at specific walking gait with an assistive device. We then are according to current emotion state to tune device. The findings show if people mistakenly use the device, unpleasant feeling would be elicited. When we properly tuning assistive device conforms to current gait, it can validly alleviate the negative emotion further becoming the positive emotion. From these results, we conclude that emotion can be affected by walking. By using walking-emotion evaluation to an assistive device, we can be according to users emotion to control device; also, we can use an assistive device to inspire users emotion to further maintain motivation. Finally, this assistive approach provides a useful treatment way for serving rehabilitation. I. INTRODUCTION Studying on the human-robotic interaction (HRI) has been received attention in recent years. Robots have been extended to assist people further improved their life. It is deserved to study a topic which is to connect the assistive device and human then as an HRI application. From the viewpoint of users, they especially expected the assistive device could comply with their mind to achieve their requirement. At the beginning period while using the assistive device, users are unfamiliar with it, they must to learn and adapt it. Thus, uers are very easy to give up or decrease their willingness to do rehabilitation while using the assistive device at the beginning period. These problems may cause users failures to regain body health, and then induce bedridden. Thus, from these reasons, we have to understand to know the mental condition of users and to keep their motivation during rehabilitation. Emotion is an internal and subjective experience. Emotional changes can significantly influence on other people, and also 1Jyun Rong Zhuang*, Guan Yu Wu, Hee Hyol Lee and Eiichiro Tanaka are with the Graduate School of Information, Production and Systems, Waseda University, 2-7 Hibikino, Wakamatsu-ku, Kita-Kyushu, Fukuoka 808-0135, Japan (*corresponding author, e-mail: gary_zhuangakane.waseda.jp). represent current mental state. The study of the application of emotional recognition has been proposed in recent years. Kolakowska et al. proposed the emotion applications for software engineering 1; Kim et al. used a stimuli control system to adjust group emotion such as kindergarten 2. Very limited studies mentioned the emotion-control applications for human rehabilitation. Thereby, it is necessary to develop a hybrid assistance system which integrates mental and physical aspect for improving the users experience. To recognize the emotion, physiological change is the most authentic and objective because the autonomic nervous system directly dominates physiological change, people cannot freely control it. As emotional recognition factors, using physiological change is thus better than other ways (such as facial expressions and speech). In former papers, we based on multiple physiological signals to design a real-time emotional recognition system 3. This system employed the deep neural network (DNN) as classifier to judge the current emotion on two-dimension (2D) emotion map 4. In physical assistance, many lower-limb assistive devices have been proposed to aid people walking. HAL 5 and SMA 6 had been proposed to restore impaired gait of the elderly or patients. Collins et al. 7 and Quinlivan et al. 8 have developed the assistive device to reduce the energy cost in walking. However, tibialis anterior (TA) muscle easily get fatigue during movement. If TA muscle loses its function, it induces people to become the equinus foot; whereby, it may cause an unexpected accident (e.g. stumbling). Thus, it is vital to assist the TA muscle. We developed an assistive walking device (named RE-Gait) to assist the ankle joint of users 9. RE-Gait has been released as a product 10 for gait training of neuro-rehabilitation of the apoplexy patients. RE-Gait merely assists the ankle joint, and then it can trigger a stretch reflex of human, the user can raise leg up while walking. Thus, RE-Gait is suitable for patients to correct the impaired gait caused by stroke. However, this is only physical assistance. According to the emotion of the user, we have to tune the device. Additionally, to prevent getting the stroke and the degradation, elderly must continue to exercise and maintain motivation, this is very important. Hence, we developed a new assistive device 11 for elderly to promote exercise; it is smaller, lighter and cheaper than RE-Gait. Elderly can thus perform ideal gait and stimulate the neural circuits of brain at the same time while using an assistive device. Besides, we suggested recognizing the emotion of user while using an assistive device. To realize the application between people and an assistive device, as the trial 12, we had attempted to be according to the heart rate variability of subject to change the mode of the assistive walking device. However, we only can affect one-dimension emotion. To further understand walking- emotion relationship, we prepared the trial in this paper. Applying the Interaction of Walking-Emotion to an Assistive Device for Rehabilitation and Exercise Jyun Rong Zhuang1, Guan Yu Wu1, Hee Hyol Lee1, Member, IEEE and Eiichiro Tanaka1, Member, IEEE 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Macau, China, November 4-8, 2019 978-1-7281-4003-2/19/$31.00 2019 IEEE6489 This paper aimed to propose an assistance approach that employs the emotion evaluation combining to an assistive walking device for improving the rehabilitation experience. II. ASSISTIVE WALKING DEVICE To prevent the bedridden issue of elderly, they must consider promoting exercise and maintaining motivation in daily life. We designed a small, lightweight and low-cost assistive walking device for elderly, as shown in Fig. 1. This device can promote exercise by guiding users to perform the correct walking motion timing. In the hardware, we designed only a single frame attached on the outside of the leg; thus, this device has highly flexible for adapting any environment in walking. The pressure sensors were attached to the sole of the device, and thus we could determine the walking phase of the human. In addition, we incorporated our developed torque limiter into this device. Torque limiter can eliminate the overload torque to prevent the device from breaking. We configured a rubber between output shaft and gear. When the surface force exceeded the maximum friction force, the output shaft slipped and protected the motor 13, as shown in Fig. 2. Torque limiter also offered the flexibility to reduce the internal stress generation on human caused by the position controllers high stiffness. In addition, users can wear shoes to equip this device. In the software, we used the Arduino with a compact servo motor as the control system. We programmed the walk mode and all-round mode in this device for getting different assistance. Walk mode used angular velocity control method that guided motion smoothly for users to correct their gait. All-round mode (angle control method) was used for all the motions that involved walking, getting up, sitting, and standing 11. Fig. 3 shows the assistance method of this device. First, motor triggers the dorsiflexion (black arrow), and then GAS muscle is contracted by stretch reflex, knee joint is thus flexion; then, due to flexion of knee, RF muscle is contracted by stretch reflex, hip joint is thus flexion (foot can be raised). User can reduce the muscle burden because they do not need to attach more actuators on other joints. Using this device can control ankle angle for keeping walking gait 14. The device was controlled through predefined targeted angle (c) and targeted walking cycle (Tc) as shown in Fig. 4. We employed the people walking phase as the trigger; thus, according to the state of pressure sensors, the device would be actuated to conduct the required targeted motion. This section is only physical assistance. To maintain motivation, we should understand the emotion of users. Figure 1. RE-Gait (left graph 10) and new assistive device (right graph). New device contains controller, servo motor with torque limiter, Arduino, Li-Po battery and pressure sensors. Users can wear the shoes to equip it. Figure 2. An overload protection mechanism using a torque limiter including the output shaft, gear, and rubber 13. Figure 3. Assistance via stretch reflex 11. By only assisting the ankle joint, the hip and knee joints can be assisted by bi-articular muscle linkage motion. Figure 4. Predefined targeted angle and targeted walking cycle of walking mode for controlling the assistive walking device 11 1 Tcc1c ,0,1,.,20=()/() (/20) ii ii titT = (1) Walk mode (angular velocity control) was used in this study. We set up the targeted angle (c0c20) that was corresponding to the targeted walking phases (t0 t20) into system; whereby, the targeted angular velocity ( T deg/sec) can be obtained via Eq. (1). We can tune the targeted walking cycle (Tc) and targeted angle amplitude (c) to meet various walking gait. III. REAL-TIME EMOTIONAL RECOGNITION SYSTEM We developed a real-time emotional recognition system to identify the current emotion (accuracy: 80%). This system can recognize nine emotions (colored area) on 2D emotion map, as shown in Fig. 5 (a) 3. In that study, we had invited 20 subjects to conduct emotion elicitation test. They had been required to watch 30 specific films while wearing two physiological detectors (brainwave and heartbeat). During experiment, they had also expressed their subjective emotion via the visualized questionnaire 15 as shown in Fig. 5 (b). We could identify the emotion of subjects by combining these two factors. By collecting all recorded physiological data and questionnaire answers, we had built the training set to create the DNN models being an emotional recognition system 3. 6490 (a) The developed real-time emotion recognition system 3 (b) Visualized questionnaire Figure 5. (a) The developed real-time emotion recognition system 3. (b) Visualized questionnaire 15 (we defined that the scores were divided into 3 areas on valence and arousal, respectively 3). Figure 6. Experiment scenario. Subjects wore physiological signals detectors then walking on the treadmill via complying with the experimental sequence. Figure 7. 2D emotion map 4 and the 2D walking condition map 16 are used to explore the walking-emotion relationship. Figure 8. Decide personal walking limitation (PWL) by three steps. IV. EXPLORE WALKING-EMOTION RELATIONSHIP A. Walking-Emotion Relationship experiment We designed the experiment to explore the walking- emotion relationship. Three young man had joined in this experiment (height: 176.75.5 cm and weight: 70.77.1 kg). They wore two physiological detectors (Emotiv and myBeat) then walking on the treadmill by following the experimental sequence: 1. Walking period: they walked according to the specified walking gait (30 seconds). 2. Questionnaire survey period: they filled out the visualized questionnaire (20 seconds) to report their feeling. 3. Rest period (30 seconds). Experiment scenario is shown in Fig. 6. We placed the markers on the ruler and applied the camera to record the gait of subjects. The metronome was also placed in this experiment. The step length m/ step and cadence step/ min could be controlled at the same time by watching the lower limbs motion from live video and listening to the rhythm from metronome. As human emotion is susceptible, it is not suitable to walk for a long time during experiments. We assumed that the 30 secs walking stimulation is enough for eliciting the walking-emotion relationship. In this paper, the 2D emotion map 4 and 2D walking condition map 16 were employed to study walking -emotion relationship. 2D emotion map is plotted by valence emotion and arousal emotion which respectively represents that emotion belongs to pleasure or unpleasure, and excited or unexcited. 2D walking condition map is drawn by cadence step/min and step length m/step of people. Former research 16 described that the step length divided by the cadence is walking ratio. They claimed the preferred walking ratio is about 0.006 which was tested by twenty-two health people. Both maps are shown in Fig. 7. B. Personal walking limitation as specified walking gait We thought that each person possesses PWL, and PPWR. Thence, the specified walking gait was different in different subjects. We proposed an experimental way to decide PWL; then the specified walking gait was defined by it, as shown in Fig. 8. PPWR, PWL and specified walking gait were determined as follows: 1. Walk comfortably in three speeds to determine PPWR. 2. PWL area can be decided by the position of fast and slow walking on the map. 3. Determine a vertical line of PPWR, and further determine the long step length (S.)-low cadence (C.) (hollow rhombus) and low S.-high C. (hollow cross), separately. The shifted points (solid cross) were decided by equality of ratios respectively from the normal walking point to each boundary points walking limitation area. Thus, the personal specified walking gait could be decided. The shifted points were used to ensure the subject can walk more comfortably to the specified walking gait. We had done this process for each subject before conducting the walking -emotion relationship experiment. Walking within the PWL can reduce other factors that affect emotion. V. WALKING-EMOTION INTERACTION INTO ASSISTIVE DEVICE A. Walking-Emotion Test while using an assistive device To realize the interaction between emotion and assistive device, we designed the test which aims to observe the 6491 emotion variation when subject walks on specific walking gait while wearing an assistive device. We assumed that users emotion would be affected by using an assistive device. If the assumption is true, we can control the users emotion further becoming positive emotion then keeping motivation during the rehabilitation. This test had conducted by the indicated protocol as shown in Fig. 9. Subject had worn the assistive walking device to walk four tests which were from normal walking to four specific points, respectively (normal to fast/ slow/ long S. with low C./ low S. with high C.). The steps of protocol: 1. Subject walked for 30 secs at normal speed (normal walking mode), then used questionnaire and recognition system to examine the current emotion. 2. Require subject to change gait to walk for 30 secs. In this period, we did not change the mode of device (here still used normal walk mode), then examine the emotion. 3. Maintain the same walking gait; additionally, we change the mode of device to let subject fit the walking gait, then examine the emotion. Through changing targeted waking cycle (Td)and targeted ankle amplitude (d), the device could be changed the walking mode. The scenario of walking-emotion test is shown in Fig. 10. A young man participated in this test. He wore two detectors and an assistive device to walk on the treadmill. B. Emotion-walking control strategy The control strategy of the emotion-walking system as shown in Fig. 11. When the subjects emotion is identified by recognition system, the emotion-walking coefficient and are used as tuning parameters to further calculate the new targeted walking cycle (Td) and new targeted angle amplitude (d) which are exhibited in Eq. 2 and 3. Td = Tc (2) d = c (3) where Tc and c represent the initial targeted walking cycle and initial targeted angle amplitude, respectively. The adjusted angular velocity ( T ) can thus be recalculated by Eq. 1. then to control the motor. Tuning walking cycle can affect cadence (step frequency) because walking cycle is the reciprocal of cadence. Besides, the angle variation of the ankle and step length exist the linear relationship 17. Tuning angle amplitude of ankle can thus affect the step length. VI. RESULTS AND DISCUSSION A. Evaluation of walking-emotion relationship Walking-emotion relationship results are shown in Fig. 12. The upper graph indicates the walking condition of three subjects. The questionnaire results and recognition results are used to answer their feeling as shown in the lower graph. Moreover, we organized the emotion variation corresponding to walking gait as shown in Table I. The results showed that when walking gait tended to accelerate (orange arrow), all subjects were evoked the positive arousal emotion. According to the average tendency, we evaluated that people felt strongly excited when they walked from normal to fast. Furthermore, when walking gait tended to deaccelerate (grey arrow), the negative arousal emotion (unexcited) with negative valence emotion (slight unpleasure) was elicited. Moreover, the findings showed that when walking gait changed fro