IROS2019国际学术会议论文集 2606

40 IEEE ROBOTICS predict their behaviors and respond with a socially appropriate emotion 1 While deficits in empathy can often make children with ASD appear uncaring these deficits arise from a lack of understanding regarding what others may be feeling rather than an inability to resonate with others emotions 2 A model for the emula tion of empathy in SARs and methods for its measurement were suggested in 3 A study investigating the relation between empathy and ER describes ER as one of the macrocomponents of human empathy 4 ER is the automatic or intentional modification of one s emotional state to promote adaptive and goal directed behaviors that involve monitoring eval uating and modifying the intensive and temporal features of emotional reactions to accomplish one s goals 5 This construct potentially offers an explanation for the emo tional and behavioral problems commonly observed in children diagnosed with ASD Impairment in ER is likely to result in impulsive reactions to emotional stimuli that manifest as tantrums aggression or even self injury And while these behaviors may appear to be deliberate or ISTOCKPHOTO COM PIKEPICTURE Interactions With an Empathetic Agent By Hifza Javed and Chung Hyuk Park Regulating Emotions and Improving Engagement in Autism Digital Object Identifier 10 1109 MRA 2019 2904638 Date of publication 9 April 2019 IEEE Robotics and Automation Magazine RAM paper presented at IROS 2019 It should be cited as a RAM paper 41JUNE 2019 IEEE ROBOTICS each repre sented a variation in inten sity level for the six basic emotions anger disgust fear happiness sadness and surprise For exam ple content happy and excited represent ed progressively increas ing intensities of the happiness emotion A suitable emotion color mapping was determined using the emotions color wheel 16 To design the facial expression of the penguin character we considered several samples from Greive 17 Based on these sample emotion cards and inputs from several par ents of children with ASD the expression design of the penguin character consists of four distinct layers text b a Figure 1 a The original Romo and b our custom designed Romo In our work we developed a framework that implements empathy based ER while engaging the user in turn based emotional interactions IEEE Robotics and Automation Magazine RAM paper presented at IROS 2019 It should be cited as a RAM paper 42 IEEE ROBOTICS then it gradually moves this temporary goal state closer to Figure 3 Emotion relevant robot motion design a Romo s turn angle control scheme b a mapping of motion primitives on the spatial domain with emotional variations and c a mapping of emotion s on Russell s circumplex model Reg regular Angle Radius Speed Angle Radius Speed Angle Radius Speed Angle Radius Speed Speed Motion Range Run Search Slow Explorer Slow Wander Reg Tempo Walk Move Casual Stroll Irritated Angry Dance Carry Out Task Excited Happy Curious InterestedSad Depressed Not Interested Arousal AngerPleasure RelaxedSad Afraid Alarmed Tense Angry Frustrated Annoyed Distressed Dizzy Miserable Sad Gloomy Bored Droopy Depressed Tired Sleepy Relaxed Calm At EaseSerene Content Satisfied Valence Pleased Glad Delighted Happy Excited Aroused Curious Astonished b c a IEEE Robotics and Automation Magazine RAM paper presented at IROS 2019 It should be cited as a RAM paper 44 IEEE ROBOTICS F female HF high functioning LF low functioning Figure 4 The GUI to execute Romo actions Figure 5 The emotion game interface IEEE Robotics and Automation Magazine RAM paper presented at IROS 2019 It should be cited as a RAM paper 45JUNE 2019 IEEE ROBOTICS AUTOMATION MAGAZINE interaction because emotions could be recognized not only from the images and animations but also from the titles given to each emotion However in an effort to ensure that participants of all ages were able to recognize the emotions correctly every participant was familiarized with all of the emotions before the start of the game with an instructor clicking each emotion and announcing the respective title All participants were also asked to say out loud which emotion they felt at each turn and they were assisted in selecting the said emotion from the panel if required Methods The children were accompanied by their parents who were able to observe the children throughout each session The pur pose and procedure of the study were explained and parental consent was obtained before the start of the session Each participant took part in exactly one individual session The research protocol for the study was approved by George Washington Uni versity s Office of Human Research and all requirements established by the Institutional Review Board protocol 111540 were followed The emotional interaction activity for ER was conduct ed as a computer application run on a laptop placed on a table with the par ticipant seated in front of it Fig ure 6 a The participant was given direction about the procedure of the game but no instruction about how long the interaction would continue was given A sample from an interac tion between a participant and the pen guin is shown on the circumplex in Figure 7 which provides a visual depic tion of how the character closely follows the participant s emotion states while performing the regulation function Another study was conducted following the emotional interaction activity during the same session Here two dif ferent robots were used Mini and Romo the embodied robot with the updated penguin character were utilized within a sensory station setup where they modeled socially appropriate responses to sensory stimuli 20 Figure 6 b depicts scenes from these sessions Here it is sufficient to mention that the video data collected from this study were annotated to mark target behaviors displayed by the partici pant comprising 1 eye gaze focus 2 imitation 3 self ini tiated interactions 4 smiles 5 vocalizations and 6 triadic interactions The duration and frequency of occurrence of these behaviors during the child robot interaction were used to compute a participant s overall level of engagement in the activity as a percentage of time spent displaying the target behaviors This was termed the engagement index Figure 6 Scenes from experimental sessions for two different activities a participants playing the emotional interaction game and b the sensory station activity with Romo and Mini a b Figure 7 The emotion trajectories followed by both agents during an interaction where the X marks the starting point of each agent s interaction Arousal Anger Pleasure RelaxedSad Afraid Alarmed Tense Angry Frustrated Annoyed Distressed Dizzy Miserable Sad Gloomy Bored Droopy Depressed Tired Sleepy Relaxed Calm At Ease Serene Content Satisfied Valence Pleased Glad Delighted Happy Excited Aroused Curious Astonished Wanting Celebrating Disgust Unhappy Participant Character IEEE Robotics and Automation Magazine RAM paper presented at IROS 2019 It should be cited as a RAM paper 46 IEEE ROBOTICS AUTOMATION MAGAZINE JUNE 2019 It is important to note that the emotional interaction game preceded this activity in every session Because the out come measure of this study was the engagement index for each participant we also investigated any identifiable trends in the children s performances of both activities The results are discussed in the following section Experimental Results To evaluate the effectiveness of this framework in conducting emotional interactions with both the TD and ASD children two metrics were obtained from the participant s sessions the length of the interaction and the success of the ER capability The length of the interaction was calculated as the number of turns taken by the participant in the interaction sequence The success of the ER capability was then calculated by the emotional difference between the participant s and the char acter s last few emotional states dependent measure as determined by the mapping of emotions on the circumplex A distance less than 10 of the maximal range in the emotional domain was considered a successful execution of ER The final 30 of the emotional exchanges over the entire length of each participant s interaction were utilized in the measure ment of this distance Table 2 summarizes the results from the emotional interaction activity for both groups listing the lengths of interaction emotional distances and success of ER for all participants For both these metrics a one way analysis of variance ANOVA was applied to the data collected from both groups Because the TD group was almost twice the size of the ASD group we used the data from only seven of the TD participants IDs 1 3 5 6 7 10 and 11 These participants were selected such that they Table 2 Results from the emotional interaction activity ID Length of Interaction Number of Turns Emotional Difference Success of ERGroup 1474 06YesTD 2254 26YesTD 3312 74YesTD 4154 33YesTD 5447 57NoTD 6582 89YesTD 7182 78NoTD 8118NoTD 971 58YesTD 10122 21YesTD 11320 89YesTD 12283 8YesASD 13154 36YesASD 14123 84NoASD 15710 03NoASD 16224 32YesASD 17115 53NoASD Figure 8 Distribution of the lengths of interactions for both groups 60 50 40 30 20 10 Length Number of Turns ASDTD Group Lengths of Emotion Interactions per Group 30 25 20 15 10 70 60 50 40 30 20 10 0 5 0 Length of Interaction Emotion Game Length of Interaction Emotion Game 0 16283 0 18117 0 20217 0 21767 0 27033 0 28517 0 36717 0 415 0 46667 0 467 0 561 b a 0 235830 328330 516170 37450 34567 Interaction Pattern Sensory Station Activity Versus Emotion Game ASD Group Interaction Pattern Sensory Station Activity Versus Emotion Game TD Group Figure 9 The correlation among participant performances in emotional interaction and sensory station activities for both groups a The ASD group and b the TD group IEEE Robotics and Automation Magazine RAM paper presented at IROS 2019 It should be cited as a RAM paper 47JUNE 2019 IEEE ROBOTICS AUTOMATION MAGAZINE were the closest matches in age and gender to the ASD participants The mean age of this TD group was now 7 43 years with a standard deviation of 2 29 years with all male participants Length of Interaction In general in the emotional interaction activity the TD group was seen to take a larger number of turns than the ASD group as shown in Figure 8 The average length of interaction for the TD group was 34 57 turns with a stan dard deviation of 16 30 turns while that for the ASD group was 14 6 turns with a standard deviation of 8 08 turns The one way ANOVA for these data generated the following results F 1 116 57 h p0 0263 Because p0 051 we concluded that there is a statistically significant difference among the lengths of interaction for the two groups with the TD group taking part in longer interactions than the ASD group Success of ER In terms of the ER function our framework appeared to perform with similar effectiveness for both the TD and ASD groups ER was successful for eight out of 11 partici pants in the TD group However out of the seven partici pants whose data were considered for analysis successful ER was achieved for four out of seven TD participants and three out of six ASD participants The one way ANOVA generated the following results F 1 110 56 h p0 471 Because p0 052 we concluded that there is no statistical ly significant difference between the effectiveness of the ER function for the two groups It can therefore be deduced that the framework can effectively regulate the emotions of both groups Effect on Engagement Subsequent to engaging in the emotional interaction activi ty with the penguin character participants of this study took part in another activity conducted with the two robots Romo and Mini which was aimed at improving users sen sory processing skills By comparing the two measures described previously with the findings from this study we were able to identify a correlation among the performances of the two groups in the emotional interaction activity and the sensory station activity Out of the 17 children who participated in the emotional interaction game one ASD participant was unable to take part in the sensory station activity The mean age of this group was now 8 2 years with a standard deviation of 14 6 years We plotted the lengths of interaction in the emotion game with the penguin character against the engagement index of the participants in the sensory station activity Figure 9 to identify any correlation For the ASD group Figure 9 a depicts a general increase in the engagement index when the participant previously experienced longer interactions with the penguin character in the emotional interaction game Though not as clear a similar trend is observed for the TD group despite multiple outliers Figure 9 b Of course for more accurate statistical analysis we will need to conduct a larger scale study to verify this finding Discussion We developed an empathetic emotionally expressive robot based tool to engage children with ASD in emo tional interactions while simultaneously performing the ER function The novel consensus based frame work was tested with an emotionally interactive agent that is the pen guin character whose emotional responses fol low the user s emotion states closely empathy before gradually guiding them to the predeter mined goal state ER Such empathetic behav iors demonstrate the capability of the frame work to effectively model interaction patterns for diverse personality types Results from the user study confirmed the feasibility of this framework as an ER tool that offers similar benefits to both groups of children Although children with ASD engaged in relatively shorter emotional interactions than did TD children we hope that continued interaction with these tools in future sessions will lead to improvements in this regard The age ranges for the participants whose data were analyzed in this study were five to 10 years for the TD group and seven to 10 years for the ASD group Therefore while the cognitive abilities of the two groups can be considered com parable the within group differences in emotional and lan guage reading abilities can perhaps be better addressed in a future study For children in both groups longer interactions in the emotion game were seen to result in higher engagement levels in the subsequent sensory station activity This is noteworthy because it points to the potential of this framework as a tool to prime higher levels of focus and engagement in various tasks for both groups In testing the application with participants some obvious limita tions were observed We noticed that some of the younger or lower functioning children needed additional assis tance in understanding the activity procedure its pur pose and their role in it In the future we plan several upgrades to this application focusing particularly on making the activity design and interface more intuitive and engaging We also intend to continue recruitment for this study to close the age gap within and between the two groups and to collect larger sets of data to improve the reliability of our findings Results from the user study confirmed the feasibility of this framework as an ER tool that offers similar benefits to both groups of children IEEE Robotics and Automation Magazine RAM paper presented at IROS 2019 It should be cited as a RAM paper 48 IEEE ROBOTICS AUTOMATION MAGAZINE JUNE 2019 Conclusions This article discussed the design of a custom animated character having a diverse set of emotional expressions The emotions were simple yet meaningful and expressive The character design was extended to an iPod based mobile robotic platform that leveraged its embodiment to allow for the expression of each emotion as a combination of facial expressions physical movements and sound effects making it well suited for use with children with ASD A novel emotional interaction framework was also developed the feasibility of which was validated through a user study Our findings indicate that this tool has the potential to perform some important functions including engaging in empathetic emotional interactions helping with ER and even priming higher levels of user focus and engagement for given tasks Acknowledgments This research was partially supported by the National Insti tutes of Health under grant 5 R01 HD082914 04 through the National Robotics Initiative program References 1 S Baron Cohen and S Wheelwright The empathy quotient An investigation of adults with Asperger syndrome or high functioning autism and normal sex differences J Autism Developmental Disor ders vol 34 no 2 pp 163 175 2004 doi 10 1023 B JADD 0000022607 19833 00 2 A P Jones F G E Happ F Gilbert S Burnett and E Viding Feeling caring knowing Different types of empathy deficit in boys with psychopathic tendencies and autism spectrum disorder J Child Psychol Psychiatry vol 51 no 11 pp 1188 1197 2010 doi 10 1111 j 1469 7610 2010 02280 x 3 A Tapus and M J Mataric Emulating empathy in socially assistive robotics in Proc 2007 AAAI Spring Symp Multidisciplinary Collabo ration for Socially Assistive Robotics pp 93 96 4 J Decety The neurodevelopment of empathy in humans Develop mental Neuroscience vol 32 no 4 pp 257 267 2010 doi 10 1159 000317771 5 C A Mazefsky et al The role of emotion regulation in autism spectrum disorder J Amer Acad Child Adolescent Psychiatry vol 52 no 7 pp 679 688 2013 doi 10 1016 j jaac 2013 05 006 6 K Sofronoff T Attwood S Hinton and I Levin A randomized controlled trial of a cognitive behavioural intervention for anger man agement in children diagnosed with Asperger syndrome J Autism Developmental Disorders vol 37 no 7 pp 1203 1214 2007 doi 10 1007 s10803 006 0262 3 7 A Scarpa S W White and T Attwood Eds CBT For Children And Adolescents With High Functioning Autism Spectrum Disorders New York Guilford Press 2013 8 C M Conner S W White K B Beck J Golt I C Smith and C A Mazefsky Improving emotion regulation ability in autism The Emotional Awareness and Skills Enhancement EASE program Autism 2018 doi 10 1177 136