IROS2019国际学术会议论文集 0967

Situation Awareness for Proactive Robots in HRI Chapa Sirithunge H M Ravindu T Bandara A G Buddhika P Jayasekara and D P Chandima Abstract Perception of the intention of humans prior to an interaction is a demanding skill during human robot interaction HRI Thisskillisevenmoresoughtafter during robot initiated HRI Initiating an interaction in an inappropriate situation can be avoided when robots are equipped with the ability to decide when to interact and when not to Many of the existing systems investigate only a few characteristics of humans which demonstrate inner state of mind and are based on complex monitoring mechanisms which limit their use in most of the scenarios This work presents an autoregressive model based on observable physical and emotional human cues to determine the level of interest displayed by a human towards an interaction with a robot This model was implemented on a service robotic platform and the behavior of the robot was controlled using the model The behavior of the robot was determined by means of proxemic approach and the nature of conversation with the human The outcomes of the model were evaluated by analyzing user feedback in different situations inside a simulated social environment Using the model robot was given the ability to analyze the situation of its human user in an emotionally intelligent manner prior to an interaction The behavior of the model was reviewed by user feedback in order to validate the fi ndings Results of the experiment are presented and fi ndings of the study are discussed I INTRODUCTION Preference based assistance with robots is a demanding aspect in human robot collaborative environments 1 In such situations adopting supportive behaviors based on emotional intelligence rather than performing a requested task are required Tailoring such intelligent behavior is as important as challenging as well This is due to the complexity of human behavior as well as the diffi culty of perceiving such behaviors outcoming the challenges in technology and environment Thus human level prediction of scenarios still needs improvement Many robots used today still are expert in only a given specifi c task Cleaning robots 2 shopping assistants 3 rescue robots 4 healthcare robots and robotic nurses 5 are examples for robotic systems specialized for a certain task This requires a lesser emotional intelligence However robotic systems have become a tool to promote social and emotional interaction among humans and robots 6 At a particular situation the robot must take decisions regarding the interaction pattern required after a conscious observation Suchroboticsystemsarealreadyused inchildcareandpublicdomains Thesestillrequire This work was partially supported by University of Moratuwa under Senate Research Grant SRC LT 2018 20 The authors are with the Intelligent Service Robotics Group Department of Electrical Engineering University of Moratuwa Katubedda 10400 Sri Lanka e mail ra chapa ra ravindu buddhikaj and chandimadp uom lk development to adjust according to unexpected situations where human intentions alter Simulating and perceiving diverse social behaviors of humans is an outstanding feature to be engraved into a robot s personality 7 In contrast there are humanoids and other robotic systems which can replicate typical human features in the form of physical appearance words movements facial expressions etc An example system where robot s responses were evolved is presented in 8 Human attributes upon robots have to be considered before designing robot s intelligence The authors in 9 investigated how robots can improve quality of life of elders by being a companion to overcome loneliness Here people s attitude that robots are performing heavy tasks but are lagging in interactive behavior could be changed during the investigation Similarly robots are accepted by many communities as a companion having the required abilities installed within In such scenarios following an etiquette simply a robotiquette accepted by humans in social environment is expected 10 11 In this paper we propose a method to identify the interest of a human towards an interaction based on observable nonverbal cues These cues were applied to an autoregressive AR model to obtain a quantitative measure for the displayed interest of the human towards an interaction This measure was referred to as the Level of interest LOI This evaluation will be deployed by the robot itself to initiate an interaction with its user if the situation is favorable for an interaction The interaction was followed by decisioning upon which behaviors and etiquettes robot has to follow in order to make the scenario more comfortable and humanlike The system evaluates unclarifi able characteristics such as movements adopted by humans often before initiating an interaction with an outsider This enhances the decision making abilities related to interaction initiation as the system outputs a measure of the emotional state in a human encounter II RELATED WORK Mimicking real world HRI behaviors and simulating scenarios based on observable information are key features during human robot collaboration HRC 12 A predictive and adaptive system based on learning by demonstration is explained in 13 This system is an example for robots with supportive behavior for the user but it does not identify many salient features which portray human intention A situation conscious model is proposed in 14 to improve the design and interactive capabilities of an industrial robot These fi ndings show that the impact of social and spatial 2019 IEEE RSJ International Conference on Intelligent Robots and Systems IROS Macau China November 4 8 2019 978 1 7281 4003 2 19 31 00 2019 IEEE7807 environment have to be considered in order to design a context aware robot In 15 an android system with the capability of monitoring noverbal behavior of humans However this cannot generate physically appealing behaviors such as a robot There has been systems which were capable to match the appearance and demeanor as well 16 Even though such systems cover psychological and physical aspects in interaction there should be emotional aspects as well in order to behave appropriately in social environments A context sensitive approach to anticipate the human behavior while the human is followed by a robot has been proposed in 17 Although this anticipated human motionwhilewalking predictionofhumanbehavior during stationary situations becomes highly dependent of the emotional state and the task of that human Hence anticipation of stationary situations is a complex process There are complex approaches to recognize human activities such as 18 However less complex techniques to monitor human behavior are admired when real time decisions have to be taken Perceiving emotional cues that show affect is important in avoiding misbehaviors of robots and improving acceptance in human community 19 Findings suggest that perception of nonverbal behavior positively impacts HRC and hence the understandability of the robot increases as well 20 A situated interaction method which uses behavioral cues such as proximity velocity posture and sound information is presented in 21 A virtual companion adjusted himself according to the situation and outputs an engagement score at each occasion Even so only the gaze behavior of the virtual companion was evolved based on behavioral cues of the user An approach behavior model for initiating an interaction based on a number of user parameters was proposed in 22 In this method prediction of walking direction was useful in measuring user interest in speaking User unaware failure was diminished in the system by approaching the user before initiating the interaction However the performance of the system reduced when a stationary user was considered Mead et al proposed a method to evaluate the perception of distance in 23 During this approach robot used gestures and verbal instructions to determine the mutual distance In 24 a probabilistic approach has been adopted to analyze the engagement in verbal cues and gestures used by a human during interaction However both these approaches are possible only after initiating an interaction by the user Hence the task performance of a human robot team can considerably be improved by mutually understanding nonverbal cues responsible for the situation 25 Therefore perception of nonverbal cues related to human behavior and relating various cues together to determine the state of the situation are the attention seeking requirements during HRI A fuzzy logic based evaluation was performed upon human pose and body movements to determine an individual s attention in 26 The pose and the speed of body based movements have been utilized as a mediator to monitor the state of a situation by a robot whether to approach a person or not Hence the decisions regarding a robot initiated interaction were made based on the situation of the human subject However there are other variables which determine the state of a human robot encounter other than pose and body movements In 27 authors have increased the number of observable cues obtained from an encounter before a robot initiates an interaction A fuzzy system takes human gaze gaze angle gaze time and whether averted or not gestures number of friendly gestures gesture speed and gesture time and changes in pose as its inputs and fi nally calculates the attention given by a person to a robot This approach evaluated a considerable number of observable nonverbal cues to determine the interactivity of an encounter Even so there are other factors in a human which can display their inner state to the outside world Therefore we present a mechanism based on autoregressive AR models considering a number of nonverbal human cues related to different aspects We used multiple AR models to represent different psycho physiological aspects of human behavior uniquely Hence the behavior that is likely to be expected from the robot at that particular situation is determined by these AR models This model determines robot s behavior by means of three variables orientation with the user proximity and the nature of conversation to have with the user III DETERMININGLEVEL OFINTEREST A Requirement of Emotional Intelligence According to the theory of planned behavior one s behavior and his her beliefs are linked together 28 In addition the theory of reasoned action explains that there exists a relationship between one s attitudes and actions 29 Hence there can be both volunatry and involuntary responses generated by a human in the presence of a robot Therefore monitoring an individual s behavior prior to interaction with that individual is essential in planning an intelligent response towards human behavior This fact has been utilized in developing the decision making algorithms for both social and industrial robots 30 Nonverbal responses that were most likely to be conveyed from a human before and during a human robot interaction scenario were used to evaluate human behavior during the study Even though most of these responses are generated as a result of the internal state of mind it is confusing to measure these responses to determine some other quantity such as attention or interest Therefore we introduced an autoregressive model which relates different cues to their accepted social interpretations 31 For instance if a person smiles and changes his her pose as an outsider approaches him her the outsider interprets that to be a friendly situation Such a situation carries the interest of both parties for an interaction such as a conversation B Level of Interest The relationship between observable human cues and planned psychophysical behavior of a robot according to the observations is mapped in Fig 1 Level of interest LOI defi nes to which extent a person is interested in an 7808 Interest mutual gaze gestures movements LOI1 LOI2 LOI3 existence of gaze Nonexistence of gaze exitance of gestures number of gestures head movements hand movements nature of conversation Psycho physiological behavior Interactive distance orientation uttering few words greeting small talk long conversation social distance proximity interpersonal distance slightly inclined very inclined while engaged in an activity relaxed observable interest positive negative interactive responses Positive interaction decision Negative interaction decision in front Fig 1 A semantic map which shows the co relation between various aspects in human behavior considered during this study and responses generated from the robot in correspondence with its observations The directions of increment and decrement of interactivity are marked in red and blue respectively Interactivity increases as the situation turns favorable for interaction Inputs of the AR model 1 1 1 Gaze 0 1 No gaze Mutual gaze exists Medium gaze Walking speed 0 stationary Fast walkSlow walk Head movements 0 Fast reaction Slow reaction Medium speed reaction Hand movements 0 Fast movements Medium speed movements Slow movements Friendly gestures 1 0 No gesturesBoth the gestures involved One gesture Fig 2 Features used as the inputs to the AR model are shown interaction with the robot Human interest was classifi ed under three aspects mutual gaze gestures and movements According to the features associated with each aspect LOI changes Therefore three different LOIs have been introduced to distinctly identify what is communicated by each aspect Depending on the magnitude of three LOIs psycho physiological behavior of the robot is tuned For instance if the magnitudes of LOIs show a less user interest towards an interaction the robot may decide to utter a few words without going for a long conversation Similarly the robot may adjust positivity or negativity in interaction decision based on the user behavior This will facilitate an encounter with least discomfort due to the violation of expectations of both parties Furthermore mutual gaze will have a signifi cant effect upon the proximity between two conversant while gestures having the least effect Therefore we used multiple levels of interest which can contribute to determine robot s behavior in different amounts In our approach most distinctly perceivable and often used cues were chosen as the input features to assess the situation Observable features mutual gaze gestures and movements were used as a representation of interest for an interaction These three parameters are sub categorized as follows Mutual gaze existence of gaze non existence of gaze averted gaze Gestures nonexistence of gestures fewer gestures a number of gestures Movements head movements hand movements The system further analyzes gaze for the existence non existence looking away or if averted Similarly gestures were evaluated according to the number of gestures used and movements as speeds on head and hands These parameters are measured and used as the inputs of an autoregressive model to determine the level of interest of an encounter C AR model to calculate LOI As several features can be used in combination to make different interaction decisions LOI is sub divided into three as LOI1 LOI2and LOI3 These are chosen based on similar features that are used as observable human cues which display interest As usually done in AR models we assumed that these similar variables are linearly related Gaze was taken as a parameter to determine LOI1 Friendly gestures and walking speed are considered in the calculation of LOI2 while initial head and hand movements determine LOI3 Fig 2 illustrates the minimum and maximum ends of each input of the AR model The equations 1 2 and 3 are used to calculate LOI1 LOI2 and LOI3 7809 LOI1 a1 1 LOI2 b1 b2 b3 2 LOI3 c1 c2 c3 3 Here a1 a2 a3 b1 b2 b3 c1and c2are constants values of which were determined empirically before the experiment Predefi ned example scenarios were modeled using 1 3 assigning 1 for all the above constants Depending on the interactivity of the situation these values were changed until socially acceptable interaction decisions were made by the model Magnitudes of the three LOIs are evaluated before making decisions regarding interaction with the user Psycho physiological behavior required to initiate the interaction is determined in this way Three features which determine robot s behavior are found These are the orientation between robot and user proximity and the nature of conversation Here the nature of conversation was decided by the duration of the conversation The distance that has to be maintained between the robot and the user was determined by the proximity Orientation determines to which extent the robot should face the user These parameters are determined as per their sub categories as follows Nature of conversation greeting few words small talk few sentences long conversation large number of sentences Proximity socialdistance interactivedistance interpersonal distance Orientation directly towards user in front slightly inclined towards user very inclined Therefore the nature of conversation was decided as a greeting or a small talk or else a long conversation depending on the values obtained for LOI1 LOI2and LOI3 Similarly proximity was determined as an accepted social distance or an interactive distance or else as an interpersonal distance and orientation as an angle directly towards the user or inclined from the user IV DECISION MAKING CRITERIA Based on the features used to calculate LOI1 LOI2and LOI3 interaction decisions were taken after considering some of the values obtained for LOI1 LOI2and LOI3 All three are considered when determining the proximity The nature of conversation was determined by LOI1and LOI2 LOI1and LOI3determined the orientation of the robot LOI2is omitted here since the gestures exist only for a small duration so that the proxemic behavior of the robot will not interrupt user behavior The marginal values required to determine these output parameters are shown in Fig