IROS2019国际学术会议论文集 1359

A Probabilistic Approach to Human Robot Communication Elizabeth Cha Emily Meschke Terrence Fong and Maja J Matari c Abstract Since robots are increasingly expected to work in concert with humans in dynamic unstructured environments they will need to express information about their state and ac tions We propose a formalism for planning robot communication that employs a probabilistic representation of the robots world This representation takes on the form of a Markov Decision Process MDP and captures the uncertainty of interacting with humans The key insight of this work is that humans preferences and time need to be carefully balanced against the robots in order to minimize human annoyance The communication MDP enables the robot to reason about the effects of its actions on a human interactor We validated the model through a human subjects experiment n 44 by learning communication policies for a loosely collaborative task The results show that the communication MDP improves participants perceptions of the robot s thoughtfulness as an interactor I INTRODUCTION As robots become increasingly capable their ability to effectively communicate with humans is becoming crucial for successful human robot interactions Communication can take on many forms such as speech or nonverbal cues e g gesture light auditory To develop similar levels of fl uency as in human human interaction robots will need to employ these various types of communication actions Many past works in human robot communication have explored the research challenges associated with shaping a communication action for a particular modality For verbal communication generating natural language that accurately conveys the robots intended information or request has been a primary goal Research with nonverbal communication has looked at how particular types of cues for the same modality e g type of gesture affect human responses to the robot In contrast this work is primarily concerned with the challenges associated with a robot deciding when and how to communicate to humans For robots to be effective communi cators they will need to employ different modalities and with varying levels of urgency saliency and other communication properties The variety of environments robots will operate within also highlights the emerging need for robots to be able to adapt to different human states Most robots however adopt a myopic strategy when com municating they transmit a communication action when the need arises without consideration of the humans current state Acting greedily enables the robot to meet its own objectives but can alienate human interactors and reduce their willingness to interact in the future In collaborative scenarios where the humans task also impacts the robots this behavior negatively Cha and Meschke contributed equally to this work Elizabeth Cha Emily Meschke and Maja J Matari c are with the In teraction Lab University of Southern California Los Angeles CA USA echa emeschke mataric usc edu Terrence Fong is with Intelligent Robotics Group NASA Ames Research Center Mountain View CA USA terry fong nasa gov This work was supported by a NASA Science and Technology Research Fellowship NSF NRI IIS 1528121 and a USC URAP impacts the overall team performance as well Instead the aim of this work is to enable robots to act more considerately when communicating to improve the dynamics of the interaction We propose a formalism for planning robot communication that leverages methods from human computer interaction with the goal of achieving robot communication that is thoughtful of the human interactors time and preferences We compu tationally encode a model which captures knowledge about the humans robots and environmental state using a Markov Decision Process MDP This model is used to plan the robots communication actions while balancing the humans and robots differing goals and preferences via a joint reward function The reward function the robot optimizes incorporates the hu mans and robots task objectives and the humans receptiveness to communication Since the robots communication interrupts the humans current activities consideration of interruptibility factors such as their availability is key to designing an effective reward function We also incorporate information about past interruptions as this affects the humans attitude towards the robots interactions We validated the MDP model for communication through a human subjects experiment n 44 on a loosely collaborative task in simulation During this experiment we learned a policy for the robot s communication behavior As the model s exploration rate decreases participants rated the robot as a more considerate interactor Section II reviews relevant prior work in robotics human computer interaction and communication Section III presents our computational formulation of the robot communication model Section IV describes a human subjects experiment to validate the proposed model and Section V discusses the experiment results We conclude with a discussion of the impact of this work and its relation to HRI as well as future research directions in Section VI II RELATEDWORK Mediating the fl ow of communication has been studied extensively in human computer interaction HCI for building systems that provide notifi cations or alerts Such systems often attempt to transmit notifi cations in a manner that provides the most value to humans This cost based approach requires the system to probabilistically infer the human users state and use this information to determine whether a communication from the system will be disruptive or benefi cial While several works discuss this cost based approach at a high level only a few explore in depth how a utility function can be constructed to determine the gain or cost of a communication Instead more focus is often given to the research challenges associated with estimating a humans interruptibility or designing notifi cations with varying levels of saliency or attention from the human user Methods for estimating a humans interruptibility enable sys tems to weigh the humans current state when deciding whether 2019 IEEE RSJ International Conference on Intelligent Robots and Systems IROS Macau China November 4 8 2019 978 1 7281 4003 2 19 31 00 2019 IEEE6217 to transmit a communication by assigning a higher cost to communication actions when the human is less interruptible Interruptibility can be defi ned by several factors relating to a humans current task and engagement Availability or whether the human is already engaged in a task is one of the easiest metrics to employ However this approach ignores the variance in potential tasks which range in their physical and cognitive effort and makes certain tasks more sensitive to interruption The temporal dynamics of a humans current task also im pacts interruptibility Certain tasks that are more time sensitive are worse for interruptions since there can be critical delays that cause negative impacts to the person or task Prior work that has explored which moments of a task are best suited for interruptions have found that certain breakpoints enable easier resumption of a task at a later point 1 4 8 As a result models for predicting task breakpoints have been well researched 14 The content of the interruption has also been shown to be important as irrelevant communications taking longer to process and cause humans greater annoyance 7 Another challenge is that humans are able to perform several tasks simultaneously or quickly switching between tasks To consider these different factors some systems attempt to reason about humans attentional state using Bayesian infer ence 13 Different costs can then be assigned to interruptions in each attentional state This method offers fl exibility in that the attentional state can encapsulate different tasks break points and other variables related to human interruptibility The primary drawback is that it requires potential attentional states to be known in advance and assigned a cost which may be challenging for robots that operate in unstructured settings Noisy observations such as a robots sensor measurements can also lead to more inaccurate state estimations Since humans can be in different states of attentional awareness communication actions need to also take on varying levels of saliency urgency and other properties 3 16 18 These properties can also be used to express information about the systems desired response There has already been signifi cant research on perceived urgency or how immediately action is required by the human Higher saliency communication signals are often used for more urgent situations since they elicit more attention and a faster response from human users Communication actions that draw greater attention from the human should result in higher costs The cost of a communi cation has two components the cost of the human attending to the communication signal and the cost of the humans subsequent response 13 Previously these costs have been measured by the time each takes away from the humans main task 13 Thus if a human ignores the communication only the fi rst component exists These costs can then be weighted by those associated with the humans state Few works in HRI have employed similar techniques for mediating communication Much of the past research has focused on generating effective communication actions given a robots unique embodiment 5 A select number of these works investigated communication signals specifi cally for minimizing interruptions 20 21 The ability to effectively communicate under uncertainty is important for robots who collaborate with or require assistance from humans 6 19 However there is very limited work in HRI employing the models and fi ndings from notifi cations and interruptibility research in HCI 23 In this work our goal is to enable robots to probabilistically reason about communication actions to act as a more thought ful interactor We build off past work in HCI by addressing the research problem of designing an effective utility function for deciding when and how a robot should communicate We employ only nonverbal signals in this work as they are an important part of communication and facilitate fl uid effortless coordination Since robots are embodied agents that humans attribute more complex properties to we also aim to further investigate how past fi ndings in mediating communication are affected by the use of a robot III MODELINGROBOTCOMMUNICATION In this section we present a computational model encoding the problem of robot communication A key assumption of this model is that humans will not always respond to the robot This can occur for several reasons Communication requires the human to attend to the robot and perceive its signals which can fail when the human is busy or there is signifi cant perceptual interference The human can also choose not to interact with the robot if they are busy or annoyed with the robot This require a probabilistic representation to deal with the inherent uncertainties of interaction A MDP Model We formalize the human robot communication problem as a Markov Decision Process MDP The MDP can be described by a tuple S A T R where S is a fi nite set of states of the world states It models different confi gurations of the human robot and environ ment A is a fi nite set of communication actions that the robot can execute T S A S is the state transition function that gives a probability distribution over world states for each state and action The transition function captures the variability in human response to a communication action either from stochasticity in the environment or the human s behavior R S A R is the reward function or the expected immediate reward for taking a communication action a in state s represented by R s a The robot s policy is the assignment of a communication or signaling action s to every state s The optimal policy s is the mapping of communication actions that maximizes the expected reward B State Space The state S captures information about the human robot and environment that is important to consider when the robot is deciding when and how to communicate While we assume a dyadic interaction with one robot and one human interactor in this work this model also supports multiparty interactions which can enable the robot to specifi cally target certain humans for communication as shown in 19 Robot State The criticality of a robot s communication need defi nes the importance of the interaction Consequently when 6218 choosing a communication action the robot must carefully consider whether it requires a response from the human and if so how quickly There are several approaches for modeling the factors that affect criticality and with varying levels of abstraction Many of the potential variables employed in prior work are tied to a particular robot and therefore too precise to use in a more general model for robot behavior 2 5 22 Instead we adopt broader categories similar to those found in HCI literature to describe the robot s communication ne cessity and the desired response from the human interactor These variables include urgency bu i and the effort bei and duration bd i of the desired human response Other encodings such as the 5 notifi cation levels described in 15 can also be used Human State The goal of the robot s communication is to affect the human at the simplest level the robot provides information However in other scenarios communication is designed to elicit specifi c responses Consideration of the human s current state is essential to the success of these interactions In this work we employ the human s availability as a measure of their state Availability has been defi ned in the past as a binary variable describing whether someone is engaged in task or not Since that does not encompass other information about the task availability fi is described by factors relating to the human s attentional state task diffi culty hd i and moment of interruption hmi fi hD i hM i F Task diffi culty is a measure of how much effort is involved in a task while the moment of interruption describes whether the human is at a breakpoint in the task or not We also propose employing another variable receptiveness as a measure of the human s willigness to respond to the robot This can be determined using past interactions or inferred probabilistically As a fi rst step this work employs a MDP making the fi rst defi nition more suitable We formulate receptiveness as a summation of the human s availability fjat each past interaction j As time passes the effects of these interruptions are lessened To model this dynamic each interaction is multiplied by an exponential function such that recent interactions are weighted more vi n X j 1 e tn tj fj 1 Environment State Since the perception of communication signals can be greatly affected by environmental factors such as ambient noise levels and distance the environment state can be used when reasoning about communication actions to increase the robot s effectiveness Factors that typically impact perception of communication signals include Ambient Sound Levels High noise levels in the ambient environment can mask auditory signals while low noise levels can increase the salience of a signal Humans naturally use this information when communicating to avoid violating social protocols e g yelling in a library or to increase their effectiveness e g waving to get attention in a loud environment Fig 1 The simulated gridworld environment that the experiment took place in the human and robot collaborate to perform tasks around a farm Visibility Representing the visual noise in the environ ment is a challenging task as there are many factors and uncertainty is determining where a human is looking making it better suited for a POMDP implementation Distance Psychophysics research shows that the percep tion of both auditory and visual stimuli is affected by the distance between the human and signal source Therefore if the robot is out of range of the human interactor it should expect no response C Action Space The action space consists of communication actions the robot can take In this work we employ nonverbal signals as they are effi cient and an important component of interaction The robot can also choose to take no communication action deferring till a future time or never executing an action at all D Transition Function Due to the numerous factors affecting signal perception and human response this model assumes an unknown state transition function Since there is signifi cant uncertainty in human responses and our fi rst validation is in simulation we employ only model free techniques for solving the MDP in this work E Reward Function A primary goal of this work is to enable the robot to balance both its own and a human s preferences and needs during communication In a perfectly collaborative environment the human and robot have a shared goal and optimize the same reward function 9 With current robot applications this is often not the case as humans and robots have differing priorities Even amongst humans with the same goal there is variance in the reward functions they optimize e g driving When planning its actions the robot should take into ac count the human interactor s goals and preferences We propose a reward function with two components a term relating to the human s preferences rH and a term relating to the robot s goals rR R rR rH 2 We can consider each of these terms as the positive and negative effects on the robot s and human s tasks We also 6219 consider two other factors information gain and the human s annoyance In this work we only employ annoyance which can be described as the human s receptiveness vi the less receptive a human is when the robot takes a communication action the more they will be annoyed IV LEARNINGROBOTCOMMUNICATIONPOLICIES The primary goal of this work is enabling robots to use a prob