IROS2019国际学术会议论文集 1245

Continuous Modeling of Affordances in a Symbolic Knowledge Base Asil Kaan Bozcuo glu1 Yuki Furuta2 Kei Okada2 Michael Beetz1 Masayuki Inaba2 Abstract As robots start to execute complex manipulation tasks they are expected to improve their skill set over time as humans do A prominent approach to accomplish this is having robots to keep models of their actions based on their experiences in order to improve their action executions in the future In this paper we present such a methodology where robots start to execute some actions with random parameters and record their generic execution logs with semantic annotations in a symbolic knowledge base for robots Using the data inside logs multivariate Gaussian mixture models are fi tted to the high level action parameters for later executions These affordance models are being updated whenever a new execution is carried on In essence robots can use these continuously updated probabilistic model for improving their actions To prove the applicability we demonstrate opening a fridge door experiments with a PR2 robot I INTRODUCTION Recently we have witnessed tremendous efforts and ac complishments from roboticisits to put service and personal robots in our homes 1 2 3 4 our work environments 5 and even scientifi c laboratories 6 Although these works are fundamental milestones towards bringing robots to our daily life the skill set of robots is still limited in terms of fl exibility robustness and adaptability In other words these actions can be executed by robots only under certain condi tions with middle to high failure rates and highly generous time requirements We believe that having human like cog nition skills such as developing affordances 7 and learning by scaffolding 8 is crucial to attack these limitations and broaden the skill sets of robots Moreover most of these applications use case specifi c data which cannot be re used or adapted to new tasks If we consider how data hungry today s machine learning techniques are collecting a new set of data for different context brings a huge overhead In particular we focus on the notion of affordances one of the core concepts in ecological psychology coined by J J Gibson In 9 he argues that animals perceive their action affordances not isolated from the environment they live in In other words affordances are extendable by practicing but these improvements are constrained by the environmental features and physical laws Such explorations may also be useful for robots to improve their action executions as the development of the notion of affordances for robots has 1A K Bozcuo glu and M Beetz are with the Institute for Artifi cial Intelligence Universit t Bremen 28359 Bremen Germany asil beetz cs uni bremen de 2Y Furuta K Okada and M Inaba are with JSK Lab Department of Mechano Informatics Graduate School of Information Science and Technology The University of Tokyo 7 3 1 Hongo Bunkyo ku Tokyo 113 8656 Japan furushchev jsk imi i u tokyo ac jp k okada inaba jsk t u tokyo ac jp Fig 1 The architecture of the proposed methodology from the robot s perspective Prediction function f x denotes Eq 6 given in this paper been well studied subject with different use cases such as traversability of mobile robots 10 11 grasp ability of different objects 12 13 and human robot interactions in social context 14 We anchor the concept of affordances with the parame terization of actions that robots execute regularly In other words we do not deal with explorations of affordances as in 10 11 12 13 Instead we take the robot s affordance of the action as granted and model which motion parametrizations enable this affordance For this purpose we present a novel approach for the affordance modeling where robots sample parameters for their actions from the respective Gaussian mixture models GMM In return they update these models after every execution using self supervision In order to refrain from the aforementioned case specifi c data we implement the presented methodology within a symbolic knowledge base where robots have the possibility of annotating their generic execution logs using available semantic annotations This enables robots to use the data from other experiments if they are similar These logs are not only use case specifi c subsymbolic data for our parametriza tion learning method but a complete brain dump from the corresponding execution which contains semantic high level knowledge such as what kind of action was carried out which arm was used what the goal was and if there exist any failures together with all existing subsymbolic data like robot trajectories joint states and sensory inputs Using these logs we implement a closed loop robot planner which uses probabilistic affordance models as the feedback term to improve plan executions over the time 2019 IEEE RSJ International Conference on Intelligent Robots and Systems IROS Macau China November 4 8 2019 978 1 7281 4003 2 19 31 00 2019 IEEE5452 Employing this methodology in a symbolic knowledge base also brings several advantages First the generated models are integrated with the rest of the knowledge base in terms of data structures semantic annotations and interfaces with other components of the robot As a result robots can infer and use these affordance models using the uniform query interface in our case Prolog as they do for the rest of the knowledge resides in the knowledge base Moreover this interface provides an application layer where roboticists can apply the presented methodology to different use cases or different parameter sets just by modifying the queries inside the robot control executives Our methodology fi rst inquires which parameters lead to a successful executions and which lead to a failed one by querying knowledge base After having this training data extracted from the symbolic knowledge base multivariate Gaussian mixture models GMM are fi tted for both positive and negative datasets These models are used by the robot in terms of prediction functions for parametrization Moreover every new execution is used for updating the existing model Figure 1 depicts the overall methodology The rest of this paper is organized as follows We give the related works in the subject of action parametrization Thereafter we describe the proposed methodology together with the log format used how to obtain parameters from logs and how this methodology is integrated with the knowledge base Then we depict our experimental results together with analysis and explanation of how outside users can access the implementation and the experiment data Lastly we conclude with the fi nal remarks and future road map II RELATEDWORKS Recently Zech et al 15 present a literature survey on affordance based computational models with recent architec tures on going challenges and a systematic classifi cation In 16 Levine et al demonstrate an explanation based learning methodology which attempts to learn quantitative relationships between the actions an agent performs and the state variables describing the world The ARPlaces approach by Stulp et al 17 models probabilities of task success prior to execution based on parameters such as relative locations of the robot and the objects it needs to grasp Both studies are particularly related to our work since they also use subsymbolic action parameters and action effects rather than STRIPS style abstract features 18 These abstractions in STRIPS often fail to capture small environmental properties to help boosting performance such as the existence of a place from which several objects can be grabbed from which obviates the need for several navigation actions Moreover probabilistic representations of locations in terms of their effects on actions is deployed in a relatively large knowledge base to parameterize vague actions 19 There are some related works on Gaussian Mixture Models in robot learning applications such as learning by demonstra tion 20 21 22 and model based control 23 Finally in 24 the authors present a cloud based simulator using which the robot can fi nd a reinforcement policy for reaching and traversing affordances III METHODOLOGY The pipeline of our system is shown in Figure 1 In this section we will present our methodology in detail together with its building blocks A The Structure of the Execution Logs We make use of the format of episodic memory 25 which stores high level knowledge such as the tree of high level tasks inside the plan task parameters and failure and success states of each goal as an OWL ontology a semantic web ontology format by W3C consortium1 Subsymbolic data which contains joint states trajectories sensory data and camera images is stored in a mongoDB database Both OWL and mongoDB entries are stored with timestamps so that these two sources can be matched for queries such as What was the pose of the right arm when the fridge opening started These generic logs contain all possible information that the robot has at the time of execution Consider a scenario that you have realized that your new learning scheme would perform a lot better if you supply a new parameter say camera pose It would be just a matter of changing Prolog query to add this parameter if the executions are recorded as episodic memories During their experiments robots record such memories by using Prolog predicates described in Table I B Obtaining Plan Parameters from the Episodic Memories In 25 a set of predicates are introduced for reasoning on episodic memories We use an extended version of this set which includes predicates for Gaussian mixture modeling and silhouette value based K Means clustering analysis 26 These predicates are listed in Table II We can take the following query which is asked to the knowledge base for obtaining positive dataset of fridge opening task as an example findall ParameterList entity Act an action task context OpenFridgeDoor task start Act StartTime task success Act true belief at robot base link RobotPose StartTime pose into relative coord RobotPose FridgePose RelativePose matrix translation RelativePose RelTrans matrix rotation RelativePose RelRot append RelTrans RelRot ParameterList ParameterListofLists generate feature files ParameterListofLists positive csv This query asks for all action instances whose context is OpenFridgeDoor in the symbolic knowledge base Then it fi lters out the negative ones with task success Act true Looking at the start of each action it asks the robot s pose 1https www w3 org OWL 5453 PredicateDescription cram start action Type Context StartTime PrevAction ParentAction Inst Asserts a new action with given Type Context and StartTime There are also optional parameters PrevAction and ParentAction for maintaining task tree hierarchy Returns Inst which is the unique identifi er of the new action cram fi nish action Inst EndTime Asserts EndTime as the fi nishing time for the action Inst cram set subaction Parent Child Asserts subAction relation between Parent and Child cram add failure to action Inst Type FailureLabel Time FailureInst Asserts a failure with the given Type FailureLabel at Time Returns FailureInst which is the unique identifi er of the generated failure TABLE I Predicates used for recording the symbolic logs PredicateDescription mixed gaussian PosFile PosCluster NegFile NegCluster Gauss Given the positive and negative parameter fi les PosFile NegFile and maxi mum number of clusters PosCluster NegCluster this predicate fi rst clusterize the negative and positive datasets then fi ts multi variate Gaussian models for each At the end this predicate stores the fi nal GMM to the variable Gauss get likely location Gauss Mean Cov This predicate search through the global maximas in the GMM After fi nding them it fi ts a multi variate Gaussian model MGM to these maximas and return its mean and covariance These properties are used by the robot during the execution time for choosing an appropriate location generate heat maps Gauss This predicate is used to visualize the generated GMM whose path is given in the variable Gauss inOPENEASE TABLE II Predicates that are implemented for generating multivariate Gaussian mixture models GMM information with respect to the global reference frame to the non symbolic database Thereafter this pose information is transformed into the frame of the fridge door and returned as 7D pose information 3 dimensions for representing the position and 4 dimensions for quaternion to a temporary parameter fi le in the format of CSV C Clustering Training Data Episodic memories contain pose information with respect to a global reference frame To make use of this data in context specifi c learning applications we need a represen tation in the corresponding local reference frame In this way we can easily reuse or adapt the existing models to different modalities For instance if there exists a drawer opening model of a certain drawer in a kitchen we can reuse this model for another drawer just by substituting reference frames In our experiments this is the reference frame of the fridge door As depicted in the query in Section III B we convert the robot s pose at the beginning of open fridge door from global reference frame to the frame of the fridge door During the random experiments with the robot we have observed that it can actually open the fridge not with uniformly distributed poses but rather in different subzones such as approaching from the right with a modest proximity or approaching from the left with less proximity Thus given that these poses are scattered throughout the area we cluster them based on distance To arrive at a sensible number of clusters we analyze different number of K Means clusters up to an upper limit Then we calculate their average point silhouette value 26 and use the clustering with the lowest average This results in an optimal point distribution over all clusters D Fitting Multivariate Gaussian Mixture Models GMMs For generating affordance models we use Gaussian Mix ture Models GMMs as in 27 However we make use of both successful and failure instances in these models Namely we gather positive and negative parameter sets as described in Section III B Then we cluster both sets using K Means based on average point silhouette value Then for each cluster we fi t a Multivariate Gaussian Model MGM whose covariance matrix Ciis formulated as Ci si X k 1 X i k Xi T X i k Xi 1 where i and sidenote the index of the cluster and the size of the cluster respectively Since we have no limit for the size of training sets we keep not the samples but the covariance matrix Ci Rp pand the mean Xi Rpof clusters The density function fi X for evaluating the distribution of test data X Rpfor the MGM of cluster i is calculated as fi X exp 1 2 X X i T Ci X X i 2 Having MGMs of each cluster we form one GMM for the positive dataset and one GMM for the negative dataset by applying equal weight 1 n to each of the n clusters involved To sample from their distribution we evaluate the mixture s positive and negative density functions F pos neg X n pos neg X i 1 w pos neg i f pos neg i X w pos neg i 1 n pos neg 3 where pos and neg denote positive and negative respectively In the experiments we use the same weight for both positive and negative datasets because they include mostly isolated executions that have less occurrences On the other hand they can be chosen differently for instance using the number of data each cluster has One of the important aspects of this aforementioned methodology up is that it is effi cient and fast in terms of 5454 processing time Thus the calculation of GMMs can be repeated any time on demand or on arrival of new datasets E From Affordance Models to Prediction Function After GMMs are generated and stored as data assets in the knowledge base we derive a prediction function from these models for robots to query during the executions This function will be used by the robot for inferring the best possible position Hence the process needs to be formally defi ned since robots need to re generate these functions after every new execution we defi ne the following algorithmic steps to generate prediction functions Having two GMMs one from positive dataset and one from negative dataset we fi rst identify bounding boxes of these datasets in the space and take the union of these as the prediction function domain With a predefi ned step size we scan through the success and fail probabilities of each point If we consider each of these random training samples brings equal information gain we can take the weighted average of two GMMs according to the number of positive and negative training samples Flikeliness X nposFpos X nneg 1 Fneg X n 4 With a predefi ned step size we scan through the success and fail probabilities of each point and register ones that are the most likely to success i e global maximas By having a set of global maximas we fi t a multivariate Gaussian model MGM to this set Using the mean and covariance of this MGM we obtain a general prediction function by fmx X exp 1 2 X X mx T Cmx X X mx 5 Cmx nmx X k 1 X mx k Xmx T X mx k Xmx 6 The predicate that follows these steps to generate the prediction function is get likely pose 3 which accepts the current robot pose and temporary parameter fi les as the parameters and returns the success probability along with the mean position which implies the point with the highest probability If the current pose does not have a signifi cant probability planner moves the robot to this point Calculating global maximas by taking both positive and negative GMM into account and generating a cost function by fi tting a multi variate Gaussian model to these maximas bring two main advantages First as explained previously this method enables us to model affordances based on every trial where robots can also update the model with the negative samples to refl ect their certainty of failure Second these prediction functions have a single multivariate Gaussian model MGM which is easie