IROS2019国际学术会议论文集1245

Continuous Modeling of Affordances in a Symbolic Knowledge Base Asil Kaan Bozcuoglu1, Yuki Furuta2, Kei Okada2, Michael Beetz1, Masayuki Inaba2 AbstractAs robots start to execute complex manipulation tasks, they are expected to improve their skill set over time as humans do. A prominent approach to accomplish this is having robots to keep models of their actions based on their experiences in order to improve their action executions in the future. In this paper, we present such a methodology where robots start to execute some actions with random parameters and record their generic execution logs with semantic annotations in a symbolic knowledge base for robots. Using the data inside logs, multivariate Gaussian mixture models are fi tted to the high-level action parameters for later executions. These affordance models are being updated whenever a new execution is carried on. In essence, robots can use these continuously- updated probabilistic model for improving their actions . To prove the applicability we demonstrate opening-a-fridge-door experiments with a PR2 robot. I. INTRODUCTION Recently, we have witnessed tremendous efforts and ac- complishments from roboticisits to put service and personal robots in our homes 1, 2, 3, 4, our work environments 5 and even scientifi c laboratories 6. Although these works are fundamental milestones towards bringing robots to our daily life, the skill set of robots is still limited in terms of fl exibility, robustness and adaptability. In other words, these actions can be executed by robots only under certain condi- tions with middle-to-high failure rates and highly generous time requirements. We believe that having human-like cog- nition skills such as developing affordances 7 and learning by scaffolding 8 is crucial to attack these limitations and broaden the skill sets of robots. Moreover, most of these applications use case-specifi c data which cannot be re-used or adapted to new tasks. If we consider how data-hungry todays machine learning techniques are, collecting a new set of data for different context brings a huge overhead. In particular, we focus on the notion of affordances, one of the core concepts in ecological psychology coined by J. J. Gibson. In 9, he argues that animals perceive their action affordances not isolated from the environment they live in. In other words, affordances are extendable by practicing but these improvements are constrained by the environmental features and physical laws. Such explorations may also be useful for robots to improve their action executions as the development of the notion of affordances for robots has 1A. K. Bozcuo glu and M. Beetz are with the Institute for Artifi cial Intelligence, Universitt Bremen, 28359 Bremen, Germany asil, beetzcs.uni-bremen.de 2Y. Furuta, K. Okada and M. Inaba are with JSK Lab, Department of Mechano-Informatics, Graduate School of Information Science and Technology, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-8656, Japan furushchevjsk.imi.i.u-tokyo.ac.jp k-okada, inabajsk.t.u-tokyo.ac.jp Fig. 1: The architecture of the proposed methodology from the robots perspective. Prediction function, f(x), denotes Eq. 6 given in this paper. been well-studied subject with different use cases such as traversability of mobile robots 10, 11, grasp-ability of different objects 12, 13, and human-robot-interactions in social context 14. We anchor the concept of affordances with the parame- terization of actions that robots execute regularly. In other words, we do not deal with explorations of affordances as in 10, 11, 12, 13. Instead we take the robots affordance of the action as granted and model which motion parametrizations enable this affordance. For this purpose, we present a novel approach for the affordance modeling where robots sample parameters for their actions from the respective Gaussian mixture models (GMM). In return they update these models after every execution using self-supervision. In order to refrain from the aforementioned case-specifi c data, we implement the presented methodology within a symbolic knowledge base where robots have the possibility of annotating their generic execution logs using available semantic annotations. This enables robots to use the data from other experiments if they are similar. These logs are not only use case-specifi c subsymbolic data for our parametriza- tion learning method but a complete “brain dump” from the corresponding execution which contains semantic high-level knowledge, such as what kind of action was carried out, which arm was used, what the goal was, and if there exist any failures, together with all existing subsymbolic data like robot trajectories, joint-states and sensory inputs. Using these logs, we implement a closed-loop robot planner which uses probabilistic affordance models as the feedback term to improve plan executions over the time. 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Macau, China, November 4-8, 2019 978-1-7281-4003-2/19/$31.00 2019 IEEE5452 Employing this methodology in a symbolic knowledge base also brings several advantages. First, the generated models are integrated with the rest of the knowledge base in terms of data structures, semantic annotations and interfaces with other components of the robot. As a result, robots can infer and use these affordance models using the uniform query interface (in our case Prolog) as they do for the rest of the knowledge resides in the knowledge base. Moreover, this interface provides an application layer where roboticists can apply the presented methodology to different use-cases or different parameter sets just by modifying the queries inside the robot control executives. Our methodology fi rst inquires which parameters lead to a successful executions and which lead to a failed one by querying knowledge-base. After having this training data extracted from the symbolic knowledge-base, multivariate Gaussian mixture models (GMM) are fi tted for both positive and negative datasets. These models are used by the robot in terms of prediction functions for parametrization. Moreover, every new execution is used for updating the existing model. Figure 1 depicts the overall methodology. The rest of this paper is organized as follows: We give the related works in the subject of action parametrization. Thereafter, we describe the proposed methodology together with the log format used, how to obtain parameters from logs, and how this methodology is integrated with the knowledge base. Then, we depict our experimental results together with analysis and explanation of how outside users can access the implementation and the experiment data. Lastly, we conclude with the fi nal remarks and future road-map. II. RELATEDWORKS Recently, Zech et al. 15 present a literature survey on affordance-based computational models with recent architec- tures, on-going challenges and a systematic classifi cation. In 16, Levine et al. demonstrate an explanation-based learning methodology which attempts to learn quantitative relationships between the actions an agent performs and the state variables describing the world. The ARPlaces approach by Stulp et al. 17 models probabilities of task success prior to execution based on parameters such as relative locations of the robot and the objects it needs to grasp. Both studies are particularly related to our work since they also use subsymbolic action parameters and action effects, rather than STRIPS-style abstract features 18. These abstractions in STRIPS often fail to capture small environmental properties to help boosting performance such as the existence of a place from which several objects can be grabbed from which obviates the need for several navigation actions. Moreover, probabilistic representations of locations, in terms of their effects on actions, is deployed in a relatively large knowledge base to parameterize vague actions 19. There are some related works on Gaussian Mixture Models in robot learning applications such as learning by demonstra- tion 20, 21, 22 and model-based control 23. Finally, in 24, the authors present a cloud-based simulator using which the robot can fi nd a reinforcement policy for reaching and traversing affordances III. METHODOLOGY The pipeline of our system is shown in Figure 1. In this section, we will present our methodology in detail together with its building blocks. A. The Structure of the Execution Logs We make use of the format of episodic memory 25 which stores high-level knowledge such as the tree of high-level tasks inside the plan, task parameters and failure and success states of each goal as an OWL ontology, a semantic web ontology format by W3C consortium1. Subsymbolic data, which contains joint states, trajectories, sensory data, and camera images, is stored in a mongoDB database. Both OWL and mongoDB entries are stored with timestamps so that these two sources can be matched for queries such as: “What was the pose of the right arm when the fridge opening started?” These generic logs contain all possible information that the robot has at the time of execution. Consider a scenario that you have realized that your new learning scheme would perform a lot better if you supply a new parameter, say camera pose. It would be just a matter of changing Prolog query to add this parameter if the executions are recorded as episodic memories. During their experiments, robots record such memories by using Prolog predicates described in Table I. B. Obtaining Plan Parameters from the Episodic Memories In 25, a set of predicates are introduced for reasoning on episodic memories. We use an extended version of this set which includes predicates for Gaussian mixture modeling and silhouette value-based K-Means clustering analysis 26 (These predicates are listed in Table II). We can take the following query, which is asked to the knowledge base for obtaining positive dataset of fridge- opening task, as an example: ?findall ( ParameterList , ( entity ( Act ,an ,action , task_context ,OpenFridgeDoor ) , task_start ( Act ,StartTime ) , task_success ( Act ,true ) , belief_at ( robot ( base_link ,RobotPose ) , StartTime ) , pose_into_relative_coord ( RobotPose , FridgePose ,RelativePose ) , matrix_translation ( RelativePose , RelTrans ) , matrix_rotation ( RelativePose ,RelRot ) , append( RelTrans ,RelRot ,ParameterList ) , ParameterListofLists ) , generate_feature_files ( ParameterListofLists , positive . csv ) . This query asks for all action instances whose context is OpenFridgeDoor in the symbolic knowledge base. Then, it fi lters out the negative ones with task_success(Act, true). Looking at the start of each action, it asks the robots pose 1/OWL/ 5453 PredicateDescription cram_start_action(Type, Context, StartTime, PrevAction, ParentAction, Inst)Asserts a new action with given Type, Context and StartTime. There are also optional parameters PrevAction and ParentAction for maintaining task-tree hierarchy. Returns: Inst which is the unique identifi er of the new action cram_fi nish_action(Inst, EndTime) Asserts EndTime as the fi nishing time for the action Inst. cram_set_subaction(Parent, Child)Asserts subAction relation between Parent and Child. cram_add_failure_to_action(Inst, Type, FailureLabel, Time, FailureInst)Asserts a failure with the given Type, FailureLabel at Time. Returns: FailureInst which is the unique identifi er of the generated failure TABLE I: Predicates used for recording the symbolic logs. PredicateDescription mixed_gaussian(PosFile, PosCluster, NegFile, NegCluster, Gauss) Given the positive and negative parameter fi les (PosFile, NegFile) and maxi- mum number of clusters (PosCluster, NegCluster), this predicate fi rst clusterize the negative and positive datasets , then, fi ts multi-variate Gaussian models for each. At the end, this predicate stores the fi nal GMM to the variable, Gauss. get_likely_location(Gauss, Mean, Cov) This predicate search through the global maximas in the GMM. After fi nding them, it fi ts a multi-variate Gaussian model (MGM) to these maximas and return its mean and covariance. These properties are used by the robot during the execution time for choosing an appropriate location. generate_heat_maps(Gauss)This predicate is used to visualize the generated GMM whose path is given in the variable, Gauss inOPENEASE. TABLE II: Predicates that are implemented for generating multivariate Gaussian mixture models (GMM). information with respect to the global reference frame to the non-symbolic database. Thereafter, this pose information is transformed into the frame of the fridge door and returned as 7D pose information (3 dimensions for representing the position and 4 dimensions for quaternion) to a temporary parameter fi le in the format of CSV. C. Clustering Training Data Episodic memories contain pose information with respect to a global reference frame. To make use of this data in context-specifi c learning applications, we need a represen- tation in the corresponding local reference frame. In this way, we can easily reuse or adapt the existing models to different modalities. For instance, if there exists a drawer- opening model of a certain drawer in a kitchen, we can reuse this model for another drawer just by substituting reference frames. In our experiments, this is the reference frame of the fridge door. As depicted in the query in Section III-B, we convert the robots pose at the beginning of open fridge door from global reference frame to the frame of the fridge door. During the random experiments with the robot, we have observed that it can actually open the fridge not with uniformly-distributed poses but rather in different subzones such as approaching from the right with a modest proximity or approaching from the left with less proximity. Thus, given that these poses are scattered throughout the area, we cluster them based on distance. To arrive at a sensible number of clusters, we analyze different number of K-Means clusters up to an upper limit. Then, we calculate their average point silhouette value 26, and use the clustering with the lowest average. This results in an optimal point distribution over all clusters. D. Fitting Multivariate Gaussian Mixture Models (GMMs) For generating affordance models, we use Gaussian Mix- ture Models (GMMs) as in 27. However, we make use of both successful and failure instances in these models. Namely, we gather positive and negative parameter sets as described in Section III-B. Then, we cluster both sets using K-Means based on average point silhouette value. Then, for each cluster, we fi t a Multivariate Gaussian Model (MGM) whose covariance matrix Ciis formulated as: Ci= si X k=1 ?X i,k Xi ?T?X i,k Xi ? (1) where i and sidenote the index of the cluster and the size of the cluster respectively. Since we have no limit for the size of training sets, we keep not the samples but the covariance matrix Ci Rppand the mean Xi Rpof clusters. The density function fi(X) for evaluating the distribution of test data X Rpfor the MGM of cluster i is calculated as: fi(X) = exp ? 1 2 ?X X i ?T Ci ?X X i ? (2) Having MGMs of each cluster, we form one GMM for the positive dataset and one GMM for the negative dataset by applying equal weight 1 n to each of the n clusters involved. To sample from their distribution, we evaluate the mixtures positive and negative density functions: Fpos,neg(X) = npos,neg X i=1 wpos,neg i fpos,neg i (X) wpos,neg i = 1 npos,neg (3) where pos and neg denote positive and negative respectively. In the experiments, we use the same weight for both positive and negative datasets because they include mostly isolated executions that have less occurrences. On the other hand, they can be chosen differently, for instance using the number of data each cluster has. One of the important aspects of this aforementioned methodology up is that it is effi cient and fast in terms of 5454 processing time. Thus, the calculation of GMMs can be repeated any time on demand or on arrival of new datasets. E. From Affordance Models to Prediction Function After GMMs are generated and stored as data assets in the knowledge base, we derive a prediction function from these models for robots to query during the executions. This function will be used by the robot for inferring the best possible position. Hence the process needs to be formally defi ned since robots need to re-generate these functions after every new execution. we defi ne the following algorithmic steps to generate prediction functions: Having two GMMs one from positive dataset and one from negative dataset, we fi rst identify bounding boxes of these datasets in the space and take the union of these as the prediction function domain. With a predefi ned step-size we scan through the success and fail probabilities of each point. If we consider each of these random training samples brings equal information gain, we can take the weighted average of two GMMs according to the number of positive and negative training samples. Flikeliness(X) = nposFpos(X) + nneg(1 Fneg(X) n (4) With a predefi ned step-size we scan through the success and fail probabilities of each point and register ones that are the most likely to success, i.e. global_maximas. By having a set of global_maximas, we fi t a multivariate Gaussian model (MGM) to this set. Using the mean and covariance of this MGM, we obtain a general prediction function by: fmx(X) = exp ? 1 2 ?X X mx ?T Cmx ?X X mx ? (5) Cmx= nmx X k=1 ?X mx,k Xmx ?T?X mx,k Xmx ? (6) The predicate that follows these steps to generate the prediction function is get_likely_pose/3 which accepts the current robot pose and temporary parameter fi les as the parameters and returns the success probability along with the mean position which implies th