IROS2019国际学术会议论文集 1592

Learning to Estimate Centers of Mass of Arbitrary Objects Sean McGovern1 Huitan Mao2 and Jing Xiao1 2 Abstract This paper introduces a reinforcement learning algorithm with robot manipulation to learn an arbitrary ob ject s center of mass whose physical material composition is unknown Robot learning is through manipulation of the object in a sequence of actions The effectiveness of the algorithm is demonstrated in simulation to locate the centers of mass of rocks with complex shapes with even or uneven mass distributions and confi rmed by vertically stacking the rocks along their learned centers of mass both in simulation and in real experiments I INTRODUCTION Advances in machine learning algorithms have allowed intelligent robots to perceive more useful information from environmental sensors and to adapt to unknown environ ments Using state of the art algorithms with cameras real time detection 2 3 and tracking 4 of objects is possible These algorithms provide object detection up to but not limited to 30 FPS and also provide online adaptation for improved tracking However once an object is detected and identifi ed with techniques such as these physical information about that object is still limited to its surface shape based on optical 3D measurement systems and 3D point cloud processing techniques such as 5 An object s center of mass is one of the most essential physical attributes of an object to know for manipulation and optimal interaction For instance before humans interact with an object we make a good guess of the center of mass through previous experiences 6 and can quickly fi nd the center of mass through slightly manipulating the object It is possible to calculate an object s center of mass if its mass is evenly distributed from a 3D point cloud However the calculation of the center of mass of an object with a very complex shape may be diffi cult or overly inaccurate In addition an object that is made out of different materials may have a center of mass that is not the same as its geometric center The center of mass of an object in this case cannot be found without some interaction if the physical material composition is not visibly obvious In order to interact optimally with arbitrary objects in an unknown environment a robot will need to be able to estimate the centers of mass of objects Related work includes instances of robots learning its own center of mass for more effi cient movement and control 1 This robot uses online learning to update its own mass model for more effi cient balance control However there is little work found on a robot agent estimating the center of mass 1The authorsare affi liatedwiththeRoboticsEngineeringPro gram WorcesterPolytechnicInstitute smmcgovern wpi edu jxiao2 wpi edu 2 Huitan Mao is affi liated with the Department of Computer Science University of North Carolina at Charlottehmao4 uncc edu of unknown objects of arbitrary shape 7 explains how humans use visual observations to judge whether an object is physically stable or not to estimate its center of mass to guide their motor actions Fortunately in 8 there is similar work on a robot stacking arbitrarily shaped objects with rel atively good performance The work used extracted physical information the center of mass and dynamic characteristics from a physics engine about rocks with complex shapes to fi nd the best fi t pose in order to stack rocks in a vertical tower By assuming the center of mass of an object is known it is limited to already have a model of the object to be stacked optimally The rocks were also assumed of even mass distribution In this paper the proposed reinforcement learning al gorithm uses no prior known physical information of the arbitrary object whose center of mass is to be learned The object is also of complex shape and has a physical material composition that is not visibly obvious The following sections will fi rst introduce the general problem and assumptions Next the reinforcement learning algorithm that estimates the center of mass of an arbitrary object will be described in detail Then in simulation the proposed reinforcement learning algorithm is tested by learning the centers of mass of rocks with complex shapes The simulation uses objects with both even and uneven distributions of mass and the results from these will be shown We then use the learned centers of mass to perform a vertical stack of the rocks to demonstrate their utility and accuracy After that the results of learning are further tested in a real experiment with a robot stacking the rocks along their learned centers of mass Finally future research and possible applications will be discussed in conclusions II PROBLEM DEFINITION AND ASSUMPTIONS A Problem Defi nition For a target object A of arbitrary shape size and distri bution of mass the problem is to learn its center of mass given it is bounded by a rectangular bounding box and the center of the bounding box by reinforcement learning The learning is through manipulating A with respect to a base object B with a fl at surface The objective is to have an algorithm with a Q learning neural network that can be used to estimate A s center of mass to a desired level of accuracy We propose a reinforcement learning algorithm to learn an arbitrary object s center of mass through manipulation The reinforcement learning algorithm uses a Q learning neural network that will choose the best action to take according to Q values The Q values are updated with a reward from interacting the target object A with a base surface of B that is fl at and has a straight 90 degrees edge along the x and y 2019 IEEE RSJ International Conference on Intelligent Robots and Systems IROS Macau China November 4 8 2019 978 1 7281 4003 2 19 31 00 2019 IEEE1848 world frame axes X and Y The reward for the algorithm will be based on whether the object A is balanced or not when placed on the edge along X or Y of the base surface B The reinforcement algorithm will choose a direction to search for the localized center of mass that will result in a closer approximation with each step In order to carry out the reinforcement learning the following sub tasks need to be performed The object will need to be placed accurately within a millimeter along the edge of the base surface B Once object A is placed the object will need to be tracked for any displacement The object s orientation will need to be maintained in order to repeat the process with confi dence B Assumptions For the reinforcement algorithm in this paper to perform properly in a real world experiment the following assump tions are made Object A s appearance can be detected and from the camera images of object A a 3 D point cloud can be computed that gives a rough estimate of the object s full shape This point cloud can be used to compute the current pose of object A A rough 3 D bounding box of A can also be built The center of the rough bounding box serves as a reference point for A The object is also of a manageable size and shape to be manipulated with a robot gripper The object will maintain balance when placed at certain orientations on a fl at surface which is required to learn at least two dimensions of its center of mass It is also feasible to reset the object to its initial position and orientation during the process Lastly the robot is able to place the object at a position with suffi cient precision to achieve the required precision of estimation of the object s center of mass In the simulation implemented in this paper the center of a 3 D bounding box of an object is assumed within some distance of the geometric center of the object model The pose of the object with respect to the world coordinate system is retrieved from the simulation environment III METHODOLOGY A Q learning algorithm is used to learn each new arbitrary object A s center of mass that the robot encounters Fig 1 illustrates the Q learning process The learning algorithm s state denoted by is the distance between the center of the bounding box which is the origin of the object frame and the currently estimated center of mass It is initialized to be zero i e initially the center of mass is assumed to be the same as the center of the bounding box of the object with respect to the world coordinate frame which is the robot base Initially object A is put on top of an edge of B aligned with either X or Y axis of the world coordinate frame with its bounding box center directly above the edge Consider that the edge of B is aligned with the X or the Y axis A reward R is used to decide the Q values from which an action direction a is determined as either 1 or 1 corresponding to the increase or decrease direction of the coordinate value ym or xm respectively The ym or xm value of A is then changed by a certain step amount stepin the direction according to a See Fig 2 Q values are updated after each action on the object over the edge of B by refi tting the neural network The step amount stepfor an action is initially one centimeter and is reduced incrementally to a millimeter then a tenth of a millimeter stepwill only be reduced once the center of mass component of the object is found within a precision of its current value Fig 1 Q Learning Algorithm for Center of Mass Estimation It is important that we determine the object A s coordinate frame XAYAZAat the center of the bounding box such that when the XAYA plane is aligned to the fl at surface of B a stable pose of object A can be achieved in order to fi nd xm and ym In addition when object A is rotated to make its ZA axis aligned with the fl at surface of B there should also be a stable pose so that zmcan be found Otherwise only xmand ymcan be found The order that the center of mass components will be found can be ym xm then zm This process is outlined in Algorithm 1 At the start of each component search the robot agent will place the object A in the center of the fl at surface of B at the initially found stable pose Next the robot can begin placing object A in that orientation along the edge of B that is parallel to the world coordinate axis X or Y in order to estimate either ymor xm Throughout the procedure the orientation of the object is unchanged on the edge of B to compute the same component ymor xm If the object stays balanced then the true center of mass is more towards the center of B The action that moves the object away from the center of B is given a greater reward and the neural network is refi t with the updated state and Q value instance j j The reward is used to train the neural network and steer future actions with similar states to the correct direction that was or should have been taken The next state j 1 which will be only stepaway from the previous will have updated Q values from the previous instance and the action with the max Q value will move the object away from the center of B again This will continue to occur until the object is no longer balanced as the center of mass is no longer supported by the contact surface of B The action to move away from the center of B will now receive a bad reward and once the neural network is refi t with the new instance the new action will be to move the object A towards the center of B The object will then be placed steptowards the center 1849 Fig 2 Learning ymcomponent of the center of mass of Rock 2 of B and should maintain balance In order to fi nd a more accurate center of mass the step size stepis now decreased by a factor of 0 1 The entire process above is repeated until the center of mass is found on the given axis to a tenth of a millimeter Once this sequence of actions and rewards are complete the object s center of mass on this axis is confi rmed The next component is then run through the same loop Once xm ym of the target object A is found A is rotated to change its contact surface to the surface of B and the zm component of A s center of mass can be found in a similar way This algorithm is a Q learning algorithm that gets a reward immediately after each step and assigns this reward to the appropriate pairs of state and Q values These pairs are then used to train a supervised neural network The neural network for this problem is used due to its ability to predict the best action given a state that it has not already visited IV IMPLEMENTATION AND SIMULATION This section describes how the Q learning algorithm is implemented in the simulation First the simulation was executed using rocks with even mass distributions These rocks have complex shapes making them ideal for testing the chosen learning method Then the simulation was performed on rocks with the same exact shapes but with uneven mass distributions The results for both simulations are shown and compared to their respective true inertial centers Finally the sets of rocks were stacked vertically in simulation using their learned centers of mass The simulations were performed using Gazebo ROS and a model of YuMi provided by Orebro University Gazebo is a simulation environment that utilizes ROS and implements a physics engine for dynamic simulations YuMi is ABB s col laborative dual armed robot and is designed to work side by side with humans The rock models were originally Blender models only containing surface areas They were imported into Solidworks and the surface areas were intersected to Algorithm 1 Learning Center of Mass Result Find xm ym zm xm ym zm 0 0 0 s0 Adjust y j 0 Found False Init NN Balanced True R 0 9 Find pose such that Object A is balanced on surface B while not Found do aj argmaxQ sj a if Adjust y then ym ym aj step Place Object A on B with current ymon edge along X end if Adjust x then xm xm aj step Place Object A on B with current xmon edge along Y end if Adjust z then zm zm aj step Place Object A on B with current zmon edge along Y end if Object A is unbalanced on surface B then Balanced False if aj 1 then maxaQ sj a R minaQ sj a 1 R else maxaQ sj a 1 R minaQ sj a R end end if Object A is balanced on surface B then if Balanced False then if Adjust y then ymis found Adjust x else if Adjust x then Rotate object A about Y to fi nd zm if Object A is unbalanced then Found True end xmis found Adjust z else zmis found Found True end end end Balanced True if aj 1 then maxaQ sj a 1 R minaQ sj a R else maxaQ sj a R minaQ sj a 1 R end end Refi t j 1 end 1850 fi ll in the rocks URDF and mesh fi les were exported from Solidworks for simulation in Gazebo Fig 3 Models of rocks with even mass distributions Rocks 1 2 and 3 are shown from right to left In the simulation environment the base surface object B is placed directly in front of YuMi The base surface object will not move from its position for the entirety of the simulation The physical environment provided by Gazebo is used to simulate whether an object would fall over given the position it was placed on an edge of the base surface object Object A s position and orientation are retrieved from the Gazebo In Gazebo Object A s position is set as the origin of the URDF model which is slightly off from the model s true inertial center The difference between the model s origin and the model s true inertial center is made to simulate any error between the bounding box center and the geometric center of the rock s in a real world experiment This is provided under the assumption that the position and orientation of the object will be obtained from the detected bounding box and the 3D point cloud perceived during real world experiments The reinforcement learning algorithm communicates with the YuMi robot to tell it where to place the object as precisely as possible on the base surface Whether object A is balanced or not is informed by the physical simulation YuMi is able to manipulate its arm in order to position its end effector into grasping positions as it would need to do in a real world experiment Just as the methodology section explained object A will fi rst be tested at its current orientation to confi rm that it will maintain balance For the simulation the algorithm checks the position of the object in the gazebo environment a few moments after placement to verify that its position has not changed within a certain tolerance If the object does not maintain balance then a new pose is attempted by rotating the object along the axis of the component currently being learned However if there is no pose that satisfi es the tolerance such as some of the rocks used in the experiment then that axis cannot be learned As stated in the assumptions only two axes of the center of mass are required to be learned The algorithm will then use the initial approximation of the center of mass and instruct YuMi to place the object A along the edge of the base object B A Arbitrary Objects with Even Mass Distributions In order to demonstrate the usefulness of learning the center of mass of objects before manipulating them the reinforcement learning algorithm was applied on three rocks of different complex shapes and sizes with even mass distri butions These rocks are shown in Fig 3 Table I shows the learned locations of the centers of mass of rocks 1 2 and 3 The true inertial centers are also given from the rocks URDF fi le The majority of errors were less than 2 millimeters The accuracy of the learning algorithm could be down to a tenth of a millimeter given the right conditions TABLE I Comparison of rocks learned centers of mass to true inertial centers N A denotes a center of mass component that could not be found at different poses Learned Center of Mass Rocks with Even Mass RockLearned xm cm Learned ym cm Learned zm cm 10 48N A0 33 20 170 430 39 30 22N A0 80 Model True Inertial Center RockTrue xm cm True ym cm True zm cm 10 420 150 18 20 300 420 26 30 350 190 36 B Arbitrary Object With Uneven Mass Distribution Fig 4 Models of rocks with uneven mass distributions Rocks 1 2 and 3 are shown from right to left The same simulation was also performed using rocks with the same complex shapes but with uneven mass distribu