IROS2019国际学术会议论文集 0870

Combined Optimization of Gripper Finger Design and Pose Estimation Processes for Advanced Industrial Assembly Frederik Hagelskj r1 Alja z Kramberger1 Adam Wolniakowski1 2 Thiusius Rajeeth Savarimuthu1and Norbert Kr uger1 Abstract Vision systems are often used jointly with robotic manipulators to perform automated tasks in industrial appli cations Still the correct set up of such workcells is diffi cult and requires signifi cant resources One of the main challenges when implementing such systems in industrial use cases is the pose uncertainties presented by the vision system which have to be handled by grasping In this paper we present a framework for the design and analysis of optimal gripper fi nger designs and vision parameters The proposed framework consists of two parallel methods which rely on vision and grasping simulation to provide an initial estimation of the uncertainty compensation capabilities of the designs In case the compensation is not feasible with the initial design an optimization process is introduced to select the optimal pose estimation parameters and fi nger designs for the presented task The proposed framework was evaluated in dynamic simulation and implemented in a real industrial use case I INTRODUCTION The establishment of a robotic solution for a pick and place action frequently requires an amount of set up time that makes robot solutions commercially unattractive This in particular holds when vision is used to localize the object before grasping 1 On the other hand vision is attractive compared to using hardware solutions such as bowl feeders or magazines prior to grasping due to the high costs involved One of the problems in the use of vision is that pose uncertainties become introduced due to factors such as sensor noise calibration imprecision as well as deviations in model fi tting in the actual pose estimation process The usual way of compensating for these errors is to design gripper fi ngers that are able to align the object even if grasped imprecisely This is usually achieved by designing specifi c cut outs in the fi ngers Industrial tasks especially assembly of precise production parts with low manufacturing tolerances require high precision and repeatable grasping of workpieces while being handled by the robot In this work we tackle one of those challenges and solve the part handling task for the Kendrion workpiece shown in Fig 1 objects on the left and Fig 4 Normally such parts arrive in the workcell in a defi ned position e g placed in fi xtures In our use case the position of the part was semi defi ned in the beginning the object is placed in a known pose on the table but can move and rotate freely on the table and the vision system therefore needs to determine its location accurately 1Maersk Mc Kinney Moller Institute University of Southern Denmark Odense Denmark frhag alk adwol trs norbert mmmi sdu dk 2Faculty of Mechanical Engineering Bialystok University of Technology Bialystok Poland a wolniakowski pb edu pl Rotation between templates Vision uncertainty Camera Object Gripper alignment capability Gripper optimization time offline Gripper design Object Vision computation time online Fig 1 On the left vision system parameters angle between the object templates used to establish the piece orientation control the vision system uncertainty red area at the expense of on line computation time in the middle On the right off line gripper optimization fi nds fi nger design parameters such that the gripper alignment capability blue area can compensate for the vision uncertainty red area In industrial set ups the purpose of grasping action is often to precisely place the workpiece in a defi ned position such as in a fi xture or a production machine In these type of actions the accuracy of the object detection is less important than the precise pose of the object grasped by the gripper fi ngers In this paper we refer to the term uncertainty to describe the noise of the pose detection performed on the object Given these conditions the problem of fi nger design can be formulated as fi nding the right geometry that achieves the optimal robustness to compensate an object s pose un certainty An ideal gripper would be able to grasp the object placed imprecisely such that it still ends up in the expected pose We refer to the magnitude of the uncertainty that the gripper fi ngers can compensate for as their alignment capability An additional problem is to estimate the actual amount of uncertainty introduced by vision that the gripper design needs to compensate for This often means that many trial and error tests are required until a suitable solution is found In this paper we formalize the imprecision of the vision solution as well as the alignment capabilities of the gripper in simulation With the presented framework it is possible to ensure that prior to implementation in the real world both 2019 IEEE RSJ International Conference on Intelligent Robots and Systems IROS Macau China November 4 8 2019 978 1 7281 4003 2 19 31 00 2019 IEEE2022 components fi t together see Fig 1 The left part of the fi gure illustrates the infl uence of the vision systems parame ters and the time invested on computing the precise pose on the uncertainty level of the pose estimate This computation is done online and therefore the runtime cost has to be paid at every time the pose estimation and the grasping actions are executed The right part of the fi gure illustrates how the gripper fi ngers design optimization process increases uncertainty compensation for the object pose estimation The fi nger design optimization is performed off line whereas for the vision if more computation time is invested during the workcell design a smaller online vision pose estimation runtime cost can be achieved during the execution The vision setup parameters infl uence the magnitude of the pose estimation uncertainty For template matching tech niques in particular the quantization of the template match ing is responsible for the precision that can be achieved In general a higher resolutions leads to a higher precision however with the cost of an increased runtime of the algo rithm limiting the feasibility of the solution see Fig 1 the leftmost arrow A higher uncertainty of the vision system can be compensated by further optimizing the fi nger design as done in 2 which requires off line computation Thus the selection of both appropriate vision and gripper design parameters needs to be balanced for optimal results which usually requires a large number of experiments Our approach allows us to ensure that the pose estimation setting and the alignment capabilities are in an appropriate re lation by simulating both and by that saving time expensive trial and error experiments that make robot solutions very costly today II STATE OF THE ART Typically in industrial practice the design of gripper fi ngers is a result of iterated testing and improvement of the original shape proposed by an experienced engineer This requires substantial resources and causes problems for Small and Medium Enterprises SMEs which often deal with small batch size production relying on a rapid change of the production profi le Thanks to the improvements in Computer Assisted Design CAD and Manufacturing numerous tools emerge that assist in the design of particular robotic workcell elements including gripper fi ngers One of such tools is EGRIP 3 developed by Schunk GmbH Recently a new trend is emerging where the design of elements for robotic workcells is performed utilizing sim ulation In 4 the simulation framework is used in order to optimize the pick and place action parameters and the workcell confi guration for handling soft body materials In 5 a framework is introduced which allows for fast and effi cient fi nger tip design using Computer Aided Design methods and kinematic simulation This framework was later extended in 6 to make the design of multi object grasping fi ngers In our previous work 2 we have presented a framework which uses dynamic simulation for the purpose of improving the gripper fi nger design for compensation of vision pose estimation uncertainties Setting up of a new visual pose estimation system consists of several consecutive steps sensor selection algorithm selection and parameter tuning 7 These steps are usually performed manually and adjusted until performance is satis factory The process of parameter tuning is time consuming and does not result in any analysis of the capabilities To decrease the set up time and provide an analytical measure of performance the parameter tuning can be performed in simulation J rgensen et al 8 and Iversen et al 9 presented a simulation based pose estimation optimization process however the work deals with camera placement and pose estimation based on 3D data while in our work we apply op timization of parameters for a 2D template matching process The work of J rgensen et al 8 performs a pose estimation with an object placed on a table The camera position is optimized to return the highest number of detections Iversen et al 9 also optimize the position of the camera towards the object but does not perform a full pose estimation Instead the camera is positioned to optimize the precision by the Iterative Closest Point ICP In our previous work 1 the optimization of internal parameters concerning vision algorithms has shown good results although not done in simulation Bayesian Optimization is used to optimize the algorithm parameters to increase detection A signifi cant difference between the previously mentioned approaches and our is that we have an adjustable parameter which addresses both uncertainty and runtime In this paper we present a novel framework for optimizing both gripper and vision for a complete pick and place action To our knowledge this is the fi rst approach wherein both the alignment capabilities of the gripper and the precision of the vision system is optimized concurrent to accommodate the presented task requirements III SYSTEM OVERVIEW In this section we describe the proposed framework for fi nger design in combination with pose estimation parameter design depicted in Fig 2 Both methods can be treated separately with a common integration process when dealing with real use cases in production facilities The input to the individual methods is represented as a combination of parameters describing the geometry and physical features of the objects for which the fi ngers and the vision system is to be designed Furthermore additional information on the environment e g camera parameters object placement in the workcell etc is introduced to the design procedure Our method consists of three subparts which are 1 Finger design The initial requirement of the method is to determine the proper gripper base actuator There are several options to chose from e g pneumatic hydraulic or servo electric which with their specifi c properties must comply with the task requirements In the second step the base shape length width and hight of the fi ngers is determined Based on this shape 2023 Task Parameters Object Model Grasp Position Camera Placement Camera Calibration Object Alignment Compensation Object Detection Accuracy Can Compensate Real World implementation of the obtained parameters YESNO Evaluation in Simulation Create object detection system Rotation quantization Design and optimize fi ngers Imprint design parametrized fi nger design Finger DesignObject Detection Evaluation in Simulation Finger Parameters Object Detection Parameters Optimization Evaluation in Simulation Finger Design Imprint design parametrized fi nger design Object Detection Rotation quantization Fig 2 Graphical representation of the workfl ow for the fi nger design and pose estimation framework the specifi c features in relation to the object geometry are designed A basic description of the design process using the imprint method 10 and the parameterized method 2 is given in Section IV B After the fi n gers are constructed their performance is evaluated in several simulation based grasping experiments The output of the evaluation determines how good the alignment properties of the fi nger design are 2 Object detection In the fi rst step the appropriate cam era must be selected to comply with the task require ments In the second step the position of the camera in the workspace of the robot must be determined The installation position is crucial to get the best object detection performance and cover the biggest workspace area of the robot When the initial requirements are set the object detection parameters e g template match ing precision and runtime are determined and their performance evaluated in simulation A description of the detection procedure is given in Section IV A The output of the simulation based evaluation gives the uncertainty level of the vision system based on the design parameters If the alignment capabilities of the fi nger design can compensate for the vision uncertainty or vice versa the combined solution is ready to be implemented in the real set up If this criterion is not met both sets of parameters are optimized dashed block 3 Parameter optimization If the outputs of the design evaluation do not meet the task requirements the parameters must be optimized to achieve better per formance In the fi nger design optimization process we can optimize the fi nger design features to that extent that the new optimized design can compen sate the pose estimation uncertainties On the other hand if the initial design of the fi ngers satisfi es the task requirements pose estimation in terms of higher precision versus runtime is optimized This process is repeated until a good tradeoff between the performance of both methods is achieved With this framework we ensure that the shape of the fi ngers will compensate for the vision pose estimation un certainties for any environmental condition of the proposed task IV METHODS A Vision system Object pose estimation is performed with our previously developed pose estimation system 11 The system is de signed for industrial pose estimation and returns very precise and robust object poses This system needs a semi defi ned pose object placed on the table but able to move and rotate freely and thus have limitations We decided to use this system as opposed to 2D or 3D 12 1 feature matching systems which provide full 6D poses as the pose estimation system was incorporated and benchmarked by the SDU robotics team at the World Robot Summit Assembly Challenge WRC2018 13 and it produced accurate and reliable pose estimation results In order to achieve very high precision the system gen erates templates online to match them locally The rotation is estimated by matching templates corresponding to each rotation respectively Increasing the number of generated templates will therefore increase algorithm runtime while decreasing the rotation uncertainty see Fig 3 Both runtime and uncertainty are parameters desired at a minimum Be cause those two parameters are dependent a balance must be found depending on the task specifi cations Determining the system runtime as a result of the number of templates is a simple task as the template generation can be performed independently of the task scene To determine the uncertainty of the real world system would require an extensive dataset with precise labeling In this paper we build a simulation system to verify our pose estimation uncertainty automatically 1 Simulation of vision system In our work we use but simplify the simulation system presented by J rgensen et al 8 and Rossmann et al 14 Our synthetic images thus do 2024 Fig 3 Plot showing the resulting runtime Time s and standard deviation of the angle error err from varying quantization of rotation As the runtime increases it is seen that error decrease Fig 4 Comparison of synthetic and real image of the Kendrion object not correspond completely with the real images Instead to ensure that the pose estimation performance in simulation and real world matches a generalization term is added as according to 15 Here different types of Gaussian noise is added to the image to generalize methods trained on synthetic images We evaluated that our simulation system gives approximately the same pose estimation results without an exact rendering thus requiring fewer parameters and a simpler set up compared with 14 The three external simulation factors in the system are 1 object 2 scene and 3 lighting All three are diffi cult to obtain from the real world Therefore we perform a simplifi cation as follows Object As the pose estimation is based on CAD models we use the same model as our object model However such objects are often metallic with specular surfaces and the CAD model does not include color information Thus a single color intensity is chosen for the object surface Scene Obtaining the full scene model with all com plexities is impossible but as the pose estimation is in 2D we can insert an image as background A standard textured background is chosen i e the fi rst instance of the Brodatz Dataset This enables us to determine the system s performance for any unknown scenes Lighting The last factor is the lighting condition We include a single light behind the camera as we want to include shadow errors in the model but do not want to determine all possible light sources As the camera is calibrated no distortion is present in the image Finally the position of the camera is obtained by placing a checkerboard on the table and calculating the position in relation to the camera Fig 5 Example image from the verifi cation dataset The object is placed in a fi xture where the two rightmost screws are visible The threaded holes are placed uniformly on the table 2 Verifi cation of simulation To verify the correlation between our synthetic images and the real world represen tation we use the test dataset from 11 This dataset does not compare the detections with ground