IROS2019国际学术会议论文集Robot-Based Machining of Unmodeled Objects via Feature Detection in Dense Point Clouds

Robot-Based Machining of Unmodeled Objects via Feature Detection in Dense Point Clouds* Dennis Hartmann, Michael Mende, Denis Stogl, Bj orn Hein and Torsten Kr oger AbstractMachining applications using robots are still not common in industrial settings. Reasons are the unintuitive programming concepts which typically require expert knowl- edge and the infl exibility regarding small alterations of the workpieces. We present a prototypical solution for an intuitive and fl exible robotic machining concept for unmodeled work pieces. For this we use a high resolution laser scanner to record very dense point clouds. Algorithms to detect linear edges with obtuse angled corners, linear edges with acute angled corners, linear inner edges and circular edges were developed, demonstrated and validated. To accurately execute generated trajectories in practice, an algorithm to directly calibrate the transformation between the sensor and the milling tool was developed. For the algorithms and the calibration process a repeatability tolerance of 0.2mm is achieved. I. INTRODUCTION Robot-basedmachiningwithindustrialmanipulators promises to be a fl exible alternative to state of the art technologies of processing and fi nishing work pieces 1. For small batch sizes or one-of-a-kind work pieces, which are increasingly common in manufacturing 2, it is advanta- geous to be able to quickly and intuitively program the robot to perform the desired actions within defi ned tolerances. Currently, single industrial robots for small batch sizes are often programmed by Teach-in 3, i.e. an expert operator uses a panel to drive the robot to approach relevant points and stores them in a robot program. A path is then constructed by interpolating between these points linearly or circularly either in joint space or work space. During execution it is possible to perform sensor-based corrections of the path to adjust for tolerances but even small alterations of the work piece require to repeat at least partly the Teach-in process. We present a solution to reduce the effort and expertise re- quired for industrial robot programming for machining tasks while adhering to sub-millimeter tolerances. The operator uses a custom input device with integrated pose tracking, force sensor, LC-Display and input buttons. The device and the associated programming paradigms will be presented in detail in a separate publication. Using this device one can select a parametrized process model, e.g. deburring or welding, and select the properties of the workpieces features, e.g. linear edges and circular edges. The operator then moves the tip of the device to the feature *This work was supported by the Federal Ministry of Education and Research of Germany within the Program Research for Civil Security - Project ROBDEKON (Funding No. 13N14678) The authors are with the Institute for Anthropomatics and Robotics, Karlsruhe Institute of Technology, Germanydennis.hartmann, michael.mende, denis.stogl, bjoern.hein, Fig. 1. The combined end effector with laser scanner and miller is scanning the 3D-printed test object. points of the workpiece to advise the robot where to search for specifi ed features. These feature points have an accuracy of 1cm and thus are not suffi cient to generate a precise tool trajectory and corresponding program for the robot. In our system a 2D-line laser scanner is mounted on the end-effector of the robot to generate high resolution 3D point clouds by moving the robot. The tool for fi nishing the work piece is part of the same end-effector (see Fig. 1). This combined end-effector allows precise hand-eye calibration of the transformation frame from the sensor to the fi nishing tools center point. To record the point clouds for the segments, scan trajec- tories are generated from the user input feature points. Each segment of the workpiece is then scanned separately by the robot. Using the high resolution point clouds and the feature points defi ned by the user, the selected features are extracted separately for each segment by the following steps: 1) Downsample the point cloud using a voxel grid to speed up computation at the cost of accuracy (op- tional). 2) Compute normals of the point cloud. 3) Extract planes using the region growing algorithm 4. 4) Compute model parameters of planes with random sample consensus (RANSAC) 5 using least median of squares regression (LMedS) 6 as the model evaluator. 5) Intersect planes to get edges and corners using the 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Macau, China, November 4-8, 2019 978-1-7281-4003-2/19/$31.00 2019 IEEE7771 input feature points as guidance. 6) Create execution path by discretizing the edges be- tween the corners. 7) Project discretized points onto local point cloud surface to adjust for height differences. 8) Adjust projected execution path according to the se- lected process model, e.g. transformations based on the used tool. To achieve sub-millimeter tolerances in practice, the trans- formation between the frame of the 2D-laser scanner and robot fl ange has to be known to a high degree. Alternatively, it is suffi cient to know the transformation between the frame of the laser scanner and the tool used for machining, since they are mounted rigidly on an integrated end-effector. The main contribution of this paper is a robot system that uses approximate positions of feature points of an un- modelled workpiece and generates high precision execution paths for milling. Our system can determine linear edges with obtuse angled corners, linear edges with acute angled corners, linear inward facing edges and circular edges both inward and outward facing. For milling we developed an algorithm to precisely calibrate the transformation frame from the sensor to the millers tool center point. This paper is structured as follows: In section II rele- vant previous work is discussed; Section III describes the approach and implementation; In section IV experimental results are presented and discussed and section V concludes the paper. II. RELATED WORK This section gives a brief overview of different methods for programming a machining task for a robot. A. Teach-In The most usual approach for programming industrial robots is the control panel provided by the manufacturer. With the control panel the operator moves the robot to poses in which the tool is in contact with the workpiece and saves these position in the robots program. This process is very time intensive and has to be repeated even if the workpiece was changed only slightly. Alternatively, for light weight robots equipped with force-torque sensors it is possible to move the end effector by hand to these positions. For larger robots this poses safety risks for the operator. A simplifi cation of the Teach-in process is shown in 7. Instead of using the teach pendant of the robot or hand- guiding the robot with force sensing, the authors developed two different input tools to program robot poses. This increases the speed of teaching and it is safer than hand- guiding the robot but the achieved accuracy of 0.71mm is not suffi cient for machining tasks. B. CAD Based Programming Robot programming with the help of Computer Assisted Design (CAD) is an Offl ine programming method, i.e. the robot is not directly involved during programming. There exist software packages that can generate a machining robot program from CAD-models of workpieces, see 1. A dif- fi culty for this programming method is the discrepancy between the model and the real world both for the robot and the workpiece. Machining requires very small tolerances, thus the calibration of the robot and the precise location of the workpiece must be done with high accuracy. In 8 trajectories for machining are generated based on CAD-models with tolerances for misalignment. A worker fi xates the work piece at a position defi ned in advance. A laser scanner mounted on a robot arm scans the work piece and computes the pose difference based on edges of the work piece. The generated trajectory from the CAD-model then gets transformed according to the pose difference. The reported errors were 0.31mm for x-translation, 0.43mm for y-translation and 0.36for rotation around the z-axis. Skotheim et al. 9 developed a system that uses laser triangulation mounted on a robot end-effector to record point clouds of objects and match them to know CAD models by a matching algorithm using oriented point pairs and a refi nement done by the Iterative Closest Point(ICP) algorithm. They have reached a mean error to ground truth of 1.81mm that was suffi cient for grasping the objects but is too large for robotic machining. Automatic programming based on CAD is also used by Posada et al. 10 for robotic deburring. A 2D laser scanner is mounted on the robot end-effector and records 3D point clouds by moving the robot. The workpiece is then located by generating a reference point cloud from the CAD model and using ICP to match the recorded point cloud with the reference point cloud. During motion generation, the freely choosable degree of freedom of the millers rotating axis is taken into account. This can increase the available work space of the robot. To adjust for inaccuracies during location of the workpiece and small deformations of the workpiece itself, the sensor is also used during execution to correct the movement of the robot. Error measurements were not reported. C. (Semi-)Automatic Programming Zhang et al. 11 illustrate semi-automatic deburring of aluminum wheels. A worker draws the deburring path on the wheels with a marker. A camera mounted on the robot guides the robot in x-direction and y-direction and orientation around the z-axis using visual servoing. To ensure a smooth result, before starting the machining step, the normal surface along the marked trajectory is measured with a force torque sensor and a special tool. By the constraint to always stay perpendicular to and at the surface, the remaining 3 degree of freedoms (DOF) can be inferred. Ziliani et al. 12 use a force-control approach to au- tomatically deburr casting workpieces of unknown shape using a specially designed milling tool. The tool has ball bearings above and below the milling surface, which can guide the tool along the shape by applying a constant normal force. Since a 2-DOF SCARA robot is used, complex-shaped workpieces that require more degrees of freedom cannot be machined. 7772 III. METHOD Here we describe the design and implementation of our system. We specify the data collection, the feature extraction algorithms and the calibration algorithm. A. Data from Input Device The necessary input data from the operator is sent from the input device to an OPC UA (Open Platform Communi- cations Unifi ed Architecture) 13 server. When a new task is triggered, our algorithm fetches the following data from the OPC UA server: The base transformation frame of the work piece The path of consecutive features consisting of: Segments with each segment having two points for linear segments and three or four points for circular segments, The type of the segment: circular or linear. Process parameters. B. Recording the point cloud Scan trajectories are generated individually for each seg- ment. Each segment is extended by 1.5cm at both ends to ensure the whole segment is recorded with the input data tolerance of 1cm. The path is interpolated in Cartesian space, linear segments with resolution of 0.1mm and circular segments with resolution of 0.001. The line laser scanner is positioned perpendicular to the segment angled at 45(see Fig. 1) for linear edges the orientation is the same for the full edge, for circular edges the orientation is perpendicular to the tangent. There are two possible orientations for the scan line to be perpendicular one looking directly at the edge and one where the top surface is blocking the line of sight to the side surface. Thus for the fi rst segment of a path we record one scan line in the middle of the segment and try to detect an edge. If for both orientations an edge cannot be found or the distance of the edge to the user input segment is more than 1cm, the process is aborted. Otherwise the orientation with an edge point found is chosen for the whole path of consecutive segments. The generated trajectories are then driven by the robot with a fi xed Cartesian speed of 5cm per second while the laser scanner records a point line with 500Hz. The individual scan lines therefore have a distance of 0.1mm. These are transformed to the robot base frame and concatenated to a 3D point cloud for this segment. Since our robot updates its joint positions every 12ms, the transformation of the scan lines has to be interpolated linearly, which introduces only a small error because of the fi xed Cartesian speed of the robot. C. Feature Extraction Algorithms To reduce the computational cost, the recorded point cloud can be downsampled by using a voxel grid. In this work a leaf size of 0.15mm is used. For each voxel the centroid of every point within the voxel is computed and then used as the new point for this voxel. Next, the normals corresponding to each point get com- puted using the normal estimation algorithm of the Point Cloud Library (PCL) 14 with a neighbor radius of 0.75mm. The next steps are different for linear and circular edges. 1) Linear Edges: The point cloud is segmented into the individual surfaces by using the region growing algorithm 4. The numbers of neighbours is set to 50, thto 1.25 and cthto 0.025. For each segment, plane model parameters are computed using the RANSAC algorithm 5 with LMedS 6 as the evaluator of the parameters. The threshold is set to 0.1mm and the model parameters are optimized by using LMeds on the inliers. The extracted planes have to be differentiated between the planes which belong to the workpiece and other planes, e.g. to a table. For this the convex hull of the inliers of each extracted plane is computed. These are then sorted in descending order with the distance of the user input points to the convex hull as the comparison operator. It is assumed that the top plane of the workpiece has a normal vector with a difference to the z-axis of the robot base frame of less than 15 . Thus, the fi rst plane in the list which satisfi es this is identifi ed as the top plane. Furthermore the side planes of the work piece need to have an angle to the top plane of more than 75. For both corners up to two side planes are chosen for further computation. Since the scan is made with the laser scanner perpendicular to the edge, a second side plan will only be available for obtuse corner angles. If the corner is at an obtuse angle and two side planes are identifi ed, the corner can be easily computed by the intersection of the top plane with the two side planes with Eq. (1) as can be seen in Fig. 2. P = d1(n2 n3) d2(n3 n1) d3(n1 n2) n1 (n2 n3) ,(1) with n1,n2,n3 being the plane surface unit normals, d1,d2,d3 the distances of the planes to origin and P the position vector of the corner. For acute angled corners, only one side plane is recorded and identifi ed. Intersecting this plane with the top plane Fig. 2.Corner with obtuse angle which was computed by intersection of the top plane with two side planes. 7773 Fig. 3.Corner with acute angle that was determined by our line search algorithm. provides the line parameters for the edge. To extract the corners of this edge a radius search of the cloud along the computed line is implemented. The two user input points for this segment are averaged and projected onto the line. Starting from this point along the line in both directions the point cloud is searched for nearby points in a radius of 7.5mm in steps of the same size. At the step where no more points are found, the search is reversed with a step size of 0.01mm and the corner is identifi ed when the radius search returns a point again. Fig. 3 illustrates the result of the algorithm. 2) Circular Edges: After computing the normals of the point cloud, the top plane of the workpiece is extracted using the RANSAC algorithm using a plane model. This plane is used to clip the point cloud in two parts 5mm below the plane along the z-axis. In the lower part of the point cloud, the RANSAC algorithm is used to compute parameters for a cylinder model of the side of the workpiece. For circular edges we make the assumption that the top plane and side of the workpiece include an angle of 90 degrees. Thus, the intersection of the cylinder with the top plane is a circle inside the top plane. For whole circle segments the start and end points are the same and are computed by projecting the input start point on the circle. For circular segments that are part of a path, the start point of the circular segment is the end point of the previous segment projected onto the circle and the end point of the circular segment is the start point of the next segment projected onto the circle. In this case the following linear segment has to be computed fi rst. 3) Projection into Point Cloud: Workpieces in practice do not have the perfect geometric shapes we assumed in our algorithms and have imperfections such as burrs. To adjust for this as a last step we project the execution path partly back onto the point cloud. For linear edges we adjust the height and for circular edges we adjust the height and position in the x/y-plane. a) Height Adjustment: The path between the start and end points is discretized in 0.1mm steps along the line or circle. For each discretized point a local neighborhood of the points of the top plane is averaged. The distance of this average point to the discretized point is projected onto the Fig. 4.Linear edge with curved height. The red points are on the plane intersection. The blue points have had their height adjusted. normal of the top pla