IROS2019国际学术会议论文集 0726

Abstract We have successfully achieved manipulation and assembly of microbeads having the size of 100 m diameter by hemispherical end effectors with high stability and accuracy The motivation of achieving assembly of actual cells lies in the great significance of it in tissue regeneration and cell analysis Firstly the most difficult problem we need to solve is the releasing problem caused by adhesion force The viscosity on cell surface is much larger than the microbeads which makes cell releasing challenging Secondly the cell can generate its deformation then contact area with end effector will change during grasping process This may influence the adhesion force and also bring problem to releasing Thirdly cell is much smaller around 15 m in diameter so we need to fabricate smaller end effector to achieve successful manipulation and ensure the stability in the meantime In this paper we realize the manipulation by decreasing the adhesion forces and apply vibration to release a cell stably We found the appropriate scale size for the end effector is around 10 m diameter It can not only grasp a 15 m cell but also bring little interference to the environment As a demonstration of the proposed manipulation method the repeated experiments were conducted to explore the dependence of adhesion force on the grasping distance which can be helpful in the improvement of successful rate Finally we achieved automatic cell assembly using Hela cells I INTRODUCTION Tissue regeneration plays an important part in biological field Manual operations of cells cannot guarantee high stability and accuracy Due to the rapid development of micro nano operation micromanipulation with micro robotics has drawn more and more attention especially in the aspects of such as micro measurement 1 sing cell manipulation 2 and cell assembly 3 The main process for robotic micro manipulation is grasping and releasing This manipulation is highly controllable and reproducible because the movement of micro robot is definable and deterministic However the grasping problem caused by small size of cells and the releasing problem caused by adhesion force of cells This work was supported by the National Natural Science Foundation of China under Grants 61873037 the Grant in Aid for Scientific Research JP19H02093 from the Ministry of Education Culture Sports Science and Technology of Japan and China Postdoctoral Science Foundation BX20190035 J Chen X Liu S Done P Li Q Tang D Liu Q Huang and T Arai are with the Intelligent Robotics Institute School of Mechatronical Engineering Key Laboratory of Biomimetic Robots and Systems Ministry of Education Key Laboratory of Intelligent Control and Decision of Complex System Beijing Institute of Technology Beijing 100081 CHINA phone 86fax 86 10 68915812 e mail 120170169 Masaru Kojima is with the Department of Systems Innovation Graduate school of Engineering Science Osaka University 1 3 Machikaneyama Toyonaka Osaka 560 8531 Japan Osaka University Tatsuo Arai is also with the Global Alliance Lab the University of Electro Communications Tokyo Japan are the two main problems we need to overcome The tools for micro manipulation are classified into single end effector and dual end effector 4 12 Manipulation with single end effector are mainly employing the adhesion force electromagnetic field or negative pressure Appling adhesion force to grasping is simple but it cannot guarantee high success rate and sometimes the targets lose grip during transportation Applying magnetic field to manipulate is only suitable for paramagnetic objects Thus it cannot be used for cell manipulation Fine force control is also challenging while applying negative pressure for manipulation Grasping with dual end effector such as microgripper and micro pipette is just like human grasping object using two fingers Thus this bio mimic method achieves stable grasp and greatly improved the grasping success rate However the problem of losing cells during transportation still exist because of the lack of precise cooperation between dual end effectors and small contact area between the end effector and the target Adhesion force in contact area between end effectors and cells makes releasing difficult Many methods have been developed by researchers They can be classified into the independent releasing and the assisted releasing The independent releasing applies the movement and behavior control of end effectors to overcome the adhesion force W Rong has developed a vacuum tool to generate pressure to help releasing 4 E Kim released micro object applying vibration generated by piezo actuator 13 K Takahashi has transported micro object by changing the voltage in electric field 14 K Chen have fabricated special structure of micro gripper which has an extra end between two end effectors 15 This extra end will push micro object to separate it from end effectors Some researchers concentrate on accelerate the movement of end effectors to achieve high success rate of releasing However this may lead to low accuracy and we cannot control the position of micro object after high frequency vibration Thus it comes to the trade off that ensure success rate and improve the accuracy at mean time T Chen released 20 100 m polystyrene sphere by the vibration of two fingered microhand combine different directions movement 16 Finally they achieve 100 success rate with accuracy of 3 5 m 4 5 m Assisted releasing depends on changing property of substrate or end effectors This releasing method does not rely on the movement of microhand itself For example J Dejeu controlled adhesion force by changing chemical environment such as pH value 17 18 O Fuchiwaki utilized ultraviolet curing substrate to change its adhesion force 19 S Saito applid Au coated method to control adhesion force 20 Automatic Cell Assembly by Two fingered Microhand Junnan Chen Xiaoming Liu Shengnan Dong Pengyun Li Xiaoqing Tang Dan Liu Masaru Kojima Qiang Huang Fellow IEEE and Tatsuo Arai Fellow IEEE 2019 IEEE RSJ International Conference on Intelligent Robots and Systems IROS Macau China November 4 8 2019 978 1 7281 4003 2 19 31 00 2019 IEEE2429 D H riban coated substrate with agarose gel film to help releasing 21 W Driesen detached the micro object and the end effector with the assistance of substrate edge 22 However those assisted releasing have the drawbacks of time consuming and poor repeatability Furthermore the assisted releasing mainly depends on the changing properties of surface and environment thus it is non universal and may cause some damage to the manipulated target especially biological living cell Many researchers have applied a variety kinds of micro robotics combined diverse manipulating method to achieve micro assembly For instance W Hu achieved assembly of glass microbeads with bubbles controlled by optically induced heating 23 D H riban achieved building microscopic subsystem by applying piezo gripper 21 T Hayakawa achieved oocyte transportation and trap using micro pillar array and vibration induced flow 24 J Chen achieved assembly of 100um microbeads to be in a line 25 In this paper in order to manipulate cells we fabricated 10 m hemispherical end effectors as Fig 1 shows This end effectors is smaller than diameter of cells so it can control the contact area Furthermore while grasping a cell the end effectors will not contact with the substrate and bring little interfaces to experiment environment due to its small size In order to grasping the cells with high stability and to control the contact area the optimal distance between the two end effectors is explored In order to simplify the manipulation procedure we coated end effector fixed on parallel mechanism with gel to make it easier adhere Then applying vibration to achieve successful releasing Finally applying image processing algorithm we achieved automatic assembly of T figure with Hela cells This paper is divided into six sections Section II introduces the configuration of the micro manipulation system Section III introduces how we fabricated suitable size of end effectors Section IV introduces grasping and releasing experiment Section introduces the assembly results In Section we come to a conclusion of the paper II SYSTEM SETUP As shown in Fig 2 the micro manipulation system mainly includes three units control unit observation unit and operation unit The components of each unit are as follows The control board is the key component of the control unit We program on PC Windows to control the board The board can generate PWM and deliver PWM to the step motor driver to achieve the movement of liner actuator Sigmakoki SGSP 13ACT B0 The liner actuator is of 13mm maximum travel and 2mm s maximum speed The X Y Z stage is formed by three this kind of liner actuators The power that liner actuator need is provided by the step motor driver The control board combined with the AD DA board can achieve AD DA convertor between piezo driver Matsusada Precision Inc PZJ 0 15P and strain gauge amplifier First the control board send the signal to the piezo driver Then the piezo driver powered the piezo actuator and make it deform Finally the piezo actuators drive the movement of the parallel mechanism The strain gauge amplifier also obtain signals from strain gauges adhere to piezo actuator and transfer signals to the control board What s more the control unit is also responsible for the communication with PC windows The observation unit contains the inverted microscopy OLYMPUS IX73 and the image acquisition and processing program on PC Windows This inverted microscopy has six slot objective turntable and its focusing distance is 10mm with 10 x obj lens The focus distance is fixed only when your Figure 2 Configuration of micro manipulation system Figure 1 Two end effectors of the microhand 2430 eyepiece and obj lens are chosen The main component of the operation unit is two fingered microhand fix on 3D microhand stage Two end effector of microhand can move together through this 3D microhand stage The right end effector is also fixed on the parallel mechanism The parallel mechanism can generate different deformation to achieve accurate control of the right end effector according to the forward kinematics the inverse kinematics and the Jacobian matrix of the mechanism Thus two end effectors can move together and end effectors fixed on the parallel mechanism can move independently in the meantime This make the grasping and releasing process to be dexterous like human finger operation What s more the cell s are also on a 3D stage for cell transportation These three units above constitute the micro manipulation system We apply this system to achieve manipulation of cells in micro scale III FABRICATE HEMISPHERICAL END EFFECTORS FOR CELL MANIPULATION This section introduces how we fabricate suitable size hemispherical end effector to achieve stable grasping As we all know cell can generate deformation In order to control the contact area within a range of cell size the end effector requires the same or a smaller scale Furthermore a big end effector will touch the substrate and brings some interfaces with its working environment The diameter of cell is only 15 m Thus as Fig 4 shows we fabricated four sizes of end effectors and did repeated grasping experiment ES means end effector size The grasping success rate is as the TABLE I shows The end effectors of 5 m diameter makes it difficult to adjust at same plane for grasping The end effectors of 15 m and 20 m diameter always brings interference not only to the liquid but also to the substrate Finally comprehensive considering the grasping stability and success rate we fabricated the end effector of 10 m diameter The fabrication processes are shown in Fig 3 Glass capillary is micropipette puller by the puller Sutter P1000 into a needle shape Next the needle shape end effectors is forged to be 10 m spherical using the microforge MF 900 Finally the spherical end effectors is polished to be hemispherical using the microgrinder EG 401 In order to make the two flat surfaces parallel to each other in the horizontal direction we need to compensate the installation angle of the end effectors Thus we set the angle between the end effectors and the polish plate to be 15 degrees After finishing the procedures above the 10 m hemispherical end effectors are obtained IV GRASPING AND RELEASING OF CELLS In order to achieve stable grasping and accurate releasing of cells firstly we coated the right end effectors effector independently movable with agarose gel to ensure the cell stick to it Then we applied vibration with suitable frequency and amplitude to release the cell in the desired position Figure 3 Fabrication processes of hemispherical end effectors Figure 4 Different size of end effectors TABLE I GRASPING SUCCESS RATE ES 5 m 10 m 15 m 20 m success rate 70 98 80 50 2431 However we found that the releasing success rate is greatly influenced by grasping distance between two end effectors We focused on optimizing its suitable distance to ensure the stable grasping and the easy releasing A Procedure for grasping cells inspired by human grasping Concerning the grasping the operation of two fingered microhand is similar to the grasping skill of the human s thumb and forefinger As Fig 5 a shows the two end effectors are directed together to approach to the cell and to increase the distance between two fingers Then as shown in Fig 5 b we make the cell more close to the left end effector Finally as shown in Fig 5 c the distance between the two fingers is decreased to achieve stable grasping The hemispherical shape end effector increases the area of contact surface between the cell and the end effectors so the grasping process become more stable We can achieve 98 of grasping success rate As Fig 6 shows we can set different grasping distances This will cause a critical influence on the releasing We will discuss this releasing problem in part B B Releasing to overcome adhesion force After coating agarose gel on the contact surface of the right Figure 8 Image processing steps Figure 5 Grasping processes Figure 7 Vibration based releasing Figure 6 Different grasping distance 2432 finger which is fixed on the parallel mechanism the cell always stick to this end effector We focus on the grasping distance which will influences releasing As Fig 6 shows GD means grasping distance TABLE shows the transport success rate in different grasping distance Considering success rate and keep cell figure finally we set the grasping distance to be 13 m As shown in Fig 7 we take the advantage of parallel mechanism to generate vibration in x direction In our cell releasing and placing strategy first the vibration is applied then the fingers open to release the grasped cell stably and to place it accurately at the target location By changing the frequency and amplitude of vibration we found that with 25Hz frequency and 5 m amplitude exact cell releasing with good success rate of 100 When we decrease frequency and amplitude we can seldom achieve release When we increase frequency and or amplitude beyond these values the accuracy will decrease So in our strategy 25Hz frequency and 5 m amplitude vibration is adopted Then with the thorough grasping and releasing strategy we finally achieve assembly work V CELL ASSEMBLY This section mainly introduces the experiment of cell assembly Applying the grasping and releasing method mentioned above and combining with visual servo algorithm we achieved automatic cell assembly A Experiment preparation Before starting the experiment suitable environment are required to be installed The Hela cell is adopted whose average diameter is 15 m The cells are dealt in the cell culture fluid in order to prolong the cell survival time during manipulation The temperature is 25 B Cells Detection In order to achieve automatic assembly of cells we need to acquire the position information of the cells Traditional Hough circle detection cannot detect all of the cells because some cells do not have perfect circle contour Thus our method applies our customized image processing to achieve more perfect cell detection The main step for detection is as shown in Fig 8 Firstly the basic image processing algorithms such as Gaussian blur morphologically operation and contour detection are applied Then the contour lines are detected and the minimum circles of contour are fitted Finally if the area is within the range of the cell area the centers of circumcircle are computed as the coordinates of cells Applying the method introduced above all of the cell positions can be obtained within the microscopic view This section is a crucial part for automatic assembly C Assembly of cells After the detection of cells the assembly task is carried out The control flow chart for the cell assembly is shown in Fig 9 Based on the grasping releasing method and the position information of cells we could achieve the assembly of cells one by one The strategy we use is to assemble from right to left and from up to down As our excellent demonstration we can show the achieved assembly of BIT figure our school name Although the positioning accuracy in this assembly is 4 0 5 m every character is well readable It takes 20s for one cell assembly TABLE TRANSPORT SUCCESS RATE GD 14 m 13 m 12 m 11 m Transport success rate 80 95 95 95 Figure 9 Control flow chart for automatic assembly of single cells Figure 10 Assembly with single cells 2433 VI CONCLUSION In this paper the main challenge we overcome in our automated cell assembly are the grasping problem due to the small size of cells and the releasing problem caused by the surface adhesion In order to solve the grasping problem we fabricate 10 m diameter hemispherical end effectors to achieve stable grasp and to ensure the appropriate operation environment We coated the end effector fixed on the parallel mechanism with agarose gel to ensure stable contact of the cell We