外文翻译--果树采摘机器人及控制系统研制【中英文文献译文】
收藏
资源目录
压缩包内文档预览：
编号：30435747
类型：共享资源
大小：1.99MB
格式：ZIP
上传时间：2019-12-12
上传人：好资料QQ****51605
认证信息
个人认证
孙**（实名认证）
江苏
IP属地：江苏
12
积分
- 关 键 词：
-
中英文文献译文
外文
翻译
果树
采摘
机器人
控制系统
研制
中英文
文献
译文
- 资源描述：
-
外文翻译--果树采摘机器人及控制系统研制【中英文文献译文】,中英文文献译文,外文,翻译,果树,采摘,机器人,控制系统,研制,中英文,文献,译文
- 内容简介：
-
果树采摘机器人及控制系统研制摘要机器设备组成的操纵者,效应器和基于图像的视觉伺服控制系统是为了采摘苹果而开发的。5自由度的机械手PRRRP结构几何优化提供准线性行为和简化控制策略。匙形效应器与气动驱动爪为了满足采摘苹果的要求。苹果采摘机器人自主完成收集任务使用一个应用模块。通过使用支持向量机与径向基函数,果实识别算法开发了检测并自动查找苹果在树上。控制系统,包括工业电脑和交流伺服驱动程序,进行了机械手和效应器融合、将苹果摘下来的。原型机器人装置的有效性经实验室测试和现场实验的领域。机器人采摘苹果的成功率是77%,平均采摘时间大约是15秒/苹果。1 序言在中国,农村经济的快速发展和水果种植领域结构的不断调整,如苹果、柑橘和梨自1993年以来达到8 - 9百万公顷,占四分之一世界上的水果种植总面积。然而,水果收集任务,以50%-70%/小时,仍然依靠体力劳动。所以收获将自动化,因为农业人口在中国逐渐减少。此外,由于果树高,采摘工作已经将使用梯子,这使得手工收割十分危险和低效。因此,未来需要对苹果采摘进入机械化和自动化。机械采摘实验在一些地区已经进行了假设模拟收获,但开发这一战略还没有广泛。有选择性的采摘,这是必要的,需要复杂的机器人技术。总之,有必要设计一个与人类感知能力相似的智能机器人。以下这个为实例,这台机器需要检测水果、计算水果的位置,然后选择不破坏果皮或果树，从而进行采摘。研究水果收获机器人发生在1980年代。河村建夫、Namikawa Fujiura,Ura所言(1984)首次开发了一个果园机器人。后来, Rabatel,Pellenc、Journeau Aldon(1987)开发了一个机器人。从那时起,他们关于这方面的知识进行了开创性的研究。此外,一些相关的研究农业机器人在温室进行了。例如,番茄收获,黄瓜收获,樱桃收获,草莓收获。然而,大多数水果收获机器人的文献目前没有进入生产或销售。相反,他们仍然处于研究发展阶段。为此,支持进一步的研究和开发是很重要的,以提高性能和减少这些机器人的初始安装费用。基于上述概念,本研究打算开发和评估竞争低价设备自动收获,即,一个苹果采摘机器人。首先,一个完整的机器人包括组件描述的操纵者,效应器和基于图像的视觉伺服控制的系统描述。其次,机械手的几何优化以获得准线性行为并简化描述的控制策略。第三,气动驱动的结构爪设计来满足采摘苹果的要求。基于这个设计,采摘机器人的自动执行部分使用一个应用模块来检测和采集任务找到苹果在树上,控制系统进行机械手和效应器方法来摘苹果。研究采摘机器人,来有效的验证实验室测试和现场实验测试的领域执行。实验结果是本文的重要贡献。本文组织如下:例如在第二节中,机械手,效应器和基于图像的视觉伺服控制系统。在第三节中讨论实验结果显示的可行性机器人系统的提出。最后,在第四节中总结对机器人未来研究的建议。2 材料和方法21 苹果采摘机器人的机械结构苹果采摘机器人的原型主要考虑的是模型设计效率和成本效益。它主要由自主车,5自由度机械手,效应器,传感器、视觉系统和控制系统组成。自主研发水果采摘的机械结构如图1所示。1.移动车辆 2.水果篮子 3.活动带 4.末端效应器 5.采集单元6. 电动绞盘棒 7.运动小手臂 8.大臂 9.运动大手臂 10.活动的腰部11.腰部 12. 升降平台 13.中心控制系统图1 苹果采摘机器人的原理图2.1.1 自主移动车辆车辆的移动方式为履带式移动。气动泵供应电力，电子硬件数据采集和控制,机械手的效应器切水果。全球定位系统(GPS)技术用于自治导航的移动车辆,其典型的速度1.5m/s。2.1.2 机械手与其他结构相比 ,联合结构是有效的三维位置和方向空间。采摘机器人是一个随机的操作空间分布,可能存在很多障碍机器人。联合操纵与自由的多具有任意曲线拟合功能。因此很容易躲开障碍,操作时相应的关节效应器到达指定的位置。因此,采摘机器人与五自由度机械手 (PRRRP)结构安装在自主移动车辆设计。第一个自由度是用于提出整个机械手。Xo、Xc、Yc为摄像机坐标系的轴。L1,L2,L3为机器人的腰,大臂和手臂。q1、q2为第三关节的腰和手臂。水平垂直。U0为平面坐标轴,中心坐标xg,yg，投影中心用来协调目标,按照目标苹果的图像特性的差异来进行辨别，要采摘的目标就会出现在显示器上。主要的直臂k1,k2武器的控制参数Dd，根据运动度的像素单元就会调整一个合适的角度。交流模拟信号A / D、数字CCD电荷装置，D /数字模拟直流电流。要求自由度GPS全球定位系统目标的的颜色,强度、饱和IBVS基于图像的视觉伺服，PBVS定位视觉伺服PRRRP棱镜的转动 ，从而进行判断。中间三个自由度的旋转,其中,第二个驱动臂设计旋转腰部,第三和第四的转动轴移动终端上下操作符。这个自由度允许效应器朝任意方向移动。最后,手臂的灵活用于伸长, 机器人控制命令会根据实测值的误差达到目标位置,从而实现苹果的采摘。上面的讨论表明,五自由度机械手的设计应足以执行苹果采摘的操作。机械手的机械结构如图2所示。机械手的升降是通过升降平台来进行的,能够应付特殊情况下苹果的采摘。旋转接头和灵活的关节是由伺服电机驱动的。机器人机械手的运动参数和机械结构如表一所示。 图2 机械手的照片2.1.3 实测值的误差效应器机制是由生物学特性来操作目标对象的。苹果采摘机器人的操作对象主要是像苹果一样的球形。匙形效应器(图3所示)根据球面生物学特性而设计的水果,通过切断茎来实现的。图3 末端效应器的图片效应器包含以下部分:钳子用于掌握水果，电动切割装置将苹果从枝桠上切断。效应器的打开和关闭是由一些适合快速行动的气动设备,快速响应特征效应器的开关控制。转移模式实现能量传递的动力是使用压缩气体压力传输。苹果根茎是安装在夹具机制的电动刀切断的。苹果被抓住时,电源通过直流电使得爪刀旋转,切断效应器感应到的目标。2.2 传感器非结构性和不确定的操作特性，个体差异和随机的操作对象的不同,使得苹果采摘机器人应该足够应对复杂的环境。在水果夹紧过程中,水果的生物学特性,包括果皮的薄和脆弱需要效应器的高度掌控。它要求传感器来控制高精度的把握力。此外,手臂的旋转,目标的位置和准确捕获还需要经过传感器来检测和定位。此外,为了避免损坏设备,导致受伤,未能确定水果、避免手臂的碰撞也需要传感器有效的感知操作环境。2.2.1 效应器上的传感器传感器的上的效应器,包括摄像头传感器、位置传感器、碰撞传感器和压力传感器。如图4所示。视觉传感器,它使用高分辨率的电荷耦合装置(CCD)摄像机，采集系统通过串行总线(USB)接口的视频窗口捕获技术形成图像, 在完成图像采集、水果搜索和认可扮演一个重要角色。获得广泛应用。视觉传感器的位置是在一个眼手并用的模式。在图4中可以看出的两对红外双光电位置传感器光电电池。此外, 传感器的开关位置通常是用来限制安装在手臂上的电切刀。手臂开始减速时,效应器通过视觉传感器上传来的采摘对象的图像。手臂停了下来,钳子夹水果当蓄电池的两双都是模糊的。在这一点上,压力和碰撞传感器采用力敏感电阻。当压力爪感到一定的压力传感器,电动切割器旋转和切断花梗。开关位置传感器便会操作电切刀停止工作。碰撞传感器用于采摘过程中躲避障碍物。模拟信号来自力敏感电阻和红外线光电管，在工业计算机与数据采集模块通常是不相容的。因此,他们之间需要调制传输之前数据采集模拟信号。图5显示了传感器信号调制电路。图4 末端传感器的布局 图5 传感器信号的调制电路2.2.2 传感器为避免操作的碰撞控制旋转关节的角度和位置用来控制关节的灵活度。霍尔传感器安装在腰的转动关节,主要负责小手臂关节的两端。在工作环境中,小手臂的运动空间大。其中位置传感器，压力传感器，图像传感器，碰撞传感器图，效应器的传感器。图5 为信号调制电路。2.3 视觉系统苹果采摘机器人视觉系统的关键成分是公认的苹果图像处理方法。它影响机器人的可靠性和直接决定的能力,快速、准确进行水果识别。然而,在早期的研究中,存在一些尚未解决的问题，低准确率和时间消耗等在一定程度上限制了苹果采摘机器人在自然环境中的实时和多任务处理能力。为了克服这些缺点,运用识别视觉系统组成的彩色CCD相机自动获取原始的苹果图像和一个工业计算机处理图像识别定位水果。因为富士苹果在中国是最受欢迎的,我们研究集中这种多样性。识别和定位过程如下。首先,由于自然环境的形象采集设备使用,原始的未经加工的苹果图像不可避免的包含噪声,影响其质量。一个向量中值滤波器应用于图像增强预处理。它不仅可以有效地去除噪声突显出苹果果实的前景,也能保持良好的图像边缘。其次,大多数的苹果采摘在自然图像条件下通常包括树枝和树叶让问题变得更加复杂。只通过传统的形象分割算法,很难达到预期的效果。基于颜色直方图色调的统计,强度饱和模型,图像分割算法用来开发双阈值和区域增长，从复杂的背景识别苹果果实。色度组件是适用于提取轻色调的苹果,这避免了不同的照明水平对图像的影响。这种算法简单,所需的处理时间简短。苹果提取通过不同的特征来确定空间位置,并提供手臂相应的运动参数。对于颜色特征提取,色度组件色调和饱和度,通常与颜色特征提取识别不同。然而,在我们的研究中,苹果果实、树枝和叶有特定形状,及其不同的形状是巨大的。因此,苹果的形状特性对于对象识别是非常重要。形状特征的选择规则基于旋转不变性和规模。考虑到苹果果实图像的特点,圆形的方差,方差椭圆,圆周边比率和正方形区域是用来描述苹果的形状特征轮廓。这四个特征向量与形状特征提取。对应的特征值的计算后,他们作为每个样本的特征向量,用于训练和分类。最后,基于支持向量的分类算法并建立了辨识苹果方法。仿真和实验表明, 基于苹果的颜色特征和形状特征，支持向量机方法和径向基函数(RBF)的内核是对于苹果被发现最好的认可。Research PaperDesign and control of an apple harvesting robotZhao De-An, Lv Jidong, Ji Wei*, Zhang Ying, Chen YuSchool of Electrical and Information Engineering, Jiangsu University, XueFu Road No.301, Zhenjiang, Jiangsu Province 212013, PR Chinaa r t i c l e i n f oArticle history:Received 9 February 2011Received in revised form4 July 2011Accepted 17 July 2011Published online 6 August 2011A robotic device consisting of a manipulator, end-effector and image-based vision servocontrol system was developed for harvesting apple. The manipulator with 5 DOF PRRRPstructure was geometrically optimised to provide quasi-linear behaviour and to simplify thecontrol strategy. The spoon-shaped end-effector with the pneumatic actuated gripper wasdesigned to satisfy the requirements for harvesting apple. The harvesting robot autono-mouslyperformeditsharvestingtaskusingavision-basedmodule.Byusingasupportvectormachine with radial basis function, the fruit recognition algorithm was developed to detectand locate the apple in the trees automatically. The control system, including industrialcomputer and AC servo driver, conducted the manipulator and the end-effector as itapproached and picked the apples. The effectiveness of the prototype robot device wasconfirmed by laboratory tests and field experiments in an open field. The success rate ofappleharvestingwas77%,andtheaverageharvestingtimewasapproximately15sperapple.Crown Copyright 2011 Published by Elsevier Ltd on behalf of IAgrE. All rights reserved.1.IntroductionIn China, with the rapid development of the rural economyand the continuous adjustment of planting structures, fruitcultivation areas, such as apple, citrus and pear, havereached 8-9 million ha since 1993, accounting for one-quarterof the total fruit cultivation area in the world. However, fruitharvesting tasks, which take 50%e70% of the total workinghours, still depend on manual labour (Xu & Zhang, 2004.Harvesting is expected to be automated because the farmingpopulation is gradually decreasing in China. In addition,since the fruit trees are tall, harvesting work has to be con-ducted using step ladders, which makes manual harvestingdangerous and inefficient. Therefore, there is a strong desireto mechanise and automate harvesting. Mechanical har-vestingexperimentshavebeenperformedontheassumptionof once-over harvesting in some areas, but exploitation ofthis strategy is not yet widespread (Hancock, 1999). Selectiveharvesting, which is commonly used, requires sophisticatedrobotic technology. In short, it is necessary to design anintelligent robot with human-like perceptive capabilities. Forinstance, the machine needs to detect fruit, calculate theposition of the fruit and then pick it without damaging thepericarp or the fruit tree.Research on fruit harvesting robots took place in the 1980s.Kawamura, Namikawa, Fujiura, and Ura (1984) first developeda fruit-harvesting robot for orchards. Later, Grand, Rabatel,Pellenc, Journeau, and Aldon (1987), developed an apple-harvesting robot. Since then, their pioneering studies werefollowed by many research papers covering several aspects(e.g., ;Edan, Rogozin, Flash, & Miles, 2000; Foglia & Reina, 2006;Hwang&Kim,2003;Kondo&Ting,1998;Muscato,Prestifilippo, Abbate, & Ivan, 2005; Sakai, Osuka, Maekawa, &Umeda, 2007, 2008; Sarig, 1993; Van Henten, Hemming, VanTuijl, Kornet, Meuleman, 2002). In addition, several relevantstudiesonagriculturalrobotsingreenhouseshavebeencarried* Corresponding author. Tel.: 86 511 82028322; fax: 86 511 82028322.E-mail address: (J. Wei).Available at journal homepage: /locate/issn/15375110biosystems engineering 110 (2011) 112e1221537-5110/$ e see front matter Crown Copyright 2011 Published by Elsevier Ltd on behalf of IAgrE. All rights reserved.doi:10.1016/j.biosystemseng.2011.07.005out; for instance, tomato harvesting (Monta et al., 1998),cucumber harvesting (Van Henten, Van Tuijl, Hemming,Kornet,Bontsema&VanOs,2003),cherryharvesting(Tanigaki, Fujiura, Akase, & Imagawa, 2008), strawberry har-vesting (Hayashi et al., 2010). However, most of the fruit har-vesting robots discussed in the literature are not currentlymanufacturedorsold.Instead,theyremainintheresearchanddevelopment stages. To this end, it is important to supportfurtherresearch anddevelopment toimprove the performanceand reduce the initial set-up costs of these robots.Based on the concepts above, this study intends to developand evaluate a competitive low price device for automaticharvesting, i.e., an apple-harvesting robot. Firstly, a detaileddescription on the components of the robot including themanipulator,theend-effectorandtheimage-basedvisionservocontrol system is described. Secondly, the geometrically opti-misation of the manipulator to gain a quasi-linear behaviourand simplify the control strategy is described. Thirdly, the end-effectorwiththepneumaticactuatedgripperdesignedtosatisfythe requirements for harvesting apple is described. Based onthis design, the harvesting robot autonomously performs itsharvesting task using a vision-based module to detect andlocate the apple in the trees, and control system conducts themanipulator and the end-effector to approach and pick apple.To verify the validity of the developed harvesting robot, thelaboratory tests and field experiments in an open field wereperformed. The experimental results are the important contri-bution of this paper.The paper is organised as follows: in section 2 the maincomponents of the robot are presented in detail, i.e., themanipulator, the end-effector and the image-based visionservo control system, respectively; in section 3 the experi-mental results are discussed to show the feasibility of therobot system proposed; finally, in section 4 conclusions aredrawn and suggestions for future research are made.2.Material and methods2.1.Mechanical structure of apple harvesting robotA prototype model of the apple harvesting robot is designedfor both efficiency and cost effectiveness. It mainly consists ofan autonomous vehicle, a 5 degree of freedom (DOF) manip-ulator, an end-effector, the sensors, the vision system andcontrol system. The mechanical structure of fruit harvestingrobot self-developed in this paper is shown in Fig. .The autonomous mobile vehicleA crawler type mobile platform was selected as the mobilevehicle. It carried the power supplies, pneumatic pump,electronic hardware for data acquisition and control, and themanipulator with the end-effector for cutting the fruit. Globalposition system (GPS) technology was used for autonomousnavigation of the mobile vehicle, whose typical speed was1.5 ms?.The manipulatorCompared with other structures, as described in Sakai,Michihisa, Osuka, and Umeda (2008), joint structure is effec-tive for any position and orientation in three-dimensionalspace. The operation of a harvesting robot is a random largespace distribution, where a lot of obstacles may exist aroundthe robot. A joint manipulator with multi-degrees of freedomhas an arbitrary curve fitting function. It is therefore easy toavoid obstacles by operating the corresponding joints whenthe end-effector reaches the object position. Therefore,aharvestingrobotmanipulatorwith5DOFprismatic-revolute-revolute-revolute-prismatic (PRRRP) structure to bemounted on autonomous mobile vehicle was designed. Thefirst DOF was used for uplifting the whole manipulator. TheNomenclatureSymbolsCR, CRU, CU, CLU, CLAvoidance sensors numberXc, Yc, ZcThe camera coordinates axesXo, Yo, ZoRobot coordinates axesL1, L2, L3Lengths of waist, major arm and minor armq1;q2;q3Joint angles of waist, major arm and minor arm.u, vImage plane coordinates horizontal and verticalaxesuo, voImage centre coordinatexg, ygProjection centre coordinate of target fruitex, eyThe difference of target fruit image featurebetween xg, ygand uo, voM ? NImage plane pixels of video camerajexmaxj;jeymaxj Maximum of ex and eyDq1; Dq2; Dq3Joint deviationangles of waist, major armandminor armk1, k2Control parameters of armsDd The angle to adjust for the movement of a pixel withunit of degree per pixel.AbbreviationsACAlternating CurrentA/DAnalog, DigitalCCDCharge Coupled DevicesD/ADigital, AnalogDCDirect Current.DOFDegree of FreedomGPSGlobal Position SystemHISHue, Intensity, SaturationIBVSImage-Based Vision ServoPBVSPosition-Based Vision ServoPRRRPPrismatic Revolute Revolute Revolute PrismaticRBFRadial Basis FunctionRSTRotation Scale, TranslationSVMSupport Vector MachineUSBUniversal Serial BusVFWVideo for Windowsbiosystems engineering 110 (2011) 112e122113middle three DOF were for rotation, among which, the seconddriving arm was designed to rotate around the waist, and thethird and fourth ones were rotation axes to move the terminaloperator up and down. This DOF allowed the end-effector tomove towards an arbitrary direction in the work space. Thefifth, and last, DOF was flexible and used for elongation, whichmade the end-effector reach the target location according tothe robot control commands, thus achieving the harvesting offruit (Zhao, Zhao, & Ji, 2009; Zhao, Zhao, & Shen, 2009). Thediscussion above shows that 5 DOF manipulator designedshould be sufficient to perform the harvest operation. Themechanical structure of the manipulator is shown in Fig. 2.The lifting of manipulator was performed by the pump-driven lifting platform, which was able to cope with thespecial circumstances of tall fruit crops. The rotary joints andflexible joints were driven by servo motors. Motion parame-ters of the robot manipulator mechanical structure are shownin Table .The end-effectorThe mechanism of end-effector is determined by operationand biological characteristics of the target object. The opera-tion objects of harvesting robot are mainly spherical fruit suchas apple. A spoon-shaped end-effector (shown in Fig. 3) isdesigned according to biological characteristics of sphericalfruit, which are picked by means of cutting off the stalk.The end-effector contained the following parts: a gripper tograsp the fruit and an electric cutting device to separate thefruit from the stalk. The opening and closing of end-effectorgripper was determined by some pneumatic devices, whosequick action, fast response characteristics were suitable forthe switching control of the end-effector. Pressure trans-mission was a transferring mode using compressed gaspressure to achieve energy transference. The apple stalk wassevered by an electric cutter installed in the side of grippermechanism. When the fruit was grasped, the direct current(DC) motors transmited power by flexible wire to drive thecutter rotating around the gripper,cutting off the stalk in frontof end-effector at any position.Fig. 1 e Schematic diagram of the fruit harvesting robot.Fig. 2 e Photograph of the manipulator.biosystems engineering 110 (2011) 112e1221142.2.The sensorsThe non-structural and uncertain features of the operatingenvironment, and the individual differences and randomnature of the operating objects, determines that fruit har-vesting robots should have intelligent sensibility to theircomplex environment (Edan et al., 2000; Zhao, Zhao, & Ji, 2009;Zhao, Zhao, & Shen, 2009). During the process of clamping thefruit, the biological characteristics of fruit including its thinand fragile pericarp put a high demand on grasping force ofend-effector (Monta, 1998). It required sensors to control thegrasping force accurately. In addition, the rotation of arm, itstraveling position and accurate capture also required thesensors to detect and locate fruit (Jiang, Cai, & Liu, 2005; Qiao,Wu, & Zhu, 1999). Furthermore, in order to avoid damagingequipment, causing injury and failing to pick fruit, collisionavoidance of the arm also needs sensors to perceive theoperating environment effectively.2.2.1.The sensors on end-effectorThe layout of sensors on end-effector, which includes a visionsensor, a position sensor, a collision sensor and a pressuresensor, is shown in Fig. 4. The vision sensor, which uses high-pixel colour charge coupled devices (CCD) video camera withuniversal serial bus (USB) interface and the video for windows(VFW) capture technology to form image acquisition system,plays a key role in completing image acquisition, fruit searchand recognition. To obtain a wide visible-field and not influ-encedby end-effector, the position of the vision sensoris in aneye-in-hand mode. In Fig. 4, it can be seen that there is thephotoelectricpositionsensorwithtwo pairsofinfrareddoublephotoelectric cells. In addition, the switch position sensorwhich was usually used to limit for electric cutting knife wasalso mounted on the position sensor. The arm began decel-eration when the end-effector moved towards the target fruitguided by the vision sensor and the first pair of photodiodeswas obscured by the fruit in the holder. The arm stopped andthe gripper clamped fruit when the two pairs of photocellswere obscured. At this point, both the pressure and collisionsensors adoptedforce sensitive resistance. When the pressuresensor on the gripper felt a certain pressure, the electric cutterrotated and cuts off pedicel. The cutter stopped working whenthe switch position sensor operated. The collision sensor wasused for obstacle avoidance during the process of harvesting.Analogue signals derived from the force sensitive resistanceand infrared photoelectric tubes are usually incompatiblewith the data acquisition module inside industrial computer.Therefore, they require modulation before transmission to thedata acquisition module. Fig. 5 shows the sensors signalmodulation circuit.2.2.2.The Sensor on manipulator for collision avoidanceControl of the angle of the rotating joints and position controlof the flexible joints was fulfilled using 8 Hall sensors,installed on the rotation joints of waist, the major arm, theminor arm and both ends of flexible joints. In the workingenvironment, the movement space of minor arm was wide;Fig. 3 e Photograph of the end-effector.PositionSensorPressureSensorVisionSensorCollisionSensorFig. 4 e Layout of sensors on end-effector.Fig. 5 e Sensors signal modulation circuit.Table 1 e Motion parameters of manipulator mechanicalstructure.Joint MotionparametersLift platform0 me0.8 mRotation joint of waist?180?e180?Rotation joint of major arm?80?e80?Rotation joint of minor arm?80?e80?Flexible joint0 me0.8 mbiosystems engineering 110 (2011) 112e122115and the probability of collision with obstacles was high.Therefore, the collision sensor was fixed in the minor arm todetect obstacles. Five groups of micro switches were fixed ondifferent positions in the minor arm to obtain real-timeinformation from obstacles. Noting that software program-ming processes signals conveniently, the five groups ofavoidance sensors were designated CR、CRU、CU、CLU、CLin accordance with their position. The distribution of theminor arm collision avoidance sensors is shown in Fig. 6.2.3.The vision systemsFor the vision system of the apple harvesting robot, the keyingredient was the image processing method that recognisedand located the fruit. It affects the robots dependability andalso determines its ability to directly, quickly and accuratelyrecognise in the fruit real time (Bulanon, Kataoka, & Okamoto,2004). However, in the earlier research (Bulanon, Kataoka, &Ota, 2002; Liu, Zhang, & Yang, 2008; Plebe & Grasso, 2001;Zhao, Yang, & Liu, 2004), there exist some unsolved issuessuch as low accuracy rate and time consumption, which tosome extent restricted the real-time and multitasking abilityof the apple harvesting robot in the natural environment.To overcome these shortcomings, a real-time automaticrecognition vision system consisting of a colour CCD cameraforcapturingoriginalappleimagesandanindustrialcomputer for processing images to recognise and locate thefruit was developed. Since the Fuji apples are the mostpopular in China, our research focused on this variety.The recognition and location procedure is as follows.Firstly, due to the natural environment and the imageacquisition device used, the original unprocessed apple imageinevitably includes noise that influences its quality. A vectormedianfilter wasappliedtoimageenhancementpre-processing. It can not only remove noise effectively andhighlights the apple fruit in foreground, but it also maintainsgood image edges.Secondly, most apple images acquired in the naturalconditionsusuallyincludebranchesandleaveswhichcomplicate matters. By using only a conventional imagesegmentation algorithm, it was difficult to achieve anticipatedeffect. Based on hue histogram statistics from the hue,intensity and saturation (HIS) model, the double thresholdand region growing method was employed to develop animage segmentation algorithm for identifying apple fruitfrom complex background. The chromaticity component isirrelevant when lightness is extracted and this avoided theinfluence of different illumination levels on the images. Thealgorithm was simple, and required little processing time.The apple features were extracted to determine the spatiallocation, and provide corresponding motion parameters forarm. For colour feature extraction, the chroma componentshue and saturation, are usually extracted as colour featuresfor recognition. However, in our study, apple fruit, branchesand leaves havespecific shapes, and their differences in shapeare large. Therefore, the shape feature is important in appleobject recognition. The selected rule of shape features wasbased on invariance in rotation, scale and translation (RST).Taking account of characteristics of apple fruit images,circular variance, variance ellipse, tightness, ratio betweenperimeter and square area were used to describe the outlineshape features of apple. These four feature vectors wereextracted as shape features. After the calculation of the cor-responding eigenvalues, they were used as feature vectors ofeach sample and used for training and classification.Finally, a new classification algorithm based on supportvector machine was constructed to recognise the apple fruit.Simulation and experiment shows that the support vectormachine (SVM) method with radial basis function (RBF) kernelfunction based on both colour features and shape featureswas found to be the best for apple recognition. Details of thealgorithm can be found in Wang, Zhao, Ji, Tu, and Zhang(2009).2.4.The control systemThe hardware structure is shown in Fig. 7. At the centre of thecontrol system was the host computer, which integrates thecontrol interface and all of software modules to control thewhole system. The sensor signal acquisition system andimage acquisition system constituted the input section whichwas used to collect external environment information for theFig. 6 e Layout of sensors on minor arm.Servodrivers14Incrementalphotoelectricencoder14Cutter of theend-ffectorUSB interfacePosition limitedsensorDrive motorfor cutterAirpumpCollision sensorInfrared sensorCCD Vision sensorElectricvalveSignalmodulationcircuitGripper of theend-ffectorAC Servomotors andload joints14Data acquisition moduleHost control computerRS232/RS422convertersFig. 7 e Hardware structure of apple harvesting robotcontrol system.biosystems engineering 110 (2011) 112e122116harvesting robot. The output section included a servo drivenmotor, air pump and end-effector.2.4.1.Host computerA Kintek KP-6420i (Kintek Electronics Co., Ltd., Miaoli Hsien,Taiwan, China) industrial computer with Intel Pentium41.7 GHz processor and 512 M memory was selected as the hostcontrol computer, which was responsible for collecting wholesensor signals, processing images online, calculating theinverse kinematics of manipulator and completing the controlalgorithm. The host computer transmitted instructions to thealternating current (AC) servo driver through a serial port tocontrol the joint motors of waist and arms. HighTek HK-5108(Shenzhen FangXingLiuTong Industrial Co., Ltd., Shenzhen,China) RS-232/RS-422 converters were chosen for serialcommunicationfunctions.Adataacquisitionmoduleinstalled inside host computer was responsible for datacollection from all the sensors except for the vision sensorandoutput control of the electrical cutter. A KPCI-847H (BeijingKeRuiXingYe Technology Co., Ltd., Beijing,China)module with16 channels A/D and D/A converter was used.Fig. 8 e Communication link diagram of the host computer and servo drivers.Fig. 9 e Geometrical relations of manipulator joints.Fig. 10 e Perspective projection diagram of fruit in 3-Dspace.biosystems engineering 110 (2011) 112e1221172.4.2.Servo DriveMechanismThe purpose of motion control was to guarantee that the fruitharvesting robot achieved movement with an arbitrary angleto grab the object fruit accurately and rapidly. Therefore thekey issue was to control the drive motors of each joint. Theclosed-loop servo system used to control rotary joints andflexible joints, consisted of Delta ASDA-AB (Delta Electronics,Inc, Taipei, Taiwan, China) series servo drivers, ECMA (DeltaElectronics, Inc, Taipei, Taiwan, China) series AC servo motorsand incremental photoelectrical encoders, which included theplanetary PH (Hubei Planetary Gearboxes Co., Ltd., Wuhai,China) series reduction gear and the Danaher IDC EC2(DanaherMontion Co.,Ltd., Petaluma, Cal., USA)series preciselinear electric actuators. To improve the safety of operation,an electromagnetic brake was fitted to each motor. The ASDA-AB series servo driver not only has three control modesincluding position control, speed control and torque control,but it also included serial communication functions for RS-485, RS-232 and RS-422. Considering the practical applica-tions of the system, the rotation joints for the waist, majorarm and minor arm employed a position control mode, whilstthe flexible joint of the electric pusher employed a speedcontrol mode. The communication between host computerand servo drivers of all the joint motors employed the RS-422mode. Therefore, the frame computer not only set theparameters and adjusted the gains, but it also monitored theoperating state of the servo driver and alarm conditions. Thecommunication link diagram for the host computer and servodrivers is shown in Fig. .Manipulator control strategyThefruitharvestingrobothadanintegratedsystem,comprisingenvironmentperception,dynamicdecision-making and behaviour control. Motion control was the mostbasic and important ingredient. The robot vision servo controlincluded two methods, a position-based vision servo (PBVS)and image-based vision servo (IBVS) (Lippiello, Siciliano, &Villani, 2007; Mariottini, Oriolo & Prattichizzo, 2007). IBVSwas usually used to control the manipulator according toimage features and separate vision reconstruction problemsfrom robot control. This method simplifies robot control andavoids targets outside the camera visible-field. Thus, it isa commonly usedas a methodforrobot control. Tothis end, inour harvesting robot manipulator control system, the IBVScontrol method was employed to achieve location and thepicking motion for the target fruit.The structural diagram for fruit harvesting robot manipu-lator is shown in Fig. 2, where the installed CCD camera is inan eye-in-hand mode. At the start of the picking process, the(u0, v0)uvexey(xg, yg)OFig. 11 e Location diagram of target fruit in image plane.Fig. 12 e Flowchart of small step transformation algorithm.biosystems engineering 110 (2011) 112e122118flexible joint contracted in the minor arm during the processof searching for target fruit. Therefore, the harvesting robotmanipulator can be regarded as a three-joint robot manipu-lator, and the relationship between camera coordinate systemand robot coordinate system can be obtained according togeometrical relation shown in Fig. 9. The camera coordinatesaxes (Xc, Yc, Zc) parallel to corresponding axes in robot coor-dinates (Xo, Yo, Zo). L1, L2, L3are the lengths of the waist, majorarm and minor arm respectively, and q1;q2;q3are the jointangles of the second, third and fourth DOF.Apples with radius of 40 mm (average radius of the apples)were considered as research objectives. Their projection wasa circle on the image captured by video camera. Perspectiveprojection of a fruit in 3-D space is shown in Fig. 10, andformed in the video camera. Feature information of targetfruit in image plane is shown in Fig. 11. For a two-dimensionalimage captured by a video camera, the origin is a point in theupper right corner. Symbols of u and v denote horizontal andvertical axes respectively. The image feature of target fruit ischaracterize as ex and ey, which are the errors betweenprojection centre coordinate (xg, yg) and image centre coordi-nate (uo, vo). During joint control of harvesting robot manip-ulator, image feature of ex varies along with the change ofwaist joint angle q1, and image feature of ey varies along withthe change of major arm joint angles q2and minor arm jointangles q3.It can be seen that the manipulator with 5 DOF PRRRPmechanical structure was geometrically optimised to simplifythe control strategy, and the control algorithm designed toavoid complicated jacobian operations. At the same time, thevision systems software gave only planar information of thetarget fruit in our robotic system. The distance informationbetween target fruit and camera was unknown. Hence themanipulator jacobiancould not bedirectlyused in our system.The process of picking target fruit can be presented asfollows. Firstly, each module of harvesting robot was ini-tialised, and the manipulator made to approach the fruit treesat a proper location. Then the video obtained image infor-mation of target fruit, and the recognition and location wereobtained by image processing software such that the centroidcoordinate xg, ygof target in image and the errors ex and eyobtained by comparison with the image centre coordinate uoand vo.Secondly, the robot was controlled to move with small stepaccording to the calculated deviations ex and ey, and eventu-ally it drove them to be zero. Assuming that image planepixels of video camera are M ? N, then jexmaxj M=2 andjeymaxj N=2. The flowchart of the small step transformationalgorithm is shown in Fig. 12. When the deviations of thesmall step movements of the waist, major arm and minor armwere zero, then the centroid of target fruit was coincidentwith image centre. During the process of eliminating devia-tions ex and ey, each joint angle was required to move. Thiswas calculated according to Eq. (1)Dq1 ex ? DdDq2 k1? ey ? DdDq3 k2? ey ? Dd(1)where Dq1; Dq2; Dq3are joint angles of waist, major arm andminor arm respectively; k1,k2are the control parameters ofarms; Dd is the angle to be adjusted for the movement ofa pixel with unit of degree per pixel.Then, the host computer sent instructions to the flexiblejoint to spread.Afterthe objectfruit enteredinto thegripper ofend-effector, the flexible joint stopped spreading. The gripperwasthenclosedandtheelectricalcuttercutofftheapplestalk.Finally, the flexible joint backed to its initial position.Thereafter, the gripper was opened and fruit slid along theflexible tube into the basket.To achieve continuous picking the above steps wererepeated.2.4.4.System software designA WindowsXPsystemwas employed as anoperatingplatformfor its good stability and security. Visual C 6.0 was selectedas programming development tool for the host computer. Inthe system, multiply tasks needed to be processed simulta-neously. Noting that a single-thread might lead to datacommunication jams and not guarantee real-time control,a multi-threading event-driven approach was adopted for theprogram control system software. The main thread wasresponsible for the management of visualisation controlinterface, system initialisation; the sub-thread was respon-sible for communication and synchronisation. The sub-threadFig. 13 e Main program flowchart of robot harvesting task.biosystems engineering 110 (2011) 112e122119system involved video capture, motion control, elongationtest of flexible joint and extraction test of prism sub-threads.The main program flowchart for the fruit harvesting robot isshown in Fig. 13.3.Experiment resultsIn this section, the results of a feasibility study of the systemperformed through laboratory tests along with field validationarepresented.Thelaboratoryexperiments wereperformedonthe prototype operating in simulation working conditions.This stage was helpful to set up and optimise the componentsof our system. Finally, the performance of the harvestingrobot was verified in field tests.3.1.Laboratory tests3.1.1.Recognition and Location experimentFor the control system of the fruit harvesting robot, live videowindows on the control softwareinterfacewas usedto displaythe real-time process of picking. Target recognition windowsshowed the accuracy of target recognition, where red “”implied image centre and blue “” implied the centroid of theobject fruit. The position of object fruit could be easily shownin the images. In the target location windows, the track thecentroid of target fruit with regard to image centre during thelocation process was marked with a blue line.Recognition and location test results of object fruit can beseen in Fig. 14. It is obvious that in the figure, accuraterecognition and smooth location track made following fruitgrabbing possible, which verified that the designed robot hasgood tracking performance to meet the requirements ofaccurate real-time recognition and location.During the process of picking operations, video imagesignals needed to be acquired dynamically and continuously,and handled frame by frame. In the video, the size of oneframe of dynamic images was 320 ? 240 pixels. The recogni-tion time for 100 continuous and dynamic images is shown inTable 2.From Table 2, the average recognition time of 100 frameimages was 352 ms. From these results, it was concluded thatthe developed recognition algorithm met the requirements ofreal-timeoperationandthatthesystemcouldbeusedtoguidea robot manipulator as it approached an apple in real-time.3.1.2.Harvesting experimentsA photograph of the fruit harvesting robot operating duringlaboratory simulation harvesting tests is shown in Fig. 15.Under laboratory conditions, apples with radius about 40 mm,were hung on fresh branches in different directions. Theperiod of image acquisition was 100 ms. 100 picking tests werecarried out in 10 different positions.The test results were as follows: successful picking occa-sions 86, and failed occasions 14. Therefore the success ratewas 86%. Without regard to the set-up time, the average timeof picking one apple is 14.3 s. This was high enough to meetTable 2 e Dynamic images recognition time.Image frame12345678910111213141516171819202122232425Recognition time(ms) 235 235 315 390 315 310 390 390 310 390 390 310 390 310 235 315 390 390 310 390 390 390 315 390 395Image frame26272829303132333435363738394041424344454647484950Recognition time(ms) 310 390 390 310 315 390 390 315 310 390 390 310 390 390 310 390 390 310 315 390 315 310 390 390 390Image frame51525354555657585960616263646566676869707172737475Recognition time(ms) 315 390 395 310 390 390 310 315 390 390 315 310 390 390 310 390 390 310 390 390 310 315 390 395 310Image frame767778798081828384858687888990919293949596979899 100Recognition time(ms) 315 390 310 315 390 395 310 390 390 310 390 390 310 315 390 390 310 315 390 390 315 390 390 315 390Fig. 14 e Recognition and location results in laboratory tests.biosystems engineering 110 (2011) 112e122120requirements of continuous harvesting operations. The mainreasons for failure could be attributed to the experimentalenvironment, where the soft foliage and apple vibrationduring operation resulted in a decrease in precision posi-tioning. In addition, occasionally the cutting knife failed to cutthe apple stalk.3.2.Field testsTo further verify the reliability and adaptability of harvestingrobot system, field tests were carried out in the BeijingChangping orchard during October 2009.The recognition result in the orchard is shown in Fig. 16.There 7 apples were well recognised, which indicated that therecognition algorithm could identify apples efficiently. Whereapples are behind branches and leaves and apples cover eachother, the apples cannot be picked directly. Those withoutlabel “” in Fig. 16, would be recognised after picking a certainnumber of apples.In practice, once an image such as that in Fig. 16 wasacquired, the vision system of robot located and picked theapple which had the minimum distance from the imagecentre of the visible-field of the camera. The vision system ofrobot located the next target fruit. Continuously pickingexperiments (shown in Fig. 17) were carried out in an orchardwith a complex environment. In 10 min, 39 apples were rec-ognised, of which 30 apples were picked and put into thecontainer successfully. Six apples failed to be picked sincetheir image was blocked by branches. Three were picked butfell down to the ground due to their small size and the grippernot clamping tightly. After calculation, the mean recognitiontime for picking was 15.4 s and the picking success rate was77%, which indicates that the prototype machine and controlsystem could be used to carry out the picking operationoutdoors.4.Conclusions and future researchA self-developed fruit harvesting robot and its control systemwas developed. The main components of the robot, i.e., themanipulator, the end-effector and the image-based visionservo control system, have been described in detail. Themanipulator was geometrically optimised to gain a quasi-linear behaviour and simplify the control strategy, and theend-effectorwiththepneumaticactuatedgripperwasdesigned to satisfy the requirements for the harvesting ofapples. The harvesting robot autonomously performed itsharvesting task using a vision-based module to detect andlocalise the apple in the trees, and control system regulatedthe manipulator and the end-effector to approach and pickthe apples. The validity of systems was confirmed by per-forming laboratory tests and field experiments in an openfield.Future research needs to be focused on the following threeaspects for practicability and the commercialisation of therobot: (1) optimisation of the existing software programs andalgorithmstoreducecomputation.Atthesametime,improving the speed and accuracy of picking for the blockedor swinging fruits thereby increasing the practicallity of therobot; (2) considering the complexity and unknown nature ofworking environment, the further research should be focusedon real-time obstacle avoidance, improving picking successrate and harvesting efficiency; (3) improving the mechanicalFig. 16 e Apple recognition results in an orchard.Fig. 17 e Harvesting experiments in an orchard.Fig. 15 e Harvesting experiments in laboratory tests.biosystems engineering 110 (2011) 112e122121structure of the robot to enhance the configuration. Forexample, by replacing the manipulator and end-effector withdifferent systems or using different freedom degrees to pickfruit with different shapes and sizes, to achieve all-purposesof the robot. This would increase its versatility, lower itsoverall cost, and promote its commercialisation.AcknowledgementsThis work was supported in part by Research Fund for theDoctoral Program of Higher Education of China under Grant20093227120013,inpartbyNationalHighTechnologyResearch and Development Program of China under Grant2006AA10Z254, in part by a project funded by the PriorityAcademic Program Development of Jiangsu Higher EducationInstitutions, and in part by Innovate Foundation for GraduateStudent of JiangSu Province under Grant CXZZ11_0573.r e f e r e n c e sBulanon, D. M., Kataoka, T., & Ota, Y. (2002). A segmentationalgorithm for the automatic recognition of Fuji apples atharvest. Biosystems Engineering, 83(4), 405e412.Bulanon, D. M., Kataoka, T., & Okamoto, H. (2004). Developmentof a real-time machine vision system for the apple harvestingrobot. Sapporo, Japan Annual conference of society of instrumentand control engineers (SICE) (pp. 595e598).Edan, Y., Rogozin, D., Flash, T., & Miles, G. E. (2000). Robotic melonharvesting. IEEE Transactions on Robotics and Automation, 16(6),831e835.Foglia, M. M., & Reina, G. (2006). Agricultural robot for radicchioharvesting. Journal of Field Robotics, 23(6), 363e377.Grand, dE., Rabatel, A. G., Pellenc, R., Journeau, A
- 温馨提示:
1: 本站所有资源如无特殊说明，都需要本地电脑安装OFFICE2007和PDF阅读器。图纸软件为CAD,CAXA,PROE,UG,SolidWorks等.压缩文件请下载最新的WinRAR软件解压。
2: 本站的文档不包含任何第三方提供的附件图纸等，如果需要附件，请联系上传者。文件的所有权益归上传用户所有。
3.本站RAR压缩包中若带图纸，网页内容里面会有图纸预览，若没有图纸预览就没有图纸。
4. 未经权益所有人同意不得将文件中的内容挪作商业或盈利用途。
5. 人人文库网仅提供信息存储空间，仅对用户上传内容的表现方式做保护处理，对用户上传分享的文档内容本身不做任何修改或编辑，并不能对任何下载内容负责。
6. 下载文件中如有侵权或不适当内容，请与我们联系，我们立即纠正。
7. 本站不保证下载资源的准确性、安全性和完整性, 同时也不承担用户因使用这些下载资源对自己和他人造成任何形式的伤害或损失。
人人文库网所有资源均是用户自行上传分享，仅供网友学习交流，未经上传用户书面授权，请勿作他用。
川公网安备: 51019002004831号