IROS2019国际学术会议论文集 0804

Landing of a Multirotor Aerial Vehicle on an Uneven Surface Using Multiple On board Manipulators Hannibal Paul Ryo Miyazaki Robert Ladig and Kazuhiro Shimonomura Abstract We describe the concept design and implementa tion of a unique manipulator system for a multirotor aerial vehicle The proposed manipulator system consists of three robotic arms attached to a multirotor airframe with the objective to provide the ability to manipulate single or multiple objects as well as aid in complex navigation tasks by doing contact based obstacle avoidance and acting as adaptive landing gear in uneven terrain In this paper we primarily focus on the description and experimentation of one of the tasks achievable by the proposed aerial multi manipulator system landing a multirotor aerial vehicle on an uneven surface Deploying the on board manipulator as landing gears reduces the hardware carried by the vehicle while additionally providing the ability to land in unstructured and commonly diffi cult to land terrain I INTRODUCTION The use of Unmanned Aerial Vehicles UAV is becoming progressively popular in recent years Instead of UAVs that are only able to observe there is a rise in academic publica tions exploring aerial interactions with the environment This topic is seen in numerous recent researches in the fi eld of aerial manipulation Many of these recent studies are ex ploring manipulator systems for vertical take off and landing VTOL type UAVs with varying manipulator placement and types of application Ollero et al 1 demonstrated the appli cation of aerial robots with dual arms and multi directional thrusters developed for outdoor industrial inspection and maintenance Another research on environmental interaction was presented in 2 with a dual arm aerial manipulator for valve turning operation Suarez et al 3 presented the design and experimental validation of a compliant and lightweight 3 DOF robotic arm equipped with a compliant fi nger module intended for aerial inspection and manipulation In one of our previous researches 4 we developed a manipulator installed above the UAV frame in order to give it the ability to grasp and attach itself near a desired workspace and then perform manipulation tasks using another manipulator attached at the bottom Unlike ground vehicles the process of landing is inevitable and is a vital task for aerial vehicles Yuan et al 5 described a hierarchical vision based localization framework for multirotor UAVs for landing using artifi cial location markers By detecting and recognizing the marker the UAV could estimate its pose at different heights Another work in 6 described the design of a cascaded controller structure that stabilizes velocity and position in the absence of GPS signals by using a dedicated optical fl ow sensor In 7 an image based visual servoing was used to control a VTOL Department of Robotics Ritsumeikan University 5258577 Kusatsu Shiga Japangr0340vs ed ritsumei ac jp Fig 1 Some of the various possible tasks that can be accomplished using the proposed multiple on board manipulator system on a UAV unmanned aerial vehicle while tracking and landing on a moving platform All of the these works used a marker for localization and landing However it is diffi cult to have landing pads or markers during emergency situations in an unknown environment Classical VTOL type aerial robots like the ones using a multirotor design cannot be used in many deployment scenarios because of their inability to land on certain terrains due to the danger of rollover events This applies to slopes uneven surface or even on dynamic surfaces such as unstable ship decks Y S Sarkisov et al 8 addressed two types of rollover events that can occur while landing a multirotor i e dynamic and static rollover Researchers have recently started developing landing gears for aerial vehicles to adapt its position and land on differential terrains 8 9 10 Research funded by DARPA 9 developed landing gear for a helicopter with four articulated two degrees of freedom DoF legs equipped with force sensitive contact sensors in their feet K Goh et al in 10 focused on maintaining the landing spot to prevent the landing gear from skidding sideways on the ground while adapting to different ground conditions Y S Sarkisov et al in 8 developed multicopter landing gears using torque sensors on four robotic legs having two DoF However the weight of the adaptive landing gear system can limit the maximum payload of the aerial robot to attach any other manipulators to operate on a desired workspace In this paper we propose a novel method to use a three arm on board manipulator system to autonomously adapt and 2019 IEEE RSJ International Conference on Intelligent Robots and Systems IROS Macau China November 4 8 2019 978 1 7281 4003 2 19 31 00 2019 IEEE1926 assist in landing of a multirotor aerial robot Unlike the pre viously mentioned researches on landing gears we are using a depth camera to perceive and determine relative maximum terrain height displacements to choose best possible landing posture for the robotic arms keeping the airframe horizontal to the earth surface In addition to the depth camera the three robotic manipulators are each fi tted with ultrasonic distance sensors on its tip for fi ne adjustments during the landing phase The system is designed to autonomously adapt the pose of the manipulators towards the landing surface before touching down Through multiple experiments we test the landing performance of our UAV using the manipulator system by landing on surfaces with different height conditions The manipulators are not only limited for adaptive landing but are multipurpose and can also be used to perform other manipulation tasks like picking up and carrying objects II CONCEPT UAVs with robotic arms attached on board are necessary when it is desired to achieve manipulation tasks at high altitude areas which are diffi cult to reach for human personal or ground based robots Some examples of possible tasks include picking and carrying objects removing stray objects from power lines or installation of an object in a designated high altitude location Due to their high speed high portability and ability to operate in 3D space small size UAVs are highly desirable for time sensitive complex observation or manipulation tasks that can occur e g in rescue operations or disaster relief Unlike the fi xed wing type UAV landing process which almost always requires a large space for runway VTOL type UAVs have the advantage of landing on any fl at solid surface But it is very challenging to land these vehicles in many of these areas because of the lack of fl at ground or a tarmacked base station 9 Some example landing surfaces include sloped rooftops rocky grounds and stairs or ramps indoors In these cases it will be diffi cult to provide aid during an emergency scenario Trying to land on these regions will result in rolling over of the vehicle hence limiting the ability of these vehicles to be used in these deployments 8 A Static stability Different types of terrains for legged robot locomotion is addressed in 11 According to this terrains can be classifi ed into a Flat b With deep holes whose bottom cannot be reached by the legs of the robot c With poles or rocks d With both holes and elevations and e Usual rough terrain In case of a landing can be easily achieved by not much adjustment in the manipulator positions However in rest of the cases a highest possible position is to be chosen in order to avoid terrain elevation touching the UAV airframe The stability of the robot is of high importance for safe landing as well The manipulators during landing can be seen as legs of the robot In order for the robot to safely rest on a surface the position of the manipulators and orientation of the body should be such that it is statically stable without Fig 2 Stability of the robot on a Completely fl at surface b Sloped region with robot body tilted c Sloped region with robot body horizontal to ground the requirement of any external force to balance the robot after landing As defi ned by 12 and 13 a necessary condition for a robot with point feet to be statically balanced is that at least three feet have to be in ground contact at all times placed such that they form a support surface with an area that is not equal to zero and the vertical projection of the center of gravity CoG has to be within the boundary of the support surface The stability margin is defi ned as the shortest distance from the vertical projection of the center of gravity Cg of the vehicle onto a horizontal plane to the boundary of the support area It is defi ned as positive if the center of gravity is within the support polygon and negative otherwise Fig 2 shows support polygon for a three legged robot UAV stationed on a fl at surface on a sloped surface with body angle same as slope and on a sloped surface with body horizontal to ground The support polygon is decomposed into 3 sub triangles in order to calculate the stability margin SM And the triangles are marked S1 S2 and S3 similar to 11 In case of Fig 2 a the Cg is always inside the stability polygon In case of Fig 2 b the Cg moves towards the boundary of the polygon and gradually outside if the slope angle Sexceeds certain value Fig 2 c shows landing on a slope but keeping the body horizontal to ground by adjusting the manipulator postures The Cg remains inside the polygon if the manipulator tips are always away from the vertical projection of the body The maximum slope can be determined by the minimum position of Leg3 and maximum position of Leg1 in the case shown B Our work The goal of our research is to develop a multiple arm manipulator system for a UAV to inherit multiple abilities As described in Fig 1 the manipulators in unison can be used to solve various tasks e g carry a single large object carry multiple small objects using single or dual arms avoid dangerous obstacles by physical contact during fl ight or land on a given terrain Our main focus for this paper is landing a UAV using three manipulators as adaptive landing gear using the point cloud information from the depth camera These manipulators also have a small 1 DoF gripper each as an end effector which makes it suitable for multipurpose tasks During the landing operation the grippers are opened perpendicular to the manipulator tip to provide wider area of contact to the surface 1927 Fig 3 Illustration of minimum and maximum position of the manipulators in Z axis Fig 4 Top and side view of the proposed manipulator system on a UAV III SYSTEM DESIGN The primary purpose of the proposed system is to have a UAV with multipurpose capabilities at the lowest possible weight The idea of combining the adaptable landing gears and on board manipulators led to the design of this on board multipurpose manipulator system for UAVs A System architecture A hexrotor platform with DJI N3 fl ight controller is used as the UAV airframe base The system consists of three robotic manipulators attached at equal spacing around the UAV airframe With three contact points placed at 120 the CoG is within the support polygon if there is no lateral displacements as shown in Fig 2 c The manipulators are attached in such a way that the workspace is maximized and can move above below as well as to the side of the airframe Fig 4 shows the top and side view of UAV with the manipulators attached The manipulators are named Arm1 Arm2 and Arm3 for later reference Serial servo motors Robotis Dynamixel AX 18A are used for actuation of each joints with feedback functionality to read the current joint positions of individual servos The use of serial com munication lines simplifi es the connection complexity with minimum wiring for both control and feedback A stereo vision based camera ZED mini is used to get a point cloud of the terrain to determine the best possible landing pose Additionally an ultrasonic distance sensor HC SR04 is attached near each of these grippers to provide additional surface distance information during the landing process All the on board processing and control is done by a power effi cient embedded computing device NVIDIA Jetson TX2 Fig 5 Block diagram of the proposed system mounted on the airframe The block diagram of the system is as shown in Fig 5 The communication between the different parts of the system is realized using Robot Operating System ROS running in the on board computer While the UAV s fl ight is currently controlled manually along with the control command to acti vate the landing system all image processing and adaptation processing for the landing process is done autonomously on board B Manipulator design The manipulators are of original design and constructed out of ABS and PLA material Each manipulator has 3 DoF i e shoulder yaw shoulder pitch and elbow pitch joints 1 2and 3respectively as shown in Fig 3 with link lengths L1 L2and L3 In addition a gripper is attached each at the tip to enable a wide arrangement of grasping tasks The relation between joint angles 1 2and 3and the gripper position with respect to the reference frame attached to the base of the manipulator is given in equation 1 x y z L 1c1 L2c1c2 L3s2s3c1 L3c1c2c3 L1s1 L2s1c2 L3s1s2s3 L3s1c2c3 L2s2 L3s2c3 L3s3c2 L0 1 where c1 c2 c3are cos 1 cos 2and cos 3respectively whereas s1 s2 s3are sin 1 sin 2and sin 3respectively The distance sensors attached near the tip of the elbow link L3 are directional Therefore at the time of landing it is necessary to maintain perpendicular posture of the elbow link to the ground at all times in order to measure the distance to the landing surface Therefore the sum of angles 2and 3are to be kept at 90 for the elbow link L3to maintain perpendicular posture to the surface directly under it The inverse kinematics equation used to obtain the joint angles in degrees from the gripper position in the Cartesian coordinates is given by 2 1 2 3 atan2 y x sin 1 z L2 90 sin 1 z L2 2 Table I lists the physical specifi cation of the system used for experimentation 1928 Fig 6 Mechanical joint compliance introduced in the lower part of link L3 TABLE I SPECIFICATIONS OF THE PROPOSED SYSTEM USED FOR THE EXPERIMENTATION Part Specifi cationDimension Whole robot Width590 mm Height max 480 mm Weight3500 g Rotors6 Manipulators3 Manipulator each Degrees of Freedom3 DoF 1 gripper Link lengths L1 L2 L3 45 mm 155 mm 170 mm Width x Thickness5 mm x 3 mm Weight350 g However as mentioned before since the link L3is to be maintained perpendicular to the surface during landing it will be diffi cult to land on a terrain with slope Even if the manipulators are adjusted according to the heights directly under it on this slope the UAV will not achieve stable posture after landing Therefore the construction of the lower part of the link L3consists of mechanical joint compliance realized using springs Extension springs with a spring constant of 0 951N mm were used in either sides of a joint generating a torque of 242 82N mm at the joint which is suffi cient to keep them in position when they are not adapting to a sloped surface 3 The passive mechanical compliance is introduced in both roll and pitch axis of the gripper Using this mechanism the manipulators are able to passively adjust the gripper tip according to the terrain slope after touching the surface The mechanical compliance used in the system is as shown in Fig 6 and can passively handle slopes up to an angle of 40 When the UAV takes off from the ground the compliance retracts the manipulator tip back to its original position IV PERCEPTION SYSTEM It is crucial to know the terrain information before landing in that terrain A terrain too steep or having large height variation is not suitable for landing Proper clearance under the airframe and near the propeller blades are some of the required condition for safe landing and are accounted for in our method A light weight stereo color camera ZED Fig 7 Estimating the ground reference and the distance to travel for landing Fig 8 Terrain data processing a Depth map of the terrain b Workspace of manipulators used for landing c Processed map with manipulator position on terrain for landing mini attached under the airframe is used to capture the image of the terrain under the UAV Depth map of the scene is obtained in the form of a point cloud in the on board computer A depth range of 0 15m to 12m can be detected by this camera The camera placement is such that the center of the point cloud corresponds directly with the terrain under UAV airframe The processing is done in Python and the point cloud visualization for verifi cation was achieved using Open3D 14 In addition to the point cloud data the camera has inbuilt gyroscope and accelerometer providing pose and orientation data using visual inertial stereo SLAM technology The fi eld of view FoV of the ZED mini camera is 90 in horizontal and 60 in vertical direction of the camera A default position for all manipulators is chosen on the UAV s XY plane at a local distance Lzmidfrom the center of the UAV as shown in Fig 7 Lzmidis chosen so that the manipulators do not occlude the camera during the extraction of the landing spots However the camera is still able to provide a correct odometery information even if the manipulators partially occludes the camera during landing 1929 A Extracting clearance region The fi rst stage of depth processing involves segmenting the regions to check for clearance A clearance region is defi ned as the shortest distance between the ground surface and the lowest part of a vehicle body other than those parts intended for contact Let Q be a set of four dimensional vectors denoting points in a point cloud consisting of X Y Z position and color Two circular regions one for inner clearance region Qc1 of 200mm radius and another for outer clearance region Qc2 of 400mm radius is extracted from this set in a XY plane based on Euclidean distance from the center The dimensi