IROS2019国际学术会议论文集 0969

3D Point Cloud Data Acquisition Using a Synchronized In Air Imaging Sonar Sensor Network Robin Kerstens1 Dennis Laurijssen1 Girmi Schouten1 Jan Steckel1 2 Abstract Obtaining accurate data about the environment in which a robot is located is a crucial matter when it comes to autonomous navigation and other robotic applications A popular method of acquiring this information is to use sonar rings where a robot is fi tted with multiple simple ultrasound transducers pointed in the directions where an object can appear However in a time where accurate 3D data is gain ing importance other sensing modalities are becoming more popular because of the ability to measure dense 3D point clouds In these point clouds not only the horizontal plane is measured but objects in the elevation planes can also be registered which can be very interesting and makes applications such as 3D SLAM or object recognition possible In this paper we present a way to extract complex 3D point cloud data from the entire surrounding sphere using multiple interconnected eRTIS sensors These advanced imaging sonar sensors offer the fl exibility of the popular sonar ring in combination with the benefi ts of some of the competing sensing modalities The setup presented here uses less sonar sensors and thus less external hardware while obtaining more information from the complete frontal hemispheres of each individual sensor This setup is discussed along with the issues that arise when using complex imaging sonar sensors in a network and is tested in an indoor and outdoor environment At the end of this paper is a discussion of the obtained results I INTRODUCTION It is important for robots and vehicles to have an understanding of their surroundings in order to be able to complete tasks such as corridor following obstacle avoidance mappingorSimultaneousLocalizationand Mapping SLAM While a variety of sensors are available on the market sonar remains a popular option due to its cost straightforward way of working and its robustness Also in situations where other sensing modalities struggle such as foggy or dusty environments or where it is important for the sensor to be dirt proof sonar sensors are often the best option One way of using ultrasound transducers to obtain 360 degree range information of a robot s environment is a so called sonar ring 1 These sonar rings consist out of a relatively large number of transducers e g 30 or 40 that are all mounted next to each other in a way that they form a ring around the robot 1 2 The small Field of View FOV that is typical for these transducers will lead to each transducer now looking in one direction measuring the distance to the object in front of them There is usually a small overlap between the sensors FOV that is used to increase the accuracy of the system as one object will be detected by two or three sensors This is a simple 1Cosys Lab Faculty of Applied Engineering University of Antwerp and the Flanders Make Strategic Research Centre 2 jan steckel uantwerpen be approach to the obstacle avoidance and mapping problem 3 but it leaves room for improvement better precision larger FOV A lot of research has been done to increase the performance of these perception systems based on sonar ring Using coded transmissions to increase accuracy 4 5 or other techniques that allow for simultaneous emission 3 6 7 which increases the frame rate are some of the advancements that greatly increased the sonar ring s potential Despite these improvements the use of many transducers in a ring formation needing a lot of extra external hardware for control and synchronization can seem a bit cumbersome Especially when many alternatives such as LiDAR or 3D RGB D cameras are becoming more easily available and offer several benefi ts when it comes to information acquisition a b Embedded Real Time Imaging Sonar RTISe c Fig 1 a A schematic overview of the components used in this paper The eRTIS front end is by default connected to the eRTIS backend which will handle the microphone data A USB connection between the eRTIS back end and the Raspberry Pi is used to transfer data to Raspberry Pi memory where the measurements can be stored The enclosure is fi tted with XLR connections to allow for low level hardware synchronisation bypassing the Linux OS Extra GPIO lines are added between the eRTIS back end and the Raspberry Pi to handle additional communication for optimization b The CoSys Lab eRTIS sensor 8 in a custom enclosure c An eRTIS sensor network A big advantage that these alternatives have over the traditional sonar ring is the possibility to obtain detailed 2019 IEEE RSJ International Conference on Intelligent Robots and Systems IROS Macau China November 4 8 2019 978 1 7281 4003 2 19 31 00 2019 IEEE5855 information from upward or downward directions Sonar rings are severely limited when it comes to detecting and differentiating objects above or below the sensor plane 9 This is due to the aforementioned small FOV that is inherent to these classic ultrasound transducers Point Spread Function PSF in combination with the way these sonar rings are designed So for complete 3D spherical perception of a robot s environment these ring like sonar transducer arrangements are no longer applicable In recent years there has been a lot of research towards 3D imaging sonar sensors exhibiting a large FOV that covers the entire frontal hemisphere 180 degrees in both azimuth and elevation of the sensor An example of this is the CoSys Lab eRTIS embedded Real Time Imaging Sonar sensor 8 10 shown in fi gure 1 In contrast to sonar ring setups where large numbers of sonar transducers surrounding the robot would create images of only the horizontal plane the eRTIS large FOV allows for the use of only a small number of sensors two as a minimum to cover the complete surrounding sphere This greatly reduces the overhead that comes along with using 30 or more individual transducers that each require a separate interface and measurement infrastructure Modern sensors that are capable of acquiring data in 3D present their data in the form of 3D point clouds These data sets contain the coordinates for every point that returned a refl ection strong enough to be picked up by the sensor Using this data format offers a lot of possibilities for the user as it is relatively easy to work with and does not need a lot of post processing to represent the raw data data is comprised of simple hx y zi coordinates for each point often combined with the strength of each refl ection in contrast to raw sensor data which does not saliently describe the environment Obtaining point cloud data from a sonar sensor thus has major benefi ts for the end user and the robotic application for which the sensor is intended To build on the capabilities of the eRTIS sensor it can be useful to chain multiple sensors together in a network allowing the user to capture more detail of the surrounding area Doing this requires accurate and stable synchronization and failing to do so will result in artifacts that will greatly hinder the usage of the sensor network For the measurements in this paper three eRTIS sensors have been used One sensor is positioned on the front of the vehicle and two are attached to the sides as depicted in fi gure 6 This positioning gives a coverage of the complete surrounding sphere with additional samples in front of the vehicle which is the main area of interest during navigation This can easily be extended to any number of sensors as will be explained later on in the paper The next section of the paper will dive deeper into the sonar sensor network to provide background knowledge and our alternative approach The third part of the paper will explain more about the consequences of bad synchronization in advanced imaging sonar networks The section that follows describes the way the synchronization problem is tackled in this paper The fi fth section will dive into sonar point cloud data processing with a discussion of our sensor measurement setup in the section after that Section VII will showcase the results we achieved without sensor network Finally a conclusion and discussion is held in the last section of this paper II SONARSENSORNETWORKS The idea of using sonar sensors in a network like fashion originates from the basic principle that a robot must be aware of objects in every direction The choice for ultrasound sensors is because they are ubiquitous cheap and easy to implement 1 And as stated before because the sensors are all positioned next to each other in a ring like fashion this makes it possible to achieve bearing accuracy as low as one degree in the horizontal plane 3 But the sonar ring architecture becomes far less interesting when it comes to sensing objects that do not enter the FOV of the ring And because the ring can be seen as a 1D microphone array it is also hard to gain accuracy in the elevation plane even in post processing In order to be able to see this area more advanced sonar senors are needed Because the eRTIS sensor has a 2D micropone array it can detect refl ectors accuratly in every direction 9 10 In this paper the eRTIS network is installed and operated in a way similar to how a traditional sonar ring is implemented 1 2 in the sense that they are mounted next to each other and that the FOV s partially overlap so that one refl ector is registered by multiple sensors improving certainty and accuracy This is illustrated in fi gure 2 Every sensor is connected to a central server node that can start and stop a measuring cycle and monitor the state of the sensor Later the sensors were also fi tted with a low level hardware interconnection so that very precise synchronization is feasible PC Fig 2 An example of the interconnected eRTIS sensors equipped with an added synchronization port in this case an XLR connection for simplicity and availability Because all sensors are able to record the complete frontal hemisphere a refl ector will be recorded several times increasing the amount of data we receive from it and making the system more robust All sensors are connected to a central server node or computer from which they can be operated and monitored III SONARNETWORKARTIFACTS Because an active imaging sonar sensor relies on the prop agation of sound through a medium and the corresponding refl ections that impinge on its microphone array 11 it is important to make sure the sensors do not interfere with each other when they are used in networks Figure 3 displays a measurement setup where fi ve sensors operate alongside each other The synchronization between these sensors is 5856 handled by a central node running Linux All sensors are equipped with a small embedded computer that has Ethernet connectivity and are connected to the same network using Ethernet cables Using the network for NTP synchronization we achieved approximately 1 ms of non stable delay between the nodes While certainly being better than no synchronization at all some artifacts appear when analyzing the accumulated 3D sonar data Measurements show size increasing planes that are moving away from the vehicle This is of course an unwanted effect as the vehicle will interpret this as an object coming very close and potentially causing a collision These planes are the result of the unstable NTP synchronisation combined with the Raspberry Pi s lack of Real Time Clock RTC causing clock drift The imaging sonar sensor will pick up the ultrasonic chirp that is emitted by another sensor in the chain at different positions of its own measurement cycle and because the sensors clocks are not synchronized properly this will show as size increasing moving planes To make sure these imaging artifacts do not appear in the sensor data it is necessary to have a method of synchronization that is more accurate and more stable a fi xed delay causing a stationary artifact can be mitigated later on in post processing Because of the limitations of the embedded computers that are connected to the eRTIS sensors see fi gure 1 e g bad real time performance of the Linux OS and the lack of RTC causing drift a method that bypasses the Linux OS is preferred This paper uses a method of hardware synchronization that is built onto the back end of the eRTIS sensor and offers a fi xed delay between the different sensors in the network The synchronization method will be further explained in the next section IV SONARNETWORKSYNCHRONIZATION While the eRTIS sensor already generates 3D data by itself 10 for precise scenarios where even more data is needed it can be useful to use multiple sensors A problematic factor in this case is the slow nature of sound which forces the different sensors to measure either at the same time or to give them separate windows to perform measurements The latter approach however limits the measurement rate of the entire system because the sensors can not measure at the same time reducing the overall measurement rate linearly with the number of used sensors To keep the measurement rate similar to a scenario where only one sensor is used it is necessary to measure in a highly synchronized way After trying different methods it was clear synchronizing using the Raspberry Pi running Linux OS was not an option This led to a more low level approach which takes advantage of SYNC ports on the back of the eRTIS sensor bypassing the software clock on the Raspberry Pi attached to the eRTIS back end see fi gure 1 XLR connections were added at the back of the sensor because of their robustness and availability Using these cables it is now possible to connect different sensors together and use a custom method of synchronization that introduces a constant measurement delay of approximately 400 ns This can be seen as negligible as 400 ns is only a small fraction of the time it takes for the 1a 2a 1b 2b 3a3b t t 1 t 2 t t 1 t 2 Fig 3 Three consecutive measurements display the artifact that occurs when there is a synchronization issue between the different 3D imaging sonar sensors A plane that increases in size over time and eventually disappears is caused by a sensor picking up the ultrasonic chirp that is emitted by the next sensor in the chain in combination with clock drift between the different sensors This creates a false representation of objects which can lead to serious issues if the system is used for collision avoidance applications In the image a parked vehicle is shown that is fi tted with fi ve eRTIS sensors all synchronized over Ethernet using NTP synchronization with unstable delay of around 1 ms An object marked with a blue circle is detected in every frame Over the course of three consecutive frames a plane indicated by a red circle can be seen appearing out of nowhere increasing in size and moving away from the sensors before it disappears again a few frames later eRTIS sensor to process one sample at a sampling rate of 450 kHz This means that the impact on the range accuracy of the system is less than 0 2 mm Also since the sonar measurement rate is approximately 10 Hz 400 ns will not cause any issues Figure 4 displays the timing diagram of the synchronization clk SYNC IN EN COMMMSG SYNC OUT Fig 4 The synchronization process between the different sensors The eRTIS back end will receive a SYNC IN from the previous sensor in the chain or an external clock source Upon receiving this SYNC IN signal the sensor back end will send a symbol MSG to the embedded computer attached to the sensor This will initiate the computer to accept the data coming from the sensor if the measuring process is enabled EN The back end will also set the SYNC OUT to HIGH which will be fed as SYNC IN to the next sensor in the chain The entire synchronization process introduces a fi xed measurement delay of approximately 400 ns between the different sensors Which can be seen as negligible BecausetheeRTISdatacaptureisinitiatedwhen SYNC IN goes HIGH there are two ways to control the network One way to synchronize the sonar capture is to attach an external clock to the fi rst sensor in the chain The sensors will start capturing data at the frequency of the 5857 clock provided Using the sonar sensor network in this way allows for the network to be synchronized to other sensors or mechanisms on the robot or vehicle on which it is mounted At the time of writing the maximum measuring rate that can be achieved is approximately 10 Hz Another way to operate the sensor network is to use it independently of external clocks The central server node to which the sensors are all connected can appoint a master role to a sensor in the network When a sensor is appointed as master all other sensors will continue to behave as slaves and wait for the master to set its SYNC OUT pin HIGH The master itself will still collect data only now the other sensors will follow the clock the master provides Using this method solves the measurement mismatch issues that presented itself when using the less stable NTP synchronization V SONARPOINTCLOUDDATAPROCESSING Point clouds have proven to be an effi cient way of dis playing data obtained from 3D measurements The intuitive nature of the dataset simple x y and z coordinates for every date point detected makes it possible to recreate geometries 12 and effi ciently recognize objects 13 14 To be able to use these techniques with 3D sonar in the future it is necessary to process the eRTIS output data in a way that it also has this form factor A brief overview of the process that is used to achieve this is shown in fi gure 5 For a detailed description on how the eRTIS sensor obtains its 3D data by using sonar sensing the reader is advised to read 9 10 Where 9 10 stops when the 3D acoustic images called energyscapes are presented this paper starts from the energyscape obtained from an eRTIS measurement and starts processing the data from here Figure 5 a shows the 32 recorded microphone signals for one measurement of the eRTIS After the initial processing explained in 9 10 a