IROS2019国际学术会议论文集 2181

Evaluation of a Large Scale Event Driven Robot Skin Florian Bergner Emmanuel Dean Leon J Rogelio Guadarrama Olvera and Gordon Cheng Abstract This paper evaluates and describes the large scale integration of our multi modal event driven robot skin system on our humanoid robot H1 REEM C PAL robotics The robot skin is powered by the robot and all processing of tactile perception and control are executed onboard the robot The robot skin system largely covers the humanoid and employs 1260 skin cells in 47 skin patches in total 7560 multi modal tactile sensors The robot skin system is driven by events i e the occurrence of novel information drives the acquisition transmission and processing of information rather than the synchronous sampling clock as in clock driven systems The event driven robot skin system enables effi cient multi modal large area tactile perception which is a prerequisite for realizing reactive whole body control We analyze the new robot skin system and evaluate its performance in clock driven mode and event driven mode This analysis includes modeling the required CPU load and investigating the scaling of the system in multi core systems We evaluate the robot skin system with an experiment where a large number of skin cells is activated 680 cells and a large number of events is generated The obtained results demonstrate the superior performance of the event driven system In the clock driven mode the robot skin system constantly produces 315 000 packets s while in event driven mode the system at most produces 40 000 packets s 13 The CPU load reduces from constantly 270 to at most 100 37 In clock driven mode the PC drops on average 80 000 packets s 25 of all packets while in event driven mode the package loss is practically neglectable This effi cient large scale robot skin enables the complete onboard integration into a humanoid robot without the need for additional external power or processing capabilities I INTRODUCTION A Motivation Perception in general is a key factor to develop robust and adaptable robotic systems that have the capability of realizing complex activities Among the senses of vision and audition the sense of touch increasingly becomes an attractive and interesting research area for roboticists Similar to nature the sense of touch is essential for robots when contacts and physical interactions are involved especially when robots need to coexist and collaborate with humans in shared spaces Providing the sense of touch to robots is not straightforward and has been a research topic for decades The diffi culties lie within the nature of the sense of touch In contrast to vision or audition the sense of touch is distributed throughout the whole body The sense of touch is part of the somatosensory system that includes cutaneous information touch information and propriocep tive information kinesthetic position and movement in formation The human body employs around 5 million AllauthorsarewiththeInstituteforCognitiveSystems TechnischeUniversit atM unchen Munich Germany see http www ics ei tum de cutaneous receptors 1 mechanoreceptors thermoreceptors nociceptors connected to around 1 1 million ascending nerve fi bers 2 The realization of a comparable system for robots a robot skin system is challenging considering the number of discrete sensors involved and the amount of information to transmit and process The work of 3 reviews and compares tactile sensing in robotics and summarizes requirements and design principles for evolving robot skin systems towards the capabilities of human skin Challenges are discussed in 4 The key challenges towards large scale applications such as on a humanoid robot are reliability and robustness simple deployment wiring feasible acquisition of sensor poses and effi cient and low latency transmission and processing of a large amount of tactile information These challenges are complex open problems In this work we address the large scale deployment of our event driven robot skin system to self suffi cient robot systems and evaluate the performance of this skin system We specifi cally focus on the reliable modular wiring of skin patches in large scale deployments and on the effi cient transmission and processing of a large amount of tactile information B Related Work A wide variety of solutions has emerged for providing adequate tactile sensing for different robot applications These solutions target two dominant applications 1 tactile fi ngertip sensing 6 8 with a focus on concentrated high spatial resolution super sensitive sensing shear force and vibration sensing and slip detection and 2 large area tactile sensing with a focus on distributed large scale deployment of sensors full coverage of robots multi modal sensing acquiring the pose of tactile sensors on robot links and effi cient and low latency transmission and processing of a large amount of tactile information The early work of 9 10 utilizes arrays of IR based proximity sensors 80 sensors in total on a robot arm The authors feed the acquired tactile proximity information into a planner such that the robot can avoid obstacles in real time The work of 11 introduces a fl exible bendable and stretchable matrix of pressure and temperature sensors as conformable skin The authors demonstrate the feasibility of manufacturing such a skin up to a size of 12 12 Another fl exible skin for robots that employs matrices of force sensors is presented in 12 The authors 13 improve the deployability of fl exible tactile sensor matrices through modularization and enabling simple customization the skin can be cut into shape To overcome the main drawbacks of sensor matrices namely the high number of wires and the susceptibility to row column wise failures modular solutions for robot skin have IEEE Robotics and Automation Letters RAL paper presented at the 2019 IEEE RSJ International Conference on Intelligent Robots and Systems IROS Macau China November 4 8 2019 Copyright 2019 IEEE FrontBackICS H1 1260 x Skin Cells Prox LED Force Temp Acc Port 1Port 2 Port 3Port 4 Micro Controller 47 x Skin Patches 12 x TSU Satellite 3 x TSU LogicBox Control PC PC i7 4700EQ 2 4GHz 8Gb DDR3 RAM Battery Li Ion 44 4V 27 2Ah Media PC PC i7 3612QE 2 1GHz 8GB DDR3 RAM DC DC 60 5V TSU LB port Power port 6 x skin ports 4 x Satellite Connection Boards Power port FPGAGigaBit Ethernet port TSU LB portMain power port Fig 1 Modular integration of our large area event driven robot skin system in our humanoid robot platform H1 5 emerged The RI MAN robot 14 is equipped with fi ve tactile sensing modules two per arm one on the chest Each module embeds an 8 8 force sensor matrix resulting in a total of 320 force sensors Similarly the ARMAR III robot 15 uses modules that employ force sensor matrices on its shoulders and arms The TWENDY ONE robot 16 achieves more complete coverage with force sensors distributed in its hands 2 241 sensors in its arms 2 54 sensors and its trunk 26 sensors However its sensing density is lower RoboSkin introduced in 17 uses a truly modular approach RoboSkin can cover arbitrary surfaces and is assembled from triangular skin modules Each of these skin modules employs 12 capacitive force sensors This modular approach allows great deployment fl exibility an effective reduction of wires and the containment of failures The works of 18 19 demonstrate that such a modular approach enables full coverage of robots and large scale robot skin by successfully covering a Nao robot with 200 force sensors and an iCub with 2000 force sensors Recently the work of 20 21 show progress in upgrading RoboSkin with 3D magnetic force sensors Besides a successful large scale deployment of robot skin sensors for full coverage of robots a useful robot skin system also needs to address the effi cient transmission organization and representation of tactile information The work of 22 present a query based modular readout system that is used for RoboSkin on iCub The real time middleware introduced in 23 connects this readout system with a query based organization and representation of tactile information Recently progress in neuromorphic engineering resulted in a event driven force sensor 24 that directly transduces forces into spike trains in address event representation AER The work of 25 presents a compound AER system with off the shelf sensors and serial AER for event driven communication and processing of tactile information in humanoid robots which promises higher effi ciency and lower latency First steps to an event driven middleware for standard PC systems was introduced in 26 C Contributions This work evaluates the large scale integration of our event driven robot skin system on a self suffi cient battery powered robot platform with onboard processing capabilities For this evaluation we use our humanoid robot H1 5 see Fig 1 We demonstrate the feasibility of integrating our robot skin system in a mobile robot platform with limited processing power and a closed power system with limited electrical energy yet achieving large area coverage with skin and real time multi modal tactile perception The complete integration of our large scale neuro inspired event driven robot skin system in a robot platform enables effi cient large area tactile perception a prerequisite for reactive whole body control and interaction We deeply analyze our new robot skin system and evaluate its performance To this aim we drive the system in two different modes in the clock driven mode and the event driven mode In clock driven mode a clock drives the system with continuous sampling transmit ting and processing Contrarily in event driven mode novel information the events drives the system on demand Our analysis includes the modeling of the required CPU load investigates the scaling of the robot skin system in multi core systems and discusses the trade off between required CPU load and modularity Finally in our experiment we present large area tactile perception evidencing the limits and restrictions of the clock driven robot skin system and the feasibility and superior performance of the event driven system Our recent publications present different applications with our skin equipped humanoid robot H1 5 where we successfully took advantage of the large scale event driven robot skin system described and evaluated in this paper For instance we use this robot skin system to enhance biped locomotion 27 or to realize whole body active compliance 28 29 II SYSTEM DESCRIPTION A Robot skin system for humanoid robots We realized the complete integration of our event driven robot skin system in our self suffi cient humanoid robot H1 5 achieving large area coverage see Fig 1 The integra tion of a large scale robot system in a mobile platform is challenging since such systems can only provide limited processing and electrical power Thus a tight and effi cient integration of the robot skin system with the robot platform is necessary while the whole system needs to remain fl exible enough to fi x problems or to extend the skin system without compromising the robot platform To tackle these problems we pursue a modular robot skin system realizing both a tight but yet fl exible scalable and effi cient integration of the robot skin system with the robot platform B Robot Skin Cells The smallest elements of our robot skin system are the hexagonally shaped skin cells 30 see Fig 1 These skin cells integrate the sensors our robot skin system uses to perceive cutaneous tactile stimuli from the environment Each skin cell employs the same set of sensors namely a 3D acceleration sensor to sense vibrations and net linear accelerations three capacitive force sensors to measure normal contact forces an optical proximity sensor to measure the distance to close objects and a temperature sensor to measure thermal properties of contacts Thus each skin cell provides multi modal contact and pre contact information All skin cells employ microcontrollers which provide local processing capabilities allowing the skin cells to acquire and fi lter sensory information locally before sending it to higher processing layers Furthermore these microcontrollers provide serial communication ports to up to four neighboring skin cells see Fig 1 forming a redundant robust meshed robot skin cell network The application code running on the skin cells microcontrollers supports two modes the clock driven mode and the event driven mode 31 The skin cells can change the sample rate of the sensors 0 Hz 62 5 Hz 125 Hz and 250 Hz which affects both modes A sampling rate of 0 Hz refers to an idle skin cell that doesn t send any packets When the skin cell is in the clock driven mode it sends data packets containing the sensory information of all its sensors The data packet rate always equals the sample rate However when in event driven mode the skin cells sample the sensors with the specifi ed sample rate but the event generator only creates and sends packets containing events event packets when novel information a tactile event has been detected The event driven mode considerably reduces the required transmission bandwidth C Network of Skin Patches A group of directly connected skin cells forms a skin patch see Fig 1 Skin patches can be interconnected by cables using free communication ports of some of their skin cells Then these cables supply power and communication bandwidth to the skin cells of a patch Due to the self organizing capabilities of our robot skin system 28 30 the system can automatically determine bi directional com munication paths to each skin cell within a patch and which skin cells belong to a specifi c patch Consequently this self organization allows us to arbitrarily shape and interconnect skin patches as long as each skin cell receives suffi cient power and network bandwidth D Tactile Section Units and Satellites To facilitate the connection of skin patches and networks of skin patches to a PC we developed Tactile Section Units TSU A TSU provides power and serial connections to skin patches a 1 Gbit s Ethernet communication interface to the PC and bi directionally translates between packets of the serial skin cell network and standard UDP packets Our standard TSU can power 300 skin cells and provides fi ve connections to the skin cell network 30 Unfortunately the size of such a TSU is too large for its deployment onto robot limbs To avoid the voltage drop when connecting large skin patches to long cables we revised and improved our TSU concept Our new TSU system now consists of a TSU LogicBox and up to four TSU Satellites see Fig 1 The TSU Satellites are small enough for placing them on robot limbs A TSU Satellite provides six serial connections to the skin cell network six skin ports and locally converts a high voltage level of the robot power system to the lower level for the skin cells Furthermore the TSU Satellites bundle the six serial connections and connect them through one cable to the TSU LogicBox The TSU LogicBox can support up to four TSU Satellites see Fig 1 For each TSU Satellite the TSU LogicBox bi directionally translates between packets of the serial skin cell network and standard UDP packets providing one UDP Socket connection per TSU Satellite As a result one new TSU system with one TSU LogicBox and four TSU Satellites is equivalent to four TSUs of the previous version Moreover the ability to place the TSU Satellites on robot limbs moves the power source closer to the skin patches avoiding the inherent power drops of long cables and greatly reduces the number of cables from skin patches to TSU TSU LogicBox in a six to one ratio A TSU Satellite with its six skin ports combines up to six cables to skin patches to one single cable between a TSU Satellite and its TSU LogicBox Thus for every six cables to skin patches only one cable needs to run from a body limb to the torso E Robot Skin on H1 The integration of our robot skin system on H1 5 for complete skin coverage resulted in deploying 1260 skin cells in 47 skin patches see Fig 1 This skin system incorporates 7560 sensors 3780 force 1260 proximity 1260 temperature and 1260 acceleration sensors 12 TSU Satellites and 3 TSU LogicBoxes provide power and communication H1 carries its three TSU LogicBoxes on its back A 1 GBit s Ethernet switch connects the three TSU LogicBoxes directly to the media PC Intel Core i7 3612QE 2 1 GHz of the robot The 12 TSU Satellites are distributed to the robot limbs The following table summarizes how many skin patches and cells are connected to the individual TSU Satellites TABLE I TSU SatelliteAcronym Patches Cells right upper armRUA247 right lower armRLA5151 right torsoRT487 right fl ankRF4154 right upper legRUL5108 right lower legRLL4102 left upper armLUA247 left lower armLLA5151 left torsoLT488 left fl ankLF3114 left upper legLUL5109 left lower legLLL4102 H1 s battery has a capacity of 1 2 kWh see Fig 1 and its complete skin system skin cells interfaces and power converters consumes around 86 W max 97 5 W when all LEDs are turned white Since the data acquisition on the skin cells is not event driven the sensors have to be constantly sampled the power consumption of the robot skin system is almost the same in clock and event driven mode However the event driven mode induces less computational load thus reduces the power consumption of the computers F Robot Skin Driver Our robot H1 uses the robot operating system ROS as middleware and we completely integrated our robot skin system into the ROS environment 31 A ROS skin driver node skin driver using the low level support libraries of our robot skin system connects the skin cells of each TSU TSU Satellite into the ROS environment Consequently the robot skin system of H1 employs 12 of these skin drivers which run on the media PC of H1 Furthermore the skin driver and its ROS interface support both skin operation modes the clock driven and the event driven mode G Event Generation When operating in the clock driven mode each skin