IROS2019国际学术会议论文集Arguing Security of Autonomous Robots

Arguing Security of Autonomous Robots Nico Hochgeschwenderand Gary Corneliusand Holger Voos AbstractAutonomous robots are already being used, for example, as tour guides, receptionists, or offi ce-assistants. The proximity to humans and the possibility to physically interact with them highlights the importance of developing secure robot applications. It is crucial to consider security implications to be an important part of the robot applications development process. Adding security later in the applications life-cycle usually leads to high costs, or is not possible due to earlier design decisions. In this work, we present the Robot Application Security Process (RASP) as a lightweight process that enables the development of secure robot applications. Together with RASP we introduce the role of a Security Engineer (SecEng) as an important stakeholder in any robot application development process. RASP enables the SecEng to verify the completeness of his work and allows him to argue about the applications security with other stakeholders. Furthermore, we demonstrate how the RASP supports the SecEng and also other developers in their daily work. I. INTRODUCTION Autonomous and mobile service robots are currently very prominently used, for example, as tour guides, receptionists or offi ce assistants. These robots are built to interact with humans and have the capabilities to cause serious harm to their environment. Furthermore, they have sensors that gather large amounts of data, which might contain sensitive information. A mobile service robots physical capabilities are controlled by networked computers susceptible to faults and intrusions. Which is not necessarily a problem if the robot is deployed in a research laboratory. However, in public environments where robots will ultimately be deployed, we need to obey local security rules and regulations, for example, GDPR for data protection inside the European Union 1, or security standards 2 imposed by organizations or companies if robots are deployed in their localities. Following these rules and regulations is already a demand- ing exercise for regular software application development 3. For robotic applications this is even more challenging, not only due to the fact that the robotic community missed the opportunity to establish a holistic security culture, including standards, procedures and practices, but also by the robotic specifi c characteristics of autonomy and learning abilities. In this paper, however, we fi rst aim to remedy the following three shortcomings of todays robot application development processes. First, the lack of security awareness of robot application developers 4. Second, the lack of proper development approaches in robotics which adequately Both authors equally contributed to the paper. Nico Hochgeschwender is with the German Aerospace Center (DLR) and the Bonn-Rhein-Sieg University, Germany. Gary Cornelius and Holger Voos are with the Interdisciplinary Centre for Security, Reliability and Trust, University of Luxembourg, Luxembourg.holger.voosuni.lu support the development of secure robot applications. This is due to the fact that developers either consider security aspects too late in the development life-cycle, or do not consider security at all. In addition, risk management techniques are not well known and often impose a huge overhead to small teams where developers usually have multiple tasks. Third, the lack of training developers receive regarding security related best practices 5. In this work, we make the following contributions to tackle the above mentioned shortcomings: We defi ne an iterative, structured process model called RASP (Robot Application Security Process) for devel- oping secure robot applications. The process introduces security engineers to use the process to systematically support the development of secure robot applications throughout the complete life-cycle of robot applications ranging from project initiation to robot maintenance. We demonstrate how security-related artifact outcomes of the RASP can be used to assure that a robot ap- plication is acceptably secure. To this end, we apply the Goal-Structuring-Notation 6 for arguing about the security of robot applications. We demonstrate the feasibility of the RASP with the help of a mobile service robot application in a museum environment and we report lessons learned on how se- curity engineers in robotics can implement our process model to other applications and scenarios. The remainder of this paper is structured as follows. In Sec. II, we describe the application which serves as a motivation and validation of our approach. In Sec. III, we introduce the RASP which supports the development of secure robot applications. In Sec. IV, we employ the Goal-Structuring-Notation for arguing about the security of our robot application. In Sec. VI, we discuss related work and report lessons learned. Some conclusions are drawn in Sec. VII. II. MOTIVATION: MUSEUMTOURROBOTS ASTARGETS FORADVERSARIES To motivate and exemplify the pressing need to study se- curity for mobile service robots we describe in the following paragraphs a real world application. In the context of a re- cently fi nished research project with the City of Luxembourg, we investigate mobile service robots as museum tour robots. To this end, we programmed and deployed a Pepper1robot to perform both infotainment and edutainment tasks in the Luxembourg City History Museum. 1Pepper is a humanoid robot manufactured by SoftBank Robotics (for- merly Aldebaran Robotics). 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) Macau, China, November 4-8, 2019 978-1-7281-4003-2/19/$31.00 2019 IEEE7785 Fig. 1.Pepper inside the Luxembourg City History Museum. The general idea of this application is that the robot actively approaches museum visitors in order to give them insight into what living in the city of Luxembourg was like in the 17th century. To do so, after approaching visitors the robot presents historical facts about the city (see also Fig. 1) or the robot plays an informative game with the visitor. Beside a satisfactory functional performance of the robot our project partners are in particular concerned about the security-related aspects of the overall system. This can be explained with the fact that museums which have valuable exhibits are by defi nition more sensitive in this regard. From an information security perspective we identifi ed in the context of our project at least three different ways how a robot could be used by an adversary to cause serious harm in the robots physical environment and signifi cant privacy issues. DataPrivacy:Therobotssensorscollectlarge amounts of sensitive and personal data (e.g. facial images or conversations), which could be accessed and used by an attacker that gained access to the robot for sake of Blackmailing and so forth. Physical Damage: An attacker which gained control of the robot can cause serious hazards in the museum en- vironment. This includes not only the physical damage to the valuable objects that are part of the exhibition, but also to visitors or employees of the museum. Psychological Damage: Access to the robot can also be missuses in order to cause psychological damage either to visitors or museum employees. This can be done, for example, by playing disturbing messages trough the robots speakers or tablet. Together with the museum, we identifi ed these three to be the most prominent attacker motivations in the museum environment, others can be found in the literature 5. In our application, Pepper is connected to the Internet, for the sake of using cloud-based services such as speech recognition, and is therefore at potential risk of being accessed and misused by adversaries interested in causing harm. III. RASP: ROBOTAPPLICATIONSECURITYPROCESS Even though robot applications are very complex in nature, the development of robot applications is often done in an ad-hoc manner and development processes are often not implemented. On top of this, robot developers and processes usually do not take into account security issues during the life-cycle of robot applications 4. Therefore, our proposed RASP (Robot Application Security Process) is complemen- tary to the usage of any existing software engineering process in the organization, tackling the different phases of a robot application life-cycle. A. Security Engineer Stakeholder The RASP introduces the Security Engineer (SecEng) as a new stakeholder of a robot application development process. The SecEng is leading the security related activities of the RASP to be integrated into an existing robot application development process. During the applications life-cycle the SecEng performs those activities alongside the different project phases. The activities yield to certain artifacts such as documentation of requirements, specifi cations, test pro- cedures, and so on. These artifacts can later be used to argue about the security of robot applications (see Sec. IV). Another responsibility of the SecEng is to establish a security culture among the actors participating in the application development process 3. B. Terminology Before the different phases of the RASP are explained we clarify fi rst some security-related terminology. The RASP guides through a lightweight risk analysis process which helps to identify assets, threats and vulnerabilities, taking into account the robots operation environment. A risk is defi ned as the potential for loss, damage or destruction of a certain asset as a result of a threat exploiting a vulnerabil- ity 7 8. Risk = Asset Threat V ulnerability Here an asset represents what we are trying to protect. This includes people, property and information. In case of a robot application, this means the complete robot environment including the robot itself. A threat is used to exploit a vulnerability intentionally or not and steal, damage or destroy an asset. A vulnerability is a weakness or gap in the security of a program that can be exploited by certain threats in order to gain unauthorized access to an asset. An attack uses one or more vulnerabilities to realize a threat. The risk is defi ned as an intersection of assets, threats and vulnerabili- ties. Robots can be a target for a wide range of adversaries from skilled individuals, academic research groups, non- government gangs to state-level intelligence agencies. Each having their own set of resources and capabilities 9. In our application, for example, it would not be of interest for a state-level adversary to attack a robot operating in the mu- seum environment. Therefore, the likelihood that an attacker would exploit a sophisticated vulnerability is unlikely and thus, focus needs to be put fi rst on vulnerabilities which are more likely to abuse. Having identifi ed the possible threats and vulnerabilities, a risk assessment is usually performed. For this purpose the 7786 SecEng needs to list all assets, then threats and vulnera- bilities related to them. Risk is represented as a function of likelihood and impact. The impact of a negative event taking place is usually described as loss or degradation of one of the three characteristics of data, which are: integrity, availability and confi dentiality. These impacts can be tangible, which allows them to be measured quantitatively, or intangible, measured in units generally reaching from Low to High. The SecEng assess the impact and likelihood for each possible combination and fi nally calculates the risk level. For this purpose, the SecEng can use a risk matrix 10, where the likelihood of the hazard happening is ranked between Ex- tremely Unlikely (1) and Almost Certain (5). The impact of the potential injury/damage is ranked between Insignifi cant Injury/Damage (1) and Catastrophic Injury/Damage (5). The product of likelihood and impact defi nes the risk value which is represented as: Low (0-5), Moderate (6-10), High (11-15) and Extremely High (16-25). Listing the risk of all items gives an indication of what needs to be done to lower the systems overall risk and thereby make the system acceptably secure. We propose to use this intangible solution, because it keeps the process simple and lightweight, however more detailed methods can also be used. C. Process Overview Security is a highly dynamic fi eld with new attacks and/or defense mechanisms being discovered on a daily basis. As a result, any process involving security needs to be iterative and allow to return to previous phases in order to analyze and reevaluate obtained results. RASP was designed in a way that it allows to return to earlier phases at any point. This enables the SecEng to react to changing circumstances, or fi ndings along the projects lifetime. Furthermore, it allows RASP to be integrated in an existing iterative software development process. The proposed RASP (see Fig. 2) is divided into 5 different phases which are described in the following subsection. A phase is defi ned by its actors, inputs and pro- duced artifacts. Artifacts represent the outcomes of a phase, whereas inputs, represented as information fl ow in Fig. 2, defi ne the information needed to successfully complete a phase. An input might be the artifact of the previous phases or external information generated, for example, during the application development process. The different phases are performed by the SecEng in close collaboration with the other actors of the application development process. D. Process Phases 1) Identifi cation of Threats: A detailed analysis of the scenario proposed for the robots usage is performed and an overview of the associated threats is made. It is important to identify all possible threats to assess possible consequences of an attack, including the physical safety, psychological safety and data privacy. This is done based on an abstraction of the hardware, which allows to gain an overview of the general threat plane and encourages deliberation of all involved stakeholders. Fig. 2. Workfl ow of the Robot Application Security Process First, the robots capabilities, including its sensors, ac- tuators and physical properties are described. Second, the robots environment, including assets like objects, animals, humans, location, stakeholders and the robot itself is de- scribed. Third, the threats applying to the robot based on his capabilities and environment are listed and described. The adversarys motivations are defi ned according to scenario, robot capabilities, assets and threats. 2) Analysis of Vulnerabilities and Risk Assessment: Threats are refi ned taking into account the robots concrete hard- and software stack. The information of this concrete stack is provided, for example, by artifacts produced in the application development process. Each threat is analyzed and possible vulnerabilities are listed. It is important to note that a threat can be exploited by one or more vulnerabilities. The output of this phase is the Risk Assessment document, it is composed of a list of assets, threats and vulnerabilities together with the complete risk assessment for each possible combination of threat, vulnerability and asset. The document also includes the informal defi nition of the adversarys mo- tivations. As no system can be absolutly secure all the time, there- fore the goal is to make the robot application acceptably secure depending on, for example, organizational constraints and regulations. To achieve this, it is important to identify most vulnerabilities applying to the threats. Once the list of assets, threats and vulnerabilities is completed, a risk assessment for each possible combination is performed by using the risk matrix described in Sec. III-B. During the risk assessment phase new threats might be discovered, therefore the process allows return to the fi rst phase (Identifi cation of Threats). 3) Identifi cation of Security Requirements: Taking into account the assigned risk value, the vulnerabilities are ranked according to priority. To make the system acceptably secure, the global goal is to decrease the overall risk of the system. 7787 To achieve this, solutions are proposed which consist, among others, of defense mechanisms such as intrusion detection algorithms or, if possible, different hardware components. Furthermore, recovery mechanisms need to be considered in case an ongoing attack is detected. This recovery mech- anisms can consist, for example, of a complete system shutdown, partial shutdown, or reset of some of the robots capabilities. If specifi c test constraints apply, they are also specifi ed inside the security requirements. Providing the developers with the applications detailed security requirements document directly contributes the im- plementation of secure robot applications. It is also en- couraged to provide regular security training to developers, which helps them to avoid common coding related errors not necessarily mentioned in the security requirements. 4) Analysis of Security on System-Level: Among the non- software related vulnerabilities which are checked with a help of a check list, the correct implementation of the above mentioned security requirements needs to be tested. One of the test constraints specifi ed in the security re- quirements could be, for example, that a components in- put values should always be verifi ed, because input values coming from other system components cannot be trusted. Adversaries might combine one or more vulnerabilities in order to achieve their goal. Therefore, component as well as integration tests need to always be performed. For these tests well known test methods can be used like e.g. unit testing, fuzzing 11. An external company might be hired to perform a pene- tration test on the robot system if the probable economical damage in case of failure is found to be critical. The output of thi