翻译原文.pdf

A Retrofit Network Transaction Data Logger and Intrusion Detection System for Transmission and Distribution Substations Thomas Morris Electrical and Computer Engineering Mississippi State University Mississippi State MS USA morris ece msstate edu Kalyan Pavurapu Electrical and Computer Engineering Mississippi State University Mississippi State MS USA kalyan234912 Abstract SCADA systems are widely used in electricity generation distribution and transmission control systems NERC CIP 002 009 requires bulk electric providers to secure critical cyber assets electronically and physically Transmission and distribution substations contain cyber critical assets including remote terminal units RTU intelligent electronic devices IED such as relays phasor measurement units PMU and phasor data concentrators PDC Substation critical cyber assets are isolated in electronic security perimeters using firewalls In this paper a retrofit data logger solution for serial communication based MODBUS and DNP3 network appliances is offered The retrofit data logger allows existing control systems to be updated to log network transactions in support of substation based network intrusion detection Substation based intrusion detection supports a defense in depth approach to cyber security in which multiple overlapping layers of security are used to protect critical cyber assets The data logger is an embedded bump in the wire retrofit device which captures time stamps cryptographically signs encrypts and store network traffic Network traffic is forwarded to the existing network Additionally the data logger architecture supports use of signature based and statistics based intrusion detection algorithms at the network appliance edge Keywords SCADA Cyber Security Process Control System Cyber Security Data Logging Intrusion Detection I INTRODUCTION National Electric Reliability Council NERC Critical Infrastructure Protection CIP Standards 002 through 009 require utilities and other responsible entities to place critical cyber assets within an electronic security perimeter The electronic security perimeters must be subjected to vulnerability analyses use access control technologies and include systems to monitor and log the electronic security perimeter access The Federal Energy Regulatory Commission FERC requires responsible entities involved in bulk electricity transmission to adhere to the NERC CIP 002 through 009 standards No such regulation exists for the electric distribution systems and other critical infrastructure such as water treatment and distribution and gas distribution in the United States Electronic perimeter security will minimize the threat of illicit network penetrations however persons with electronic access to SCADA systems within the electronic security perimeter still remain a threat due to the lack of authentication capabilities in these systems Additionally the lack of authentication for process control system communication protocols means that if an attacker does penetrate the electronic security perimeter he will be able to inject false commands and false responses into the process control system without detection Existing remote terminal units and intelligent electronic devices which use serial communication found in transmission and distribution substations do not support data logging of network transactions To provide a substation based intrusion detection system a serial communication data logger is required This paper documents a retrofit SCADA data logger architecture developed to capture and log MODBUS ASCII MODBUS RTU and DNP3 network traffic at the Intelligent Electronic Device IED or Remote Terminal Unit RTU edge The retrofit SCADA data logger has been developed to run on an embedded platform to support use as a retrofit device attached to an IED or RTU and to run on a PC platform Resulting data logs can be used by a substation based intrusion detection system to detect illicit cyber penetrations of the substation communication system and to detect illicit injection of false commands and false responses into the substation communication system The body of this paper includes a section discussing related works a discussion of the needs for data logging and intrusion detection in SCADA control systems at various levels in the SCADA network a description of the retrofit SCADA data logger architecture empirical results from a data logger implementation and a discussion of substation based intrusion detection using the retrofit data logger Finally the paper ends with discussion of future works and conclusions II RELATED WORKS In 1 Chandia et al propose a forensic architecture which can be used to capture SCADA control system 2010 IEEE International Conference on Power and Energy PECon2010 Nov 29 Dec 1 2010 Kuala Lumpur Malaysia 978 1 4244 8946 6 10 26 00 2010 IEEE958 2 communications for subsequent forensic analysis In the Chandia architecture agents capture data at three levels and forward collected traffic to a data warehouse for storage and future analysis Level 1 agents collect communications to and from the control system master nodes Level 2 agents collect traffic at intermediate locations in the network Finally level 3 agents collect communication traffic from downstream nodes such as RTU and IED The level 1 2 and 3 agents capture network traffic and create synopsis of the network packets according to a set of predefined configuration rules Each synopsis contains a time stamp and location details required for the forensic analysis The level agents forward the synopsis packets to the data warehouse which is located in the upstream network The data warehouse analyzes each synopsis packet and creates a data signature which is stored along with the synopsis The data warehouse supports queries on the stored data All the communications between the data warehouse and level agents occur on an isolated side channel network The proposed architecture with level agents and data warehouse is not able to characterize the position of level agent3 in downstream The level 3 agent is most similar to our data logger architecture in that it collects communication at downstream nodes Chandia et al do not specify the exact location intended behavior or architecture of their level 3 agents Our data logger architecture offers a solution for the position behavior and architecture of a level agent 3 Some SCADA control system IED and RTU vendors offer data logger features Control Microsystems Inc offers two SCADA RTU devices with data logging functionality the SCADAPack 350 and SCADAPack 357 These data logging features allow users to connect the RTU to external storage to log process data The data logger function does not support logging network transactions sent to and from the RTU Other vendors offer historians for data logging physical system parameter OSI PI historian is a popular historian product Again control system historians do not offer network transaction data logging Snort is a rule based open source network intrusion detection tool 2 Snort collects and logs network traffic analyzes network traffic searching for rule violations and alerts the administrator of suspicious activity Snort is commonly used to monitor Ethernet and TCP IP communications traffic As such Snort has been applied to monitor control system networks higher layers such as to monitor connections between the control system and the larger corporate network Rule sets have also been developed to allow snort to monitor and analyze MODBUS traffic between master nodes and RTU IED Such implementations can be used on the master nodes where sufficient processing resources are available to run snort RTU and IED do not typically have the processing power or the storage capabilities to support Snort Our data logger is intended to be a low cost solution to log data at the RTU IED level III SCADA SYSTEM OVERVIEW SCADA control systems are distributed cyber physical systems Figure 1 shows an example of an electric transmission SCADA control system Intelligent Electronic Devices IED are connected to sensors and actuators to interface directly with the electric transmission system IED such as protection relays store control parameters and execute algorithmic code such as ladder logic or C programs to directly control sub circuits in the transmission system Transmission faults lead to automated protection response such as opening a circuit breaker Protection relays continuously monitor critical parameters If measured parameters reach a pre programmed trip level the relay will take a control action The relay relay control parameters and the attached circuit breakers form a feed back control loop SCADA systems also support supervisory control and data acquisition SCADA systems include a master terminal unit MTU connected to the IED via a communication link The MTU polls the IED periodically to read physical quantities of the controlled system such a voltage and current This information is displayed on a Human Machine Interface HMI to allow situational awareness and control HMI allow the dispatcher to interact with the physical process For example a dispatcher may open a breaker to island a circuit or close a breaker during system restart The MTU IED communication link HMI and dispatcher form a second supervisory feedback control loop Figure 1 Electric Transmission SCADA Control System The communication link in SCADA systems consists of two parts the communication medium and communication protocols Communication mediums generally include wireless or wired networks Wired networks may use leased line Ethernet serial cable and fiber optic cable Wireless networks may use standardized communication systems such as IEEE 802 11 ZigBee and WirelessHART Wireless links may also use proprietary implementations Finally wireless links may include long distance solutions such as satellite and microwave There are many standards for SCADA communication including Fieldbus EtherIP Profibus MODBUS and Distributed Network Protocol version 3 DNP3 One common security flaw with all of these communication protocols is that they do not include cryptographic authentication which means RTU and MTU cannot validate the origin of commands and responses respectively IED IED C o r p o r a t e N e t w o r k PC PC PC internet firewall HMI MTU Historian firewall serial communication link control center substation NERC CIP electronic security perimeter data logger 959 3 IV ATTACKS ON SCADA SYSTEMS There are 3 primary threats to process control systems response injection command injection and denial of service Response injection attacks inject false responses into a control system Since control systems rely on feedback control loops which monitor physical process data before making control decisions protecting the integrity of the sensor measurements from the physical process is critical False response injection can be used by hackers to cause control algorithms and operators or dispatchers to make misinformed decisions Command injection attacks inject false control commands into a control system Control injection can be classified into 2 categories First human operators oversee control systems and occasionally intercede with supervisory control actions such as opening a breaker Hackers may attempt to inject false supervisory control actions into a control system network Second remote terminals and intelligent electronic devices are generally programmed to automatically monitor and control the physical process directly at a remote site This programming takes the form of ladder logic C code and registers which hold key control parameters such as high and low limits gating process control actions Hackers can use command injection attacks to overwrite ladder logic C code and remote terminal register settings Denial of Service DOS attacks attempt to break the communication link between the remote terminal and master terminal or human machine interface Breaking the communication link between master terminal or human machine interface and the remote terminal breaks the feedback control loop and makes process control impossible DOS attacks take many forms Many DOS attacks attempt to overwhelm hardware or software on one end of the network so that it is no longer responsive Other DOS attacks send ill timed or malformed network packets which cause errors in a remote device s network stack and cause the remote device unresponsive SCADA system attacks may originate from multiple points in the control system network First an attack may be launched via external network connection In this case the attacker penetrates network via network interface to gain access to the control system network Such attacks include penetration via connections to the internet or penetration through dial up connections Second an attacker may penetrate the SCADA the communication link connecting the MTU and IED In 3 Reaves and Morris discuss how to discover and connect to a proprietary SCADA radio used to form the MTU to IED communication link such as that diagramed in Figure 1 and then inject false responses and denial of service attacks into the network traffic Finally an attack may originate from an insider with physical or electronic access to the SCADA system In this case the attacker may inject commands and responses over a network ordinarily isolated from outside connections or an attacker may connect directly to control system equipment before initiating an attack V DATA LOGGING IN A SCADA NETWORK SCADA control system data loggers should monitor and log all communications traffic to and from the MTU and IED Figure 1 shows a SCADA control system with added data logger retrofits This placement of data loggers will capture all network traffic associated with the attacks mentioned in section Error Reference source not found Response injection attacks may originate from an attacker which has penetrated the communication link between the MTU and IED The data logger running on the HMI host in Figure 1 will capture all network traffic associated with such response injection attacks Command injection attacks may originate from a penetration of the corporate network via the internet or from an insider Command injection attacks may also originate from an attacker which has penetrated the communication link between the MTU and IED The data logger attached to the IED in figure 1 will capture all network traffic associated with such command injection attacks Network traffic associated with denial of service attacks against the MTU and IED will also be logged by their respective data loggers Figure 2 Data Logger Architectures Figure 2 provides an architectural overview of the SCADA data loggers as used within a SCADA control system The data logger was built with a hardware abstraction layer to support implementation in a virtual machine on a HMI host PC and as an embedded system Figure 2a shows the embedded system version of the data logger implemented using a field programmable gate array FPGA integrated circuit IC attached to an external compact flash card Figure 2b shows the virtual machine version of the data logger In this version the data logger is implemented as Linux process running in a virtual machine on the same PC which hosts the HMI The virtual machine data logger results are stored on the host PC s hard disk drive rather than on a compact flash card Each version of the data logger contains two RS 232 universal asynchronous receiver transmitters UART to monitor and forward MODBUS ASCII MODBUS RTU FPGA U A R T PROCESSOR U A R T FPGA MTU IED MTU HMI HOST HMI DL VM VIRTUAL SERIAL PORT PHYSICAL SERIAL PORT a b 960 4 and DNP3 link layer protocol data units LPDU For the embedded system version the microcontroller processes bytes as they are received from one UART and forwards the bytes to the other UART Bidirectional communication is supported The microcontroller also executes the link layer software stacks and hands off LPDU to the data logger For the virtual machine version a single physical RS 232 port is used to connect to the MTU A virtual serial port is used to connect the HMI to the virtual machine running the data logger process The virtual machine data logger processes network transactions in the same manner as the embedded system version The data logger forwards each received byte from one UART to the other UART immediately When the link layer detects it has captured an entire LPDU the LPDU is returned to the application layer for logging This provides a significant improvement of data logger architectures which capture entire LPDU before forwarding LPDU to the attached RTU or IED 1 Acquired data must be pre processed to support secure storage and time stamping for post incident analysis Equation 1 shows a logged transaction Tl after preparation for storage The original LPDU is appended with a time stamp tLPDU and a random nonce n The concatenated result is hashed with an HMAC function using key k1 Next the captured LPDU is concatenated with the hash result and the nonce Next the transaction is encrypted AES counter mode This result is stored on the compact flash card or hard disk drive The HMAC and nonce are added to ensure data integrity The time stamp is added to support temporal analysis for both online and offline analysis The HMI host PC should periodically synchronize the real time clock in the data logger with real time clocks in other nodes in the control system to support correlation of data logger results from different points in the system The frequency of time synchronization depends upon the drift of the various clocks in the system The data logger stores transactions on a compact flash card for the embedded version and hard disk drive for the virtual machine version Each logged transaction Tl include the LPDU plus 72 bytes of log information The time stamp tTM measures the time elapsed since 12 00 A M GMT January 1 2009 The time stamp is 8 bytes which provides room to