网路安全核心技术之现况与发展趋势ppt课件

1、網路平安中心技術之現況與發展趨勢網路平安中心技術之現況與發展趨勢 黃能富教授黃能富教授高速網路實驗室高速網路實驗室清華大學資訊工程學系清華大學資訊工程學系/通訊工程研讨所通訊工程研讨所: .twAgendavIntroduction of Network SecurityvContent Inspection TechnologiesvPattern Matching AlgorithmsvFlow Classification by Stateful MechanismvOpen Issues - - 駭客無所不在駭客無所不在 -v2000/3：駭客利用：

2、駭客利用DDos的網路攻擊方式，引起的網路攻擊方式，引起Yahoo、Amazon、CNN、eBay 等知名網站癱瘓等知名網站癱瘓v2001/7：Amazon 旗下的旗下的 Bibliofind 遭駭客盜走顧客的信誉卡資料遭駭客盜走顧客的信誉卡資料 v2002 中美駭客大戰中美駭客大戰v2003/1 SQL Slammer 攻擊攻擊v2003/4 大陸流光後門程式大陸流光後門程式v2003/8 Blaster 疾風病毒攻擊疾風病毒攻擊v2003/9 SoBig 老大病毒攻擊老大病毒攻擊v2003/9 大陸網軍攻擊大陸網軍攻擊v2004/3 Netsky 天網病毒攻擊天網病毒攻擊v2004/4 S

3、asser 殺手病毒攻擊殺手病毒攻擊v2005/5 國內大考中心遭駭客竄改資料國內大考中心遭駭客竄改資料v2005/6 外交部網站遭大陸網軍後門程式竊取外交機密外交部網站遭大陸網軍後門程式竊取外交機密網路平安的隱憂網路平安的隱憂v網路攻擊技術日新月異，攻擊工具易於获得網路攻擊技術日新月異，攻擊工具易於获得,界面界面淺顯易懂，不需高深技巧，即可進行攻擊。淺顯易懂，不需高深技巧，即可進行攻擊。v網路攻擊已不侷限於侵入動作，許多攻擊行為旨網路攻擊已不侷限於侵入動作，許多攻擊行為旨在阻斷網站之服務才干。在阻斷網站之服務才干。v網路通訊設備平安性缺乏。路由器及交換器僅能網路通訊設備平安性缺乏。路由器及交

4、換器僅能檢視封包第三層資訊。檢視封包第三層資訊。v防火牆著重在封包第四層資訊檢查。防火牆著重在封包第四層資訊檢查。v防毒軟體逐漸無法辨識網路攻擊。防毒軟體逐漸無法辨識網路攻擊。網路攻擊工具範例網路攻擊種類網路攻擊種類vDenial of Service (DoS), Distributed Denial of Service (DDoS)vNetwork InvasionvNetwork ScanningvNetwork SniffingvTorjan Horse and BackdoorsvWormP2P/IM 網安威脅網安威脅vP2P (Peer-to-Peer) 分享程式分享程式vIM

5、(Instant Messenger) 即時通即時通vSpyware 間諜軟體間諜軟體vAdware 廣告軟體廣告軟體vTunneling 私人隧道私人隧道P2P: A new paradigmvBottleneck of ServervPowerful PCvFlexible, efficient information sharingvP2P changes the way of Web (Internet) P2P即將破壞現存的資安架構即將破壞現存的資安架構vP2P 除了檔案分享與即時通訊，也逐漸發展出不同應用，例如除了檔案分享與即時通訊，也逐漸發展出不同應用，例如 SoftEther

6、和和 Skype。對個人用戶，利多於弊，但對企業，為資訊平。對個人用戶，利多於弊，但對企業，為資訊平安一大隱憂安一大隱憂vP2P 應用潛藏諸多風險，包括應用潛藏諸多風險，包括v洩漏企業內部機密資訊洩漏企業內部機密資訊v成為病蟲擴散的管道成為病蟲擴散的管道v下載非法檔案下載非法檔案v进犯著作權进犯著作權v佔用大量網路頻寬佔用大量網路頻寬v影響其他系統正常運作影響其他系統正常運作v呵斥員工分心，降低生產力呵斥員工分心，降低生產力 Famous P2P ExamplesvBitTorrentv eZpeervKuroveDonkeyveMulevMLdonkeyvGnutellavKazaa/Mor

7、pheusv Shareazav Direct-connectv Gnutellav Soulseekv Opennapv Worklinkv Opennextv Jelawatv PP點點通v SoftEtherv iMESHv MIBv WinMixv WinMulev SkypeInstant Messenger (IM)vMSNvYahoo MessengervICQvYamQQvAIM (AOL IM)網路攻防技術發展趨勢vIDP/IPS (Layer-7)vApplication Firewall (Layer-7)vNetwork Access Control (NAC)vDef

8、ense-in-Depth/Security SwitchA Generic Layer-7 EnginevPacket NormalizervMakes sure the integrity of incoming packetsvEliminates the ambiguityvDecodes URI strings if necessaryvPattern-Matching EnginevPolicy EnginevGather information from pattern-matching engine and issue the verdict to allow/drop the

9、 packetsPacket NormalizervIntegrity CheckingvIP Fragment ReassemblevTCP Segment ReassemblevTCP Segments may come out-of-ordervSEQ out of window sizevSegment OverlappingvURI DecodevURI hex code obfuscation (a = %61)vURI unicode/UTF-8 obfuscationvself-referential directories obfuscation (/././././ = /

10、)vdirectories obfuscation (/abc/a/./a/./a/ = /abc/a)Pattern-Matching EnginevThe most computation-intensive task in packet processing. Normally the PM engine needs to process every single byte in packet payload.vIn Snort, the PM routine accounts for 31% of the total execution timePattern Matching is

11、Expensive!30 Instructions/ Byte. 45K Instructions/1500 Byte packet50 Instructions/ 1500 Byte packetSource: Intel Corp.Content Inspection TechnologiesvPattern-Matching AlgorithmsvSoftware BasedvBoyer-Moore vAho-Corasick (AC)vWu-Manber vHardware BasedvBloom-Filter vReconfigure Hardware (FSM)vTCAM-base

12、dPattern Matching Problem DefinitionvGiven an input text T = t0, t1, , tn ,and a finite set of strings P = P1, P2, , Pr, the string matching problem involves locating and identifying the substring of T which is identical to Pj = , 1 j r, where vts+i = , 0 i m-1. And this equation can be also denoted

13、 asvtsts+m-1 = jiajmjaa10.G C A T C G C A G A G A G T A T A C A G T A A GTextG C A G A G A GAho-Corasick (AC) AlgorithmvAC is a classic solution to exact set matching. It works in time O(n + m + z) where z is number of patterns occurrences in T.vAC is based on a refinement of a keyword tree.vAC is a

14、 deterministic algorithm. That is, the performance is independent of the number of patterns.An Example of AC AlgorithmvExample: P = ab, ba, babb, bbAn example of AC AlgorithmDashed: fail transitions; those not shown leads to the rootheehrsisshershe, shehisshsheh!=h,sPatterns:hershissheAn example of

15、AC AlgorithmheehrsissText: h e i s h i sheishisReconfigure Hardware (FSM) vImplement the AC FSM in configurable Logic Elements (LEs) of FPGA.vAchieve multiple gigabit performance. (Depends on the FPGA model)vA powerful FPGA is necessary to accommodate thousands of patterns, so that its not practical

16、 and visible in commercial market.FPGA-based pattern matchingvFPGA-basedBloom FiltervGiven a string X, the Bloom filter computes k hash functions on it producing k hash values ranging from 1 to m. The same procedure is repeated for all the members of the pattern set.vThe input text is verified by ge

17、nerating k hash values in the same way. If at least one of these k bits is found not set then the string is declared to be impossible to match.vPatterns in Length n are grouped into Bn.Bloom Filter (Cont.) 1 2 3 4 5 6 7 8 9 Payload StreamA B C D E F G H I JB2 B3 B4BwFalse positive :Mim f = (0.5)K, w

18、hile m = (k x n) / Ln2 So, total space, sum(Bi) = m x (w - 1) if k = 1, n = 2048, m = 3072 bits k = 1, n = 3072, m = 4608 bits if k = 4, f = 0.0625 k = 5, f = 0.0313 k = 6, f = 0.0156 Bloom Filter (B4) Bloom Filter (B3) Bloom Filter (B2)11110m0m0mH1H2H3Hk0m11111111Group signature by length :G2 (X)G3

19、 (X)G4 (X)K Hash functions H1, H2, , HkTCAM fundamental TCAM stores data with three logic values: 0, 1, X (dont care) Multiple match modes are needed.Policy EnginevCollect the matching events from Pattern-Matching Engine.vClarify the relationship between matched patterns:vOrdered: A policy may consi

20、sts more than one pattern and should be matched in order.vOffset, Depth: The matched position should be within a certain range or location.vDistance, Within: The distance between two matched patterns should be taken into consideration also.vTrace Application StatesvSome applications are difficult to

21、 identify by using only one signature (e.g. P2P). Policy Engine needs to track the connection state like the following diagram:S0S1S2S3Msg ExchangeRequest FileData ExchangeContent Inspection TechnologiesvOur Pattern Matching AlgorithmsvHierarchical Matching Algorithm (HMA) for Intrusion Detection Sy

22、stems (IEEE Globecom2005)vA Time and Memory Efficient String Matching Algorithm for Intrusion Detection Systems (IEEE Globecom2006)vA Pattern Matching Coprocessor for Deep and Large Signature Set in Network Security System (IEEE Globecom2005)vA Fast Pre-filtering Algorithm for Pattern Matching (IEEE

23、 Globecom2006)vFlow Classification by Stateful MethodsvIM/P2P Classification v Hierarchical Matching Algorithm (HMA) for Intrusion Detection Systems vHMA is a two-tier and cluster-wise matching algorithm vReduce the amount of external memory access vReduce the access delayvReduce the required proces

24、sing cycle timevImprove the performance of IDSvLow memory requirement v1.763 times better than the state-of-the-art algorithms vEnable an efficient and cost-effective real-time IDS (1).H1(a)H1(b).H1(e)H1(f)H1(g)H1(h)H1(i).H1(z)On-chip Cache Memory (H1)DRAM (H2) pid fid(1)a(2)red(3)orange(4)green(5)y

25、ellow(6)blackF=a, e(a,a)(a,b)(a,c)(a,n)(a,z).red yellow(e,a)(e,b)(e,c)(e,l)(e,z).(e,d).green(e,e)black.orange.blblacack k.FCSExternal MemoryPre-filterFast SearchCluster-wise String SearchNarrow Searching DomainHierarchical Matching Algorithm (HMA) for Intrusion Detection SystemsHMABM-PHBMHAC-CMemory

26、 326.75 KB16.013 MB313.2 KB439 KBShiftRegByteCounterCentralControlUnitPatternID/PositionQueueHostTCAMPostPorcessorWBytesSystemArchitecturePattern Matching Coprocessor for Deep and Large Signature Set in Network Security SystemSetofAddressComparatorEnableRegisterDownloadPatternIDSubpatternNumberPatte

27、rnIDPatternPositionByteCounterPatternTableSelectorclearShortPatternPIDMatchAddrCAMhitPE0PEnPIDLongPatternPIDCentralControlUnitPattern Matching Coprocessor for Deep and Large Signature Set in Network Security SystemModuleResource UsageSelector530 LEs ( 1% of total LEs)PE150 32 Les ( 26% of total LEs)

28、Pattern Table22K bits ( 9% of memory )I/O Pin210 ( 50% of total pins)FPGAImplementationResults020406080100120140160248163264128 256TCAMWidth(Bytes)MaxPECase1Case2Case3Case4010203040506070248163264128 256TCAMWidth(Bytes)AvgMaxPECase1Case2Case3Case4SimulationResultsPattern Matching Coprocessor for Dee

29、p and Large Signature Set in Network Security SystemPre-filter: Search Filter ModelvAll the substrings that filtered by the filter are clear and impossible to contain any of the defined patterns. vAnd those substrings passed to the pattern matching algorithm may or may not contain pre-defined patter

30、ns. vThus, the search filter may generate false positive but not false negative.vThe false positive here refers to the case that a substring without any pre-defined patterns is falsely detected and accepted as with. vAn exact string matching mechanism is essential for finding out which patterns are

31、included in the accepted substring. EnteringInput stringPre-Filtering AlgorithmString Matching AlgorithmAccepted substrings, may or may not contain patternBypassed substrings, without any patternPatterns found in the substringEnteringInput stringPre-Filtering AlgorithmString Matching AlgorithmAccept

32、ed substrings, may or may not contain patternBypassed substrings, without any patternPatterns found in the substringPre-filter: Search Filter ModelSuper-Symbol FiltervThe basic idea of the proposed Super-Symbol Filter (SSF) algorithm is to treat two bytes data as a super-symbol, and the using of bit

33、map to indicate the occurrence of each super-symbol in the pre-defined patterns.For example, for the 8-bit ASCII-code, there are 65536 combinations of two bytes data, and a bitmap vector of 65536 entries is used.Match Vector Constructing Filtering phase in SSF-1 Algorithm Input String Text= ABOD COD

34、ING IS FOODDOCDOOFBADOBitmapAAABCODEFOODOOZZ001101010101010000ABBOODDC CCOODDIINNGGIISSF FFOOOODABBOODDC CCOODDIINNGGIISSF FFOOOOD10100 01100000 0000 0111SSF-2 AlgorithmvTo have better accuracy and less number of false positives, the extended SSF-2 algorithm, two match vectors are employed.vThe Firs

35、t Match Vector (FMV) is used for the super-symbols being conjugated by the first two symbols in each of the patterns.vThe Rest Match Vector (RMV) is used for the rest super-symbols in the patterns except those in the FMV. SSF-2 AlgorithmvThe algorithm looks up the FMV and RMV and detects whether the

36、 corresponding bit of each super-symbol is 1.vSince “AB and “OD are not the beginning super-symbol of any patterns (by checking FMV), the filter algorithm only outputs two substrings “COD and “FOOD. And only one substring “COD is false positive in this case.EvaluationvTo evaluate the scalability and

37、 flexibility, the popular Snort IDS signatures are employed. vIn case most bits of the bitmap are set as 1, we can expect that the SSF filtering performance will be impacted dramatically as the “hit rate will be very high.vFortunately, by tracking the growing paths of Snort rule patterns, the percen

38、tage of setting bits for the MV, FMV, and RMV is still very small (less than 5%). Thus, the proposed approaches have a great chance to adopt the fast growth of Snort releases.Number of Released PatternsSSF-1MV bitmapSSF-2FMVbitmapSSF-2RMV bitmapSnort-2.0 206632136953027Snort-2.1 261734788133296Snort

39、-2.2 266435758353382Snort-2.3 267936118453413Snort-2.4 268036118453413Defcon9 TraceFilter-AlgorithmPassed by Filter Filter outpercentageFilter cost timeACsearchcost timeTotal cost timeThroughputDefcon-1# of matched patterns : 377,508 times(9,846,572 bytes)PBF1,173,91888%10710710710710710710IDP9,775,

40、9240.8%125,810512,169637,970123AC9,852,3420%0513,081513,081153SSF-11,350,54186%117,00080,374197,374400SSF-2391,02496%126,52329,739156,262504PerformancePentium-4 3.0 GHz personal computer with 1MB level-2 cache, and installed with Intels VTune tool Parallel Bloom Filter (PBF), Database Processor (IDP

41、)Filter Percentage & ThroughputvThe filtering effectiveness of IDP scheme is pretty bad and is not capable to handle Snorts patterns. This is due to the bitmap used in the IDP scheme has only 256 entries for one byte symbol.vAnd most of the entries of are set as “1 for the Snorts patterns.vBoth

42、PBF and SSF schemes are less sensible to the growth of patterns and have a filtering percentage around 80-98%. Filter Percentage & ThroughputvThe PBF is only suitable for hardware-based implementation, the throughput of PBF is less than that of AC. vWe can see that for the Defcon-1, the system t

43、hroughput is around double speed-up (270Mbps vs 141Mbps) compared to that of original AC algorithm, and for Defcon-3, the system throughput is even more than three times speed-up (504Mbps vs 153Mbps).vThe proposed SSF schemes consume far less memory (cache-resident).Flow Classification Using Statefu

44、l MethodThe FA Example : FTPThe FAs of BitTorrent protocols. The FAs of Yahoo Messenger protocol. 網路平安研讨主題vDoS/DDoS vContent Inspection AlgorithmsvZero-day Attacks vWeb SecurityvNetwork Access Control (NAC)vWireless SecurityZero-day Attacks10 Jan, 200629 Dec, 200528 Dec, 2005MS WMF exploit publicly releasedBroadWeb released pattern updateMicrosoft released patch 11 April, 200626 Mar, 200624 Mar, 2006Vulnerability was publicly unveiledBroadWeb released pattern updateMicrosoft released