异常SMTP讯务与Email

1、1 1異常異常SMTPSMTP訊務與訊務與Email SpamEmail Spam的自動通告的自動通告 中央大學中央大學 電算中心電算中心 楊素秋楊素秋 Email: Email: .tw 2 2大大 綱綱 1.1.研究動機研究動機 2.2.異常異常SMTPSMTP訊務的監測訊務的監測 3.3.SpamSpam與異常與異常SMTPSMTP訊務的相關訊務的相關 4.4.Spam Spam 事件的自動通告事件的自動通告 5.5.結論結論3 31.1.研究動機研究動機加速加速 Email Spam Email Spam 通告通

2、告 IP IP 管理資訊查詢管理資訊查詢區網區網 Routing Table Routing Table RWhoisRWhois查詢服務查詢服務Spam event Spam event 的自動通告的自動通告異常異常SMTPSMTP訊務的監測訊務的監測Flow count Flow count 超量超量Packet DensityPacket Density分析超量分析超量SMTPSMTP傳訊主機與通告傳訊主機與通告spam relay/sender spam relay/sender 的相關的相關4 42.2.SMTPSMTP與與 SpamSpam傳訊傳訊SMTP SMTP 傳輸傳輸Cli

3、entClient詢問詢問DNS MX list,DNS MX list,建立信件建立信件delivery routedelivery route紀錄紀錄sendersender與與receiverreceiver間的多個間的多個mail relay/servermail relay/server將將 reverse-pathreverse-path加入加入mail headermail header與與SMTP relaySMTP relay建立雙向連接建立雙向連接, ,沿沿SMTP routeSMTP route傳送信件傳送信件relayrelay收進信件後收進信件後與下一與下一relay

4、 relay 建立連接建立連接/ /轉送信件轉送信件. .最後的最後的deliver relaydeliver relay將信件分送到用戶將信件分送到用戶mailbox. mailbox. 5 5SpamSpamUCE (Unsolicited Commercial Mail) UCE (Unsolicited Commercial Mail) spammerspammer利用自動搜尋程式利用自動搜尋程式持續尋找持續尋找 newsgroup (BBS boards)newsgroup (BBS boards)Join mailing listJoin mailing list網頁的網頁的mai

5、l addressesmail addresses所侵入系統的所侵入系統的mail accountmail accountRegular sequence mail accountRegular sequence mail account重複重複/ /密集寄送廣告信件密集寄送廣告信件 6 6SpammerSpammer以最低的成本以最低的成本, ,透過全球網路傳送超大量廣告信透過全球網路傳送超大量廣告信InternetInternet用戶用戶花費可觀的連線費用花費可觀的連線費用, ,時間與精力下載時間與精力下載/ /收取收取/ /刪除刪除大量大量spam.spam.ISPISP耗費更龐大的網路

6、與系統資源重複傳送耗費更龐大的網路與系統資源重複傳送junk mailsjunk mails影響影響mailmail的正常收送的正常收送 7 7為避免回覆大量的為避免回覆大量的spam complainspam complainSpammerSpammer藉由自動搜尋程式藉由自動搜尋程式尋找未設防的尋找未設防的SMTP server SMTP server 作為作為spam spam relay/senderrelay/sender傳送廣告信件往蒐集的傳送廣告信件往蒐集的newsgroup/mailing listnewsgroup/mailing list及及mail accountsmai

7、l accounts甚至透過甚至透過mailmail夾檔散播病蟲或攻擊程式夾檔散播病蟲或攻擊程式侵入網路主機侵入網路主機. .集結更大量的感染主機集結更大量的感染主機寄發寄發/ /轉送更大量的轉送更大量的spam. spam. 8 8減緩減緩SpamSpam倍數成長的主要途徑倍數成長的主要途徑(1)(1)回報回報/ /檢舉檢舉Spam eventSpam event減少一個減少一個 spam relay/senderspam relay/sender減少減少millions of spamsmillions of spams(2)(2)監測可能的監測可能的spammerspammer主機及訊務

8、主機及訊務SMTPSMTP訊務量測訊務量測篩選異常訊務量篩選異常訊務量9 9回報回報/ /檢舉檢舉Spam eventSpam event連網中心建立連網中心建立abuse Emailabuse Email帳號帳號abusedomain, spamdomain, securitydomainabusedomain, spamdomain, securitydomain接受所轄接受所轄IPIP主機的主機的Spam/ JunkSpam/ Junk通告信通告信. . 網路用戶網路用戶依據依據spam route,spam route,萃取發送主機與萃取發送主機與relay serversrelay

9、servers “Received:”, “From:” “Received:”, “From:” 紀錄項紀錄項回應給發信主機與回應給發信主機與relay serverrelay server擁有者擁有者ReportReport給給spam report sitespam report siteEX: EX: 1010偵測可能的偵測可能的spammerspammer主機及訊務主機及訊務依據依據Spam Spam 傳訊特徵傳訊特徵, ,實作異常實作異常SMTPSMTP訊務的統計訊務的統計IntensiveIntensive Obviously high SMTP connection count

10、Obviously high SMTP connection countIteration Iteration last for several hours last for several hours協助管理者監測異常的協助管理者監測異常的mailmail訊務訊務據以據以Check /var/log/maillogCheck /var/log/maillog據以據以Check user mailbox Check user mailbox 預先發現感染主機預先發現感染主機, , 通告用戶修補漏洞通告用戶修補漏洞1111通告的通告的Email Spam (2003Email Spam (200

11、3年年 7 7月至月至 1111月月) )桃園區網每月處理的桃園區網每月處理的Spam mailSpam mail通告主機總數通告主機總數. .主要的主要的abuseabuse通告信件通告信件SS 通報通報廣告郵件的廣告郵件的 relay server/senderrelay server/sendermyNetWatchmyNetWatch 通報通報CodeRed/NimdaCodeRed/Nimda感染主機感染主機(80/TCP)(80/TCP)SYN Flooding (445/TCP, 17300/TCP, SYN Flooding (445/TCP, 17300/TCP, ) ) 環

12、球或派拉蒙製片環球或派拉蒙製片通告侵犯智財權的通告侵犯智財權的eDonkeyeDonkey主機及其影片檔存放主機及其影片檔存放 Others Others 1212Table 1 通告的區網Abuse主機數分布 Spam Spam HostsHosts SYN SYN FloodingFloodingInfringerInfringerHostsHostsJul Jul 5 518186 6Aug Aug 151522225 5Sep Sep 20200 09 9OctOct11113 36 6NovNov7 71 1121213133 3異常異常SMTPSMTP訊務的監測訊務的監測異常異常S

13、MTPSMTP訊務的監測訊務的監測SpamSpam傳訊特徵傳訊特徵IntensiveIntensiveObviously high frequency of SMTP connectionsObviously high frequency of SMTP connectionsIteration Iteration Last for Many hours Last for Many hours Mean Packet sizeMean Packet sizeLittle than 100 Bytes per PacktLittle than 100 Bytes per PacktMore th

14、an 100 Bytes per packetMore than 100 Bytes per packet1414Transportation Traffic Logs Transportation Traffic Logs all network operators depend on the all network operators depend on the quantifiable traffic log data to quantifiable traffic log data to evaluate the network performanceevaluate the netw

15、ork performanceTCPDUMPTCPDUMPNetFlow, sFlowNetFlow, sFlowOthers Others 1515TcpdumpTcpdumpa raw packet capture program.a raw packet capture program.Gather the layer 4 transportation traffic logs Gather the layer 4 transportation traffic logs throughthroughThe dump transport traffic logs involved The

16、dump transport traffic logs involved the the detail fields of each IP packet headerdetail fields of each IP packet headersource/destination IP addresses, source/destination IP addresses, source/destination application ports,source/destination application ports,protocol identity,protocol identity,num

17、ber of packets, number of packets, number of bytes, number of bytes, TCP operatorsTCP operators1616NetflowNetflow router router 轉送訊務紀錄轉送訊務紀錄Flow-based layer 4 transport traffic logFlow-based layer 4 transport traffic logSource & destination IP addressSource & destination IP addressSource &am

18、p; destination application port Source & destination application port Source & destination interface#Source & destination interface#protocol identifierprotocol identifierpacket countpacket countbyte countbyte count1717利用利用NetflowNetflow log log統計區網的異常統計區網的異常SMTPSMTP訊務訊務Accumulate SMTP se

19、rv_flowAccumulate SMTP serv_flow connection counts connection counts statisticsstatisticsNetflowlogNetflowlog gathered from router of aggregate gathered from router of aggregate networknetworkThreshold_100_flowThreshold_100_flow Less than 100 connections: 99.72 %Less than 100 connections: 99.72 % Mo

20、re than 100 connections: 0.28 %More than 100 connections: 0.28 %Threshold_30_flowThreshold_30_flow Less than 30 connections: 98.61 %Less than 30 connections: 98.61 %1818Table 2. 區網的SMTP Flows 特徵項分布Smtp_flowSmtp_flowcountcountFlow #/RatioFlow #/RatioByte Byte RatioRatio1 101 10136003 (94.78 %)136003

21、(94.78 %)73.1 %73.1 %11 30 11 30 5502 (3.83 %)5502 (3.83 %)12.5 %12.5 %31 70 31 70 1370 (0.95 %)1370 (0.95 %)8.1 %8.1 %71 100 71 100 231 (0.16 %)231 (0.16 %)1.1 %1.1 %101 200 101 200 226 (0.16 %)226 (0.16 %)1.2 %1.2 %201 1000201 1000145 (0.10 %)145 (0.10 %)1.8 %1.8 % 1000 100015 (0.01 %)15 (0.01 %)2

22、.2 %2.2 %1919SMTPSMTP訊務的統計訊務的統計/ /監測監測Monitor Abnormal SMTP Traffic of Monitor Abnormal SMTP Traffic of smtp_flowsmtp_flowi iCombine Several NetFlowCombine Several NetFlow features features SMTP service port & Src_IP & Dst_IPSMTP service port & Src_IP & Dst_IPsrc_IPsrc_IPdst_IP.(25)d

23、st_IP.(25)src_IP.(25)dst_IPsrc_IP.(25)dst_IP2020統計統計/ / 監測異常的監測異常的 SMTP SMTP 訊務訊務累計累計SMTP SMTP 訊務變量訊務變量透過透過 IP protocol_idIP protocol_id & application port & application port的比對的比對, ,累計累計 flowflowsmtp_flowsmtp_flowi i pktpktsmtp_flowsmtp_flowi i bytebytesmtp_flowsmtp_flowi i 排序排序/ /篩選超量的篩選超量

24、的syn_flowssyn_flows訊務訊務Monitoring SMTP TrafficMonitoring SMTP Traffic PHP + ApachePHP + Apache2121222223232424Nov 3 20:25:58 smtp3 sendmail7645: ID 801593 hA3CPot1007645:from=, size=64607, class=0, nrcpts=1,msgid=, proto=SMTP,daemon=MTA, relay=53Nov 3 20:25:58 smtp3 sendmail764

25、5: ID 801593 hA3CPot1007645:to=, delay=00:00:06, mailer=relay, pri=30258,stat=queuedNov 3 20:26:45 smtp3 mailscanner3948: Virus W32/Yaha-P found infile ./hA3CPot1007645/disney.zip/DOCUME1DennisLOCALS1Tempsetup.exeNov 3 20:26:51 smtp3 sendmail7958: ID 801593 hA3CPot1007645:to=, de

26、lay=00:00:59, xdelay=00:00:00, mailer=relay,pri=120258, relay=9 9, dsn=2.0.0, stat=Sent(hA3CP8k1016181 Message accepted for delivery)Nov 3 20:27:00 smtp3 mailscanner3948: Virus W32/Yaha-P found infile ./hA3CPot1007645/disney.zip/DOCUME1DennisLOCALS1Tempsetup.exe252526262727sy

27、slog:Oct 26 08:24:25 smtp3 sendmail13433: ID 801593 h9Q0ON2a013433: from=, size=6998, class=0, nrcpts=1, sgid=, proto=SMTP, daemon=MTA, relay= 1 (may be forged)syslog:Oct 26 08:24:25 smtp3 sendmail13425: ID 801593 h9Q0ON2a013425: from=, size=6994, class=0, nrcpts=1, sg

28、id=, proto=SMTP, daemon=MTA, relay= 5 (may be forged)syslog:Oct 26 08:24:25 smtp3 sendmail13435: ID 801593 h9Q0ON2a013435: from=, size=6971, class=0, nrcpts=1, sgid=, proto=SMTP, daemon=MTA, relay= 1 (may be forged)syslog:Oct 26 08:24:25 smtp3 sendmail13432: ID 801593

29、 h9Q0ON2a013432: from=, size=6995, class=0, nrcpts=1, sgid=, proto=SMTP, daemon=MTA, relay= 4 (may be forged)syslog:Oct 26 08:24:25 smtp3 sendmail13434: ID 801593 h9Q0ON2a013434: from=, size=6965, class=0, nrcpts=1, 2828Mail Relay TestingMail Relay Testingmrtmrtftp:/ v

30、 test.patterns test.message host_ip_add./mrt v test.patterns test.message host_ip_add2929 ann# ./mrt -v ./test.patterns ./test.message 45mrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting

31、: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection re

32、fusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 163.2

33、5.121.245: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refusedmrt: 45: Error connecting: Connection refused3030ann# ./mrt -v ./test.patterns ./test.message mrt: : Mess

34、age acceptedmrt: : Message acceptedmrt: : Message acceptedmrt: : SMTP error (553) reading MAIL responsemrt: : Message acceptedmrt: : Message acceptedmrt: : Message acceptedmrt: : Message acceptedmrt: : Message ac

35、ceptedmrt: : Message acceptedmrt: : Message acceptedmrt: : Message acceptedmrt: : Message acceptedmrt: : Message acceptedmrt: : Message acceptedmrt: : Message acceptedmrt: : Message acceptedmrt: : SMTP

36、 error (553) reading MAIL response3131ann# ./mrt -v ./test.patterns ./test.message 28mrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP

37、error (550) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP error (553) reading RCPT responsemrt: 28: SMTP error

38、 (553) reading RCPT responsemrt: 28: SMTP error (553) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP error (550

39、) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT responsemrt: 28: SMTP error (550) reading RCPT response3232數據分析數據分析60 % 60 % 通告通告spam relay/senderspam relay/sender可由統計的異常可由統計的異常SMTPSMTP主機佇列中篩選得主機佇列中篩選得7

40、7月份的月份的60%60%8 8月份的月份的60% 60% 9 9月份的月份的60%60%1010月份的月份的100%100%1111月份的月份的100%100%異常異常SMTP/SYN FloodingSMTP/SYN Flooding訊務監測訊務監測發現發現Spam & Spam & 網路侵擾訊務網路侵擾訊務3333Table 2 區網Abuse host分布(2003年)Abnormal SMTP TrafficAbnormal www /SYN FloodingJul 60 %60 %43 %43 %Aug 60 % 60 % 48 %48 %Sep 60 % 60 %

41、 - -Oct 55 % 55 %100 %100 %Nov 100 % 100 %100 %100 %34344 Spam 事件的自動通告 Spam/Spam/攻擊訊務通告事件攻擊訊務通告事件倍數成長的倍數成長的spam spam 通告通告超量的異常超量的異常 SMTP Traffic SMTP Traffic 網路管理者網路管理者非常依賴非常依賴IPIP管理資訊查詢系統管理資訊查詢系統通告感染主機用戶與管理者通告感染主機用戶與管理者, ,修補系統修補系統自動阻斷攻擊訊務自動阻斷攻擊訊務, ,防堵攻擊訊務的持續擴散防堵攻擊訊務的持續擴散3535spam mailspam mail的自動通告系

42、統的自動通告系統自動自動Query Query IPIP管理資訊管理資訊,Email,Email通告通告藉由藉由SNMP pulling router ipRouteSNMP pulling router ipRoute MIB, MIB, 快速萃取連網的龐大快速萃取連網的龐大 routingrouting資訊資訊建立建立IPIP管理資訊查詢服務管理資訊查詢服務依據依據 NextHopNextHop integrate integrate The extracted Routing TableThe extracted Routing Table 連線單位通訊資訊檔連線單位通訊資訊檔 RWho

43、isRWhois IP IP管理資料庫管理資料庫 3636ipRouteipRoute SNMP MIB SNMP MIB儲存連網單位的儲存連網單位的routing routing 資訊資訊Network address Network address NetMaskNetMask辨識號辨識號 .....1.11NextHopNextHop 辨識號辨識號 .....1.7Mansfield G. Mansfield G. 曾藉由曾藉由ipRouteipRoute MIB M

44、IB重複搜尋各層重複搜尋各層routers ipRouterouters ipRoute MIB MIB 自動構建區域網路拓樸自動構建區域網路拓樸 3737重複萃取網段重複萃取網段IPIP位址與對應的位址與對應的NetMask/ NextHopNetMask/ NextHop位址位址分別以分別以IPIP網段位址網段位址index,index,儲存儲存NetMaskNetMask List ListNextHopNextHop List. List. 結合結合NetMask ,NextHopNetMask ,NextHop 與與SegmentSegment佇列佇列快速重建龐大的區網快速重建龐大的

45、區網ip_routingip_routing 紀錄存檔紀錄存檔3838ipRouteMask OIDip.ipRouteTable.ipRouteEntry.ipRouteMask. = IpAddress: ip.ipRouteTable.ipRouteEntry.ipRouteMask. = IpAddress: ip.ipRouteTable.ipRouteEntry.ipRouteMask. = IpAddress: ip.ipRoute

46、Table.ipRouteEntry.ipRouteMask. = IpAddress: ipRouteNextHop OID ip.ipRouteTable.ipRouteEntry.ipRouteNextHop. = IpAddress: 2ip.ipRouteTable.ipRouteEntry.ipRouteNextHop. = IpAddress: 11ip.ipRouteTable.ipRouteEntry.ipRouteNextHop.19

47、 = IpAddress: 16ip.ipRouteTable.ipRouteEntry.ipRouteNextHop. = IpAddress: 113939NextHop Dest. Netmask Seg =26, ,, 256 , , , 256 1, , , 256 37, 140

48、.138.0.0, , 25609, , , 409, , , 44040IPIP邏輯位址不包含任何管理資訊邏輯位址不包含任何管理資訊RouterRouter藉由藉由routing tablerouting table的查詢的查詢依據依據 NextHopNextHop 紀錄紀錄 switch packetswitch packetSwitch Switch 往正確的往正確的 routing interfacerouting interf

49、ace4141RWhoisRWhois分享軟體分享軟體利用利用Mark KostersDataBaseMark KostersDataBase (MKDB) (MKDB) 支援資支援資料的管理與查詢料的管理與查詢. .資料庫查詢伺服程式資料庫查詢伺服程式rwhoisdrwhoisd資料庫建置程式資料庫建置程式rwhoisd_indexerrwhoisd_indexer4242RWhoisRWhois Server Server藉由藉由IPIP管理資料庫伺服系統的建置管理資料庫伺服系統的建置, ,作為自動作為自動通告通告Spam Spam 的基礎的基礎. .讀取讀取routingrouting紀錄紀錄, ,依據依據NexthopNexthop 紀錄比對紀錄比對/ /萃取對萃取對應的管理聯絡資訊檔應的管理聯絡資訊檔構建構建RWhoisRWhois network schema network schema關聯紀錄檔關聯紀錄檔建立資料庫建立資料庫indexing, index