飞塔防火墙-诊断课件

FortiGateIITroubleshootingFortiGate5.2.1FortiGateIITroubleshootingFor目标识别网络常规行为监控非正常行为如流量突发或非典型性协议Troubleshoot物理和逻辑网络接口理解会话表使用“diagnosedebugflow”来对流量流向进行排错对资源使用问题进行排错,如当防病毒和IPS打开时高CPU或高内存占用测试没有保存到flash的OS目标识别网络常规行为在任何问题发生之前定义正常行为(基线):CPU使用率Memory使用率流量等级流量如何走向(流量)使用了哪些协议和TCP/UDP端口流量模式和分布Why?如果你知道什么是正常流量,识别非正常流量会更容易NowBaseline(Average)NormalRangeAbnormal在任何问题发生之前定义正常行为(基线):NowAbnorma网络图为何需要网络图?没有网络图，解释和分析复杂网络是困难且耗时的物理图包含所有物理网络接口,连线和端口对Layer1/2/3的问题很有效逻辑图包含路由器,逻辑设备(VDOMs)和UTM对Layer3+的问题很有效2019:db8::b108port2/24port4/27port1/8port3网络图为何需要网络图?2019:db8::b108192监控数据流&资源使用情况获取正常的网络数据

在发生问题之前不正常的行为非常难发现–除非知道什么是正常的CPU使用率RAM使用率允许通过的应用入和出的带宽…工具SNMPAlertemailLogging/SyslogFortiAnalyzer或者第三方SIEM(systeminformation&eventmanagement)Dashboard/getsystemstatusNormalTrafficspikes监控数据流&资源使用情况获取正常的网络数据在发生问题之SNMPAllowedsourceofqueriesSNMPAllowedsourceofqueries通过SNMP获取事件通知trapDestination触发FortiGatet发送SNMP消息的事件通过SNMP获取事件通知trapDestination触发#getsysstatusVersion:FortiGate-VM64v5.2.0,build0589,140613(GA)Virus-DB:22.00856(2019-09-2405:33)ExtendedDB:1.00000(2019-10-1715:46)IPS-DB:5.00549(2019-09-2300:49)IPS-ETDB:0.00000(2019-01-0100:00)Serial-Number:FGVM040000025212BotnetDB:1.00736(2019-08-2410:18)LicenseStatus:ValidVMResources:1CPU/4allowed,969MBRAM/6144MBallowedBIOSversion:04000002Logharddisk:AvailableHostname:STUDENTOperationMode:NATCurrentvirtualdomain:rootMaxnumberofvirtualdomains:10Virtualdomainsstatus:1inNATmode,0inTPmodeVirtualdomainconfiguration:disableFIPS-CCmode:disableCurrentHAmode:standaloneBranchpoint:589ReleaseVersionInformation:GAFortiOSx86-64:YesSystemtime:ThuOct900:26:542019#getsysperfstatCPUstates:2%user15%system0%nice83%idleCPU0states:2%user15%system0%nice83%idleMemorystates:44%usedAveragenetworkusage:542kbpsin1minute,1050kbpsin10minutes,512kbpsin30minutesAveragesessions:7sessionsin1minute,5sessionsin10minutes,5sessionsin30minutesAveragesessionsetuprate:0sessionspersecondinlast1minute,0sessionspersecondinlast10minutes,0sessionspersecondinlast30minutesViruscaught:0totalin1minuteIPSattacksblocked:0totalin1minuteUptime:0days,0hours,19minutes系统信息&资源使用情况#getsysstatus#getsysperf#diagnosefirewallstatisticshowgettingtrafficstatistics...Browsing:328packets,132562bytesDNS:797packets,127917bytesE-Mail:0packets,0bytesFTP:0packets,0bytesGaming:0packets,0bytesIM:0packets,0bytesNewsgroups:0packets,0bytesP2P:0packets,0bytesStreaming:0packets,0bytesTFTP:0packets,0bytesVoIP:0packets,0bytesGenericTCP:1098554packets,817573554bytesGenericUDP:1490packets,210976bytesGenericICMP:0packets,0bytesGenericIP:6packets,192bytes#diagnosehardwaredeviceinfonicport1Name:port1Driver:e1000Version:5.1.13k2NAPIFWversion:N/ABus:00:11.0Memory:0xfeb80000-0xfeba0000Baseaddress:0x1400Interrupt:18Hwaddr:00:0c:29:95:8c:faPermanentHwaddr:00:0c:29:95:8c:faState:upLink:upMtu:1500Supported:auto10half10full100half100full1000fullAdvertised:auto10half10full100half100full1000fullSpeed:1000fullAuto:enabledRxpackets:136154Rxbytes:10901815Rxcompressed:0Rxdropped:0Rxerrors:0RxLengtherr:0RxBufoverflow:0RxCrcerr:0RxFrameerr:0RxFifooverrun:0RxMissedpackets:0Txpackets:1611Txbytes:257565...Multicasts:0Collisions:0带宽利用率，系统崩溃和错误#diagnosefirewallstatistic其他工具CLIgetsystemstatusgetsystemperformancestatusdiagnosesystopdiagnosesystop-summarydiagnosehardwaresysinfomemorydiagnosehardwaresysinfoshmdiagnosenetlinkdevicelistdiagnosehardwaredeviceinfonicport1diagnosefirewallstatisticsshow...DashboardSNMPtrapsAlertemailLogs其他工具CLI#diagnosehardwaredeviceinfonicport1Description:FortiASICNP6AdapterDriverName:FortiASICUnifiedNPUDriverName:np6_2PCISlot:8d:00.0irq:58Board:FGT3700DSN:NP6KR44613000276MajorID:2MinorID:0lifid:0lifoid:156netdevoid:156netdevflags:1203Current_HWaddr:08:5b:0e:4a:2e:e4Permanent_HWaddr:08:5b:0e:4a:2e:e4phyname:np6_2_0bank_id:255phy_addr:0x20lane:0sw_port:51sw_np_port(cat)vid_phy[6]:[0x00][0x00][0x0b][0x00][0x00][0x00]vid_fwd[6]:[0x00][0x00][0x00][0x00][0x00][0x00]oid_fwd[6]:[0x00][0x00][0x00][0xcc][0x00][0x00]==========LinkStatus==========Admin:upnetdevstatus:downautonego_setting:1link_setting:1link_speed:40000link_duplex:1Speed:0Duplex:Fulllink_status:Downrx_link_status:0int_phy_link:0local_fault:0local_warning:0remote_fault:0============Counters===========RxPkts:0RxBytes:0TxPkts:0TxBytes:0HostRxPkts:0HostRxBytes:0HostRxdropped:0HostTxPkts:4HostTxBytes:198HostTxdropped:0sw_rx_pkts:0sw_rx_bytes:0sw_tx_pkts:0sw_tx_bytes:0sw_np_rx_pkts:4sw_np_rx_bytes:272sw_np_tx_pkts:0sw_np_tx_bytes:0物理层/数据链路层的Troubleshooting#diagnosehardwaredeviceinfo网络层的Troubleshooting：路由#executeping-options?data-size

定义数据包的大小,以bytes为单位df-bit

在IP头里设置DF位<yes|no>interval

两个ping直接的间隔时间,以秒为单位pattern

十六进制格式,e.g.00ffaabbrepeat-count重复ping多少次sourceauto|<源接口IP>timeout

定义多少秒后timeouttosIP的服务类型ttl存活时间time-to-live.validate-reply有效的reply数据<yes|no>.view-settings查看ping的当前设置#executeping<ipv4_address>#executetraceroute{<ipv4_address>|<host_fqdn>}网络层的Troubleshooting：路由#execut网络层的Troubleshooting：会话清空之前的过滤条件#diagnosesyssessionfilterclear设置过滤条件#diagnosesyssessionfilter?dport destinationportdst destinationIPaddresspolicy policyidsport sourceportsrc sourceipaddress列出所有匹配过滤条件的会话#diagnosesyssessionlist清空所有匹配过滤条件的会话#diagnosesyssessionclear网络层的Troubleshooting：会话清空之前的过滤条会话表：TCPsessioninfo:proto=6proto_state=65duration=3expire=9timeout=3600flags=00000000sockflag=00000000sockport=443av_idx=9use=5origin-shaper=guarantee-100kbpsprio=2guarantee12800Bpsmax134217728Bpstraffic13895Bpsreply-shaper=guarantee-100kbpsprio=2guarantee12800Bpsmax134217728Bpstraffic13895Bpsper_ip_shaper=ha_id=0policy_dir=0tunnel=/state=redirlocalmay_dirtyndrnpunlbosrsstatistic(bytes/packets/allow_err):org=864/8/1reply=2384/7/1tuples=3orgin->sink:orgpre->post,replypre->postdev=7->6/6->7gwy=/hook=postdir=orgact=snat10:57999->9:443(6:57999)hook=predir=replyact=dnat9:443->6:57999(10:57999)hook=postdir=replyact=noop9:443->10:57999(:0)pos/(before,after)0/(0,0),0/(0,0)misc=0policy_id=1id_policy_id=0auth_info=0chk_client_info=0vd=0serial=0008b037tos=ff/ffips_view=1app_list=2000app=24534dd_type=0dd_mode=0per_ip_bandwidthmeter:addr=10,bps=4872npu_state=00000000npuinfo:flag=0x00/0x00,offload=0/0,ips_offload=0/0,epid=0/0,ipid=0/0,vlan=0/0连接状态流量整形数据包统计剩余TTLNAT硬件加速协议目的端口会话处理会话表：TCPsessioninfo:proto=6p会话表：协议proto=6服务代码在IP头中常见的代码1=ICMP6=TCP17=UDP132=SCTPIPv4Header会话表：协议proto=6IPv4Header传输层的Troubleshooting:TCP状态proto_state=05总是两位数字第一位数字=客户端的会话状态（没有代理则是0）第二位数字=服务器端的会话状态StateValueExpiryTimer(default)NONE010sESTABLISHED13600sSYN_SENT2120sSYN&SYN/ACK360sFIN_WAIT4120sTIME_WAIT5120sCLOSE610sCLOSE_WAIT7120sLAST_ACK830sLISTEN9120s传输层的Troubleshooting:TCP状态prot传输层的Troubleshooting:UDP&ICMP状态虽然UDP是一个无状态协议,FortiGate仍会有两个不同的“proto_state”值:State Value未看到UDP回应 00看到UDP回应 01ICMP无状态proto_state一直标记为00传输层的Troubleshooting:UDP&ICM传输层的Troubleshooting:SCTP状态StateValueExpiryTimer(default)SCTP_S_NONE060sSCTP_S_ESTABLISHED13600sSCTP_S_CLOSED210sSCTP_S_COOKIE_WAIT35sSCTP_S_COOKIE_ECHOED410sSCTP_S_SHUTDOWN_SENT530sSCTP_S_SHUTDOWN_RECD630sSCTP_S_SHUTDOWN_ACK_SENT73sSCTP_S_MAX8n/a传输层的Troubleshooting:SCTP状态Sta会话表：会话处理标识state=logshapemay_dirty并不通用如果会话被卸载到ASIC芯片上，则不一定代表是现在的状态

hardwareaccelerationFlagMeaninglogSessionisbeingloggedlocalSessionisto/fromlocalstackextSessioniscreatedbyafirewallsessionhelpermay_dirtySessioniscreatedbytraffichittingapolicy.ndrSessionwillbecheckedbyIPSsignaturendsSessionwillbecheckedbyIPSanomalybrSessionisbeingbridged(TPmode)npuSessionispossibletobeoffloadedtoNPUwccpSessionishandledbyWCCPnpdSessioncannotbeoffloadedtoNPUdirtyNextpacketinoriginaldirectionwillberevalidatedagainstpolicyredirSessionisbeingprocessedbyanapplicationlayerproxyauthedSessionwassuccessfullyauthenticatedauthSessionisrequires(orrequired)authenticationsrc-visSessionisbeingscannedfordevicedetectionpurposes会话表：会话处理标识state=logshapemay_会话表:连接自动删除会话超时expire=89timeout=3600不活跃的会话当两个值都为0时TCP连接被拆除FIN,FIN/ACK,ACKTCP连接超时tcp-halfclose-timer:FINWAITandCLOSEWAITtcp-half-open-timer:SYNSENTandSYN&SYN/ACKtcp-timewait-timer:TIMEWAITudp-idle-timer会话表:连接自动删除会话超时高级抓包选项#diagsnifferpacket<interface>'<filter>'<level><count><tsformat><count>当抓取了这个数目的报文时自动停止抓包<tsformat>

修改时间戳的格式a

–绝对UTC时间l–当地时间高级抓包选项#diagsnifferpacket<in高级抓包选项：输出#diagsniffpacketany'icmp'4interfaces=[any]filters=[icmp]2.101199wan2in10->:icmp:echorequest2.101400wan1out6->:icmp:echorequest2.123325wan1in->6:icmp:echoreply2.123500wan2out->10:icmp:echoreply

4packetsreceivedbyfilter0packetsdroppedbykernel#diagsniffpacketany'icmp'43linterfaces=[any]filters=[icmp]2019-11-1410:28:19.769989wan2in10->:icmp:echorequest2019-11-1410:28:19.770143wan1out6->:icmp:echorequest2019-11-1410:28:19.792325wan1in->6:icmp:echoreply3packetsreceivedbyfilter0packetsdroppedbykernel报文数量时间戳高级抓包选项：输出#diagsniffpacketa诊断系统反应过慢高CPU使用率高内存使用率上一个开启的功能是什么？每次开启一个功能快速诊断CPU使用率有多高，为什么？#getsystemperformancestatus#diagnosesystop1诊断系统反应过慢高CPU使用率高CPU占用率的Troubleshooting:getsysperfstat#getsystemperformancestatusCPUstates:4%user13%system0%nice83%idleCPU0states:3%user13%system0%nice84%idleCPU1states:5%user13%system0%nice82%idleCPU2states:2%user13%system0%nice85%idleCPU3states:6%user13%system0%nice81%idleMemorystates:19%usedAveragenetworkusage:12740kbpsin1minute,3573kbpsin10minutes,1077kbpsin30minutesAveragesessions:118sessionsin1minute,11sessionsin10minutes,40sessionsin30minutesAveragesessionsetuprate:11sessionspersecondinlast1minute,0sessionspersecondinlast10minutes,1sessionspersecondinlast30minutesViruscaught:3totalin1minuteIPSattacksblocked:64totalin1minuteUptime:60days,9hours,58minutesCPU使用率网络使用率内存使用率高CPU占用率的Troubleshooting:gets高CPU使用率：临时bypass一些检查进程可以暂时bypass一些检查进程#diagnosetestapplicationipsmonitor5全局bypass更容易,之后可以调整策略来确认问题无需检查的任务继续运行CPU使用率在bypass这些进程后是否降低了？恢复某个检查进程：#diagnosetestapplicationipsmonitor5高CPU使用率：临时bypass一些检查进程可以暂时bypa内存诊断进程的内存使用率#getsystemperformancestatus#diagsystop-summaryFortiOS的内存使用率，不是某一个进程的#diagnosehardwaresysinfomem#diagnosehardwaresysinfoslab内存诊断进程的内存使用率#diagnosesystop-summaryCPU[|||||||||||||||]38.4%Mem[|||||||||||||||||||||]54.0%1009M/1841MProcesses:20(running=1sleeping=86)PIDRSS^CPU%MEM%FDSTIME+NAME*7232M34.21.71100:03.39httpclid[x5]9511M1.90.62053:07.83cw_wtpd4023M1.21.32403:02.60httpsd[x5]117327M0.01.51000:02.82pyfcgid[x4]3610M0.00.58800:47.75zebos_launcher[x12]379M0.00.5900:00.23uploadd3815M0.00.84101:52.19miglogd399M0.00.5500:01.41kmiglogd4625M0.01.482101:47.98proxyd[x6]4710M0.00.5700:00.12wad_diskd5112M0.00.71600:02.72scanunitd[x3]5361M0.03.31600:15.14ipsmonitor[x2]579M0.00.5700:00.13merged_daemons6913M0.00.71800:34.20urlfilter在‘diagsystop’中RAM复杂交叉的进程会使条目众多交叉的进程中共享数据#diagnosesystopRunTime:11days,3hoursand29minutes0U,0S,10I;500T,345F,78KFthttp48S0.04.4httpsd74S0.03.4httpsd54S0.03.4cmdbsvr23S0.03.4httpsd18618S0.02.9httpsd18645S0.02.9httpsd18643S0.02.9httpsd18646S0.02.8httpsd39S0.02.8ipsengine251S<0.02.7fgfmd69S0.02.4newcli103S0.02.4newcli18655R0.02.4newcli18651S0.02.4miglogd38S0.02.2