小型机故障(2007-9-3).doc

小型机故障记录故障机器： 主机名 H852 Boot IP 0 Service IP 0 Standby IP 0故障时间： 2007年9月3日下午2点左右故障现象： 同一网段的小型机与该主机通信正常，但跨网段的机器无法与该主机通信初步判断： 该主机的网络设置被改动，或者是网络设备物理或逻辑上的变动所致处理过程： 1 询问网络管理员，他们最近没有对网络设备作物理或逻辑上的调整 2 在本机查网络设置和日志通过对本机的配置进行查询，发现存在如下现象，现详细记录如下：1 # hostname H8522 # ifconfig -aen0: flags=4e080863,80 inet 0 netmask 0xffffff00 broadcast 55en1: flags=4e080863,80 inet netmask 0xff000000 broadcast 55lo0: flags=e08084b inet netmask 0xff000000 broadcast 55 inet6 :1/0 tcp_sendspace 131072 tcp_recvspace 131072 rfc1323 13 # netstat r Routing tablesDestination Gateway Flags Refs Use If PMTU Exp GroupsRoute Tree for Protocol Family 2 (Internet):default UG 0 1312547 en0 - - - UHSb 0 0 en1 - - - =10/8 U 3 11028 en1 - - - loopback UGHS 0 2 lo0 - - - loopback UHSb 0 0 lo0 - - - = h852_service UHSb 0 0 en0 - - - =10.10.20/24 h852_service U 5 104741 en0 - - - h852_service loopback UGHS 0 16668 lo0 - - - 55 loopback UHSb 0 269 lo0 - - - =55 h852_service UHSb 0 2 en0 - - - 55 UHSb 0 0 en1 - - - 127/8 loopback U 21 14059043 lo0 - - - Route Tree for Protocol Family 24 (Internet v6):1 :1 UH 0 0 lo0 - - - #4 # errpt # errpt IDENTIFIER TIMESTAMP T C RESOURCE_NAME DESCRIPTIONAFA89905 0903163807 I O grpsvcs Group Services daemon started92E96A0D 0903163807 P U topsvcs Local adapter misconfiguration detected97419D60 0903163807 I O topsvcs Topology Services daemon started5294E5FF 0903163807 I O topsvcs Missing or incorrect adapter address6D19271E 0903143807 I O topsvcs Topology Services daemon stopped28854E81 0903143807 I O grpsvcs Group Services daemon stopped99FA80C7 0903143807 U S haemd SOFTWARE9D30B78E 0826234007 T S tty1 RECEIVER OVER-RUN ON INPUT9D30B78E 0825124307 T S tty1 RECEIVER OVER-RUN ON INPUT9D30B78E 0825050007 T S tty1 RECEIVER OVER-RUN ON INPUT9D30B78E 0822205307 T S tty1 RECEIVER OVER-RUN ON INPUT9D30B78E 0821200307 T S tty1 RECEIVER OVER-RUN ON INPUT9D30B78E 0818012907 T S tty1 RECEIVER OVER-RUN ON INPUT9D30B78E 0812174207 T S tty1 RECEIVER OVER-RUN ON INPUT3C81E43F 0720000007 P U topsvcs Late in sending heartbeat92E96A0D 0716171507 P U topsvcs Local adapter misconfiguration detected5 # errpt aj 5294E5FF-LABEL: TS_FD_INVAL_ADDR_STIDENTIFIER: 5294E5FFDate/Time: Mon Sep 3 16:38:36 BEIST 2007Sequence Number: 95368Machine Id: 000C4ADF4C00Node Id: h852Class: OType: INFOResource Name: topsvcs DescriptionMissing or incorrect adapter addressProbable CausesAdapter is not configuredAdapter has no addressAdapter has an address that is outside the configurationUser CausesAdapter was incorrectly configured Recommended Actions Correct adapter configuration: its address must match the boot-time address expected by Topology ServicesFailure CausesA problem occurred when configuring the adapter Recommended Actions Correct adapter configuration: its address must match the boot-time address expected by Topology ServicesDetail DataDETECTING MODULErsct,force_down.C,1.11,501 ERROR ID 6zLCZG/AUwq4/w8P.SEV08.REFERENCE CODE Interface nameen1Actual addressExpected boot-time address06 # errpt aj 92E96A0D-LABEL: TS_MISCFG_EMIDENTIFIER: 92E96A0DDate/Time: Mon Sep 3 16:38:37 BEIST 2007Sequence Number: 95370Machine Id: 000C4ADF4C00Node Id: h852Class: UType: PENDResource Name: topsvcs Resource Class: NONEResource Type: NONELocation: VPD: DescriptionLocal adapter misconfiguration detectedProbable CausesLocal adapter assigned incorrect IP addressInstall CausesLocal adapter assigned incorrect IP address Recommended Actions Correct adapter IP address If the problem persists, restart Topology Services daemon.Detail DataDETECTING MODULErsct,nim_control.C,1.38,4338 ERROR ID 6BcKuG0BUwq4/2DC/SEV08.REFERENCE CODE Adapter interface nameen1Adapter offset 1Adapter expected IP address0-LABEL: TS_MISCFG_EMIDENTIFIER: 92E96A0DDate/Time: Mon Jul 16 17:15:35 BEIST 2007Sequence Number: 95344Machine Id: 000C4ADF4C00Node Id: h852Class: UType: PENDResource Name: topsvcs Resource Class: NONEResource Type: NONELocation: VPD: DescriptionLocal adapter misconfiguration detectedProbable CausesLocal adapter assigned incorrect IP addressInstall CausesLocal adapter assigned incorrect IP address Recommended Actions Correct adapter IP address If the problem persists, restart Topology Services daemon.Detail DataDETECTING MODULErsct,nim_control.C,1.38,4338 ERROR ID 6BcKuG0rQna4/yPD1SEV08.REFERENCE CODE Adapter interface nameen1Adapter offset 1Adapter expected IP address07 部分登录信息# lastroot pts/2 RAC_P5711 Sep 04 09:31 - 09:44 (00:13)root pts/1 RAC_P5711 Sep 04 09:17 still logged in.root pts/3 h851_service Sep 03 16:51 - 17:04 (00:13)root pts/2 RAC_P5711 Sep 03 16:47 - 16:58 (00:11)root pts/1 h851_service Sep 03 16:42 - 18:57 (02:15)root lft0 Sep 03 13:55 - 16:40 (02:44)root pts/1 8 Aug 24 15:24 - 16:53 (01:29)root pts/1 01 Aug 24 10:14 - 12:24 (02:10)root pts/1 9 Aug 21 20:05 - 20:11 (00:05)root pts/1 9 Aug 21 19:30 - 19:31 (00:01)root pts/1 h851_service Aug 21 15:20 - 16:57 (01:37)root lft0 Aug 14 16:27 - 16:55 (00:27)root dtremote 58:0 Aug 14 09:03 - 13:38 (04:35)root pts/1 00 Aug 09 11:08 - 15:41 (04:33)root dtremote 58:0 Aug 09 10:34 - 10:43 (00:09)root pts/0 74 Aug 03 08:59 - 13:16 (04:17)root pts/1 h851_service Jul 18 16:34 - 16:36 (00:01)root pts/1 74 Jul 17 16:47 - 23:18 (06:31)root pts/1 74 Jul 17 08:41 - 14:18 (05:37)root pts/1 74 Jul 16 18:00 - 18:02 (00:02)root dtremote 1:1 Jul 16 16:58 - 17:19 (00:20)root pts/1 1 Jul 16 16:48 - 16:55 (00:07)root lft0 Jul 16 16:28 - 17:03 (11+00:34)root pts/0 5 Jul 16 16:25 - 16:26 (00:00)root pts/0 4 May 22 09:28 - 09:29 (00:00)root pts/0 7 May 16 15:43 - 15:43 (00:00)root pts/0 3 Apr 26 23:26 - 23:28 (00:02)root pts/0 3 Apr 26 10:04 - 10:08 (00:04)root pts/0 3 Apr 25 09:51 - 11:31 (01:39)root lft0 Apr 06 10:14 - 16:19 (101+06:05)reboot Apr 06 10:09shutdown pts/0 Apr 06 10:00root lft0 Apr 06 09:45 - 10:00 (00:14)root pts/0 h851_stby Aug 25 11:05 - 11:05 (00:00)root pts/0 h851_stby Aug 21 15:07 - 15:24 (00:17)root pts/0 h851_stby Aug 21 10:53 - 11:00 (00:06)root lft0 Jul 25 19:41 - 10:15 (203+14:33)reboot Jul 25 19:37shutdown lft0 Jul 25 14:56root lft0 Jun 22 11:42 - 14:55 (33+03:13)reboot Jun 21 12:29reboot Apr 28 10:54shutdown lft0 Apr 28 10:28root lft0 Apr 28 10:27 - 10:28 (00:01)reboot Apr 28 10:23shutdown lft0 Apr 28 09:56oracle ftp 8 Apr 27 08:58 - 08:58 (00:00)last 1432 lines, 108241 charactersshutdown lft0 Apr 28 09:56oracle ftp 8 Apr 27 08:58 - 08:58 (00:00)root pts/2 5 Apr 26 10:48 - 10:53 (00:05)oracle ftp 8 Apr 26 09:45 - 09:45 (00:00)oracle ftp 8 Apr 26 09:32 - 09:32 (00:00)oracle ftp 8 Apr 26 08:58 - 08:58 (00:00)oracle ftp 8 Apr 25 09:01 - 09:02 (00:01)oracle ftp 8 Apr 24 08:59 - 09:00 (00:01)oracle pts/1 0 Apr 21 15:36 - 15:48 (00:11)root pts/1 99 Apr 21 10:01 - 10:01 (00:00)root pts/1 99 Apr 21 09:51 - 09:51 (00:00)root pts/1 99 Apr 21 09:40 - 09:41 (00:00)root pts/1 99 Apr 21 09:30 - 09:30 (00:00)root pts/1 99 Apr 21 09:20 - 09:20 (00:00)root pts/1 99 Apr 21 09:10 - 09:10 (00:00)root pts/1 99 Apr 21 09:00 - 09:00 (00:00)root pts/1 99 Apr 21 08:50 - 08:50 (00:00)root pts/1 99 Apr 21 08:39 - 08:40 (00:00)root pts/1 99 Apr 21 08:29 - 08:29 (00:00)root pts/1 99 Apr 21 08:19 - 08:19 (00:00)root pts/1 99 Apr 21 08:09 - 08:09 (00:00)root pts/1 99 Apr 21 07:59 - 07:59 (00:00)root pts/1 99 Apr 21 07:49 - 07:49 (00:00)root pts/1 99 Apr 21 07:38 - 07:39 (00:00)root pts/1 99 Apr 21 07:28 - 07:28 (00:00)root pts/1 99 Apr 21 07:18 - 07:18 (00:00)root pts/1 99 Apr 21 07:08 - 07:08 (00:00)root pts/1 99 Apr 21 06:58 - 06:58 (00:00)root pts/1 99 Apr 21 06:48 - 06:48 (00:00)root pts/1 99 Apr 21 06:37 - 06:38 (00:00)root pts/1 99 Apr 21 06:27 - 06:27 (00:00)root pts/1 99 Apr 21 06:17 - 06:17 (00:00)root pts/1 99 Apr 21 06:07 - 06:07 (00:00)root pts/1 99 Apr 21 05:57 - 05:57 (00:00)root pts/1 99 Apr 21 05:47 - 05:47 (00:00)root pts/1 99 Apr 21 05:37 - 05:37 (00:00)root pts/1 99 Apr 21 05:26 - 05:26 (00:00)root pts/1 99 Apr 21 05:16 - 05:16 (00:00)root pts/1 99 Apr 21 05:06 - 05:06 (00:00)root pts/1 99 Apr 21 04:56 - 04:56 (00:00)root pts/1 99 Apr 21 04:46 - 04:46 (00:00)root pts/1 99 Apr 21 04:36 - 04:36 (00:00)root pts/1 99 Apr 21 04:25 - 04:25 (00:00)root pts/1 99 Apr 21 04:15 - 04:15 (00:00)root pts/1 99 Apr 21 04:05 - 04:05 (00:00)root pts/1 99 Apr 21 03:55 - 03:55 (00:00)root pts/1 99 Apr 21 03:45 - 03:45 (00:00)root pts/1 99 Apr 21 03:55 - 03:55 (00:00)root pts/1 99 Apr 21 03:45 - 03:45 (00:00)root pts/1 99 Apr 21 03:35 - 03:35 (00:00)root pts/1 99 Apr 21 03:24 - 03:24 (00:00)root pts/1 99 Apr 21 03:14 - 03:14 (00:00)root pts/1 99 Apr 21 03:04 - 03:04 (00:00)root pts/1 99 Apr 21 02:54 - 02:54 (00:00)root pts/1 99 Apr 21 02:44 - 02:44 (00:00)root pts/1 99 Apr 21 02:34 - 02:34 (00:00)root pts/1 99 Apr 21 02:23 - 02:23 (00:00)root pts/1 99 Apr 21 02:13 - 02:13 (00:00)root pts/1 99 Apr 21 02:03 - 02:03 (00:00)root pts/1 99 Apr 21 01:53 - 01:53 (00:00)root pts/1 99 Apr 21 01:43 - 01:43 (00:00)root pts/1 99 Apr 21 01:33 - 01:33 (00:00)root pts/1 99 Apr 21 01:22 - 01:22 (00:00)root pts/1 99 Apr 21 01:12 - 01:12 (00:00)root pts/1 99 Apr 21 01:02 - 01:02 (00:00)root pts/1 99 Apr 21 00:52 - 00:52 (00:00)root pts/1 99 Apr 21 00:42 - 00:42 (00:00)root pts/1 99 Apr 21 00:32 - 00:32 (00:00)root pts/1 99 Apr 21 00:21 - 00:22 (00:00)root pts/1 99 Apr 21 00:11 - 00:11 (00:00)root pts/1 99 Apr 21 00:01 - 00:01 (00:00)root pts/1 99 Apr 20 23:51 - 23:51 (00:00)root pts/1 99 Apr 20 23:41 - 23:41 (00:00)root pts/1 99 Apr 20 23:31 - 23:31 (00:00)root pts/1 99 Apr 20 23:20 - 23:20 (00:00)root pts/1 99 Apr 20 23:10 - 23:10 (00:00)root pts/1 99 Apr 20 23:00 - 23:00 (00:00)root pts/1 99 Apr 20 22:50 - 22:50 (00:00)root pts/1 99 Apr 20 22:40 - 22:40 (00:00)root pts/1 99 Apr 20 22:30 - 22:30 (00:00)root pts/1 99 Apr 20 22:19 - 22:20 (00:00)root pts/1 99 Apr 20 22:09 - 22:09 (00:00)root pts/1 99 Apr 20 21:59 - 21:59 (00:00)root pts/1 99 Apr 20 21:49 - 21:49 (00:00)root pts/1 99 Apr 20 21:39 - 21:39 (00:00)root pts/1 99 Apr 20 21:29 - 21:29 (00:00)root pts/1 99 Apr 20 21:18 - 21:19 (00:00)root pts/1 99 Apr 20 21:08 - 21:08 (00:00)root pts/1 99 Apr 20 20:58 - 20:58 (00:00)root pts/1 99 Apr 20 20:48 - 20:48 (00:00)root pts/1 99 Apr 20 20:38 - 20:38 (00:00)root pts/1 99 Apr 20 20:28 - 20:28 (00:00)root pts/1 99 Apr 20 20:17 - 20:18 (00:00)root pts/1 99 Apr 20 20:07 - 20:07 (00:00)root pts/1 99 Apr 20 20:17 - 20:18 (00:00)root pts/1 99 Apr 20 20:07 - 20:07 (00:00)root pts/1 99 Apr 20 19:57 - 19:57 (00:00)root pts/1 99 Apr 20 19:47 - 19:47 (00:00)root pts/1 99 Apr 20 19:37 - 19:37 (00:00)root pts