CLKX故障导致BSC频繁重启.doc

SR 2125389的故障处理报告尊敬的湖北移动客户， 非常遗憾这次故障给您的网络造成影响。接到您的故障报告后，我们立刻全力投入处理，力求在最快时间内解决贵公司的网络问题。以下是针对这次故障处理的总结报告，供您参考：SR 号:2125389客户:China Mobile; China; Hubei; Yichang客户联系人:Liu MinCNRC工程师:Li Hongmin网元: BSC11 （BSC）版本号:1760.21-t3影响程度:6.1 故障现象描述BSC11在9月14日13:38开始循环重启。Motorola现场工程师到达现场，未发现板件有红灯报警。14:30，在拔出GCLK 0后，BSC工作正常。6.2故障现象12:00 主备GCLK出现时钟报警，工作不稳定，影响cage 0中MCAP总线，导致Cage 0中部分MMS出现故障退服，从而使相关信令链路闪断。#0 - NOT APPL - *NONE*. equipmentFailureEvent - GCLK-GCLK - YC-BSS11(YC-BSS11:SITE-0:): 0 GCLK-GCLK 0 - 14/09/2006 11:59:50. 4 Phase Lock Lost - FMIC - Major 0/5. HW Version 8h Config Tag 800118 10 9C 00 01 .#0 - NOT APPL - *NONE*. equipmentFailureEvent - GCLK-GCLK - YC-BSS11(YC-BSS11:SITE-0:): 0 GCLK-GCLK 1 - 14/09/2006 11:59:50. 4 Phase Lock Lost - FMIC - Major 0/3. HW Version 9h Config Tag 800118 10 9C 00 01 .#0 - NOT APPL - *NONE*. stateChangeEvent - MMS-MSI - YC-BSS11(YC-BSS11:SITE-0:): 0 MMS-MSI 10 0 - 14/09/2006 12:00:42. From : (Busy, Unlocked) - MMS Providing Clock. To : (Disabled, Unlocked) - Remote Alm OOS Threshold. Config Tag 800119 1#0 - NOT APPL - *NONE*. stateChangeEvent - MMS-MSI - YC-BSS11(YC-BSS11:SITE-0:): 0 MMS-MSI 7 0 - 14/09/2006 12:04:13. From : (Busy, Unlocked) - NO REASON. To : (Disabled, Unlocked) - Remote Alm OOS Threshold. Config Tag 80011a 1#0 - NOT APPL - *NONE*. stateChangeEvent - MMS-MSI - YC-BSS11(YC-BSS11:SITE-0:): 0 MMS-MSI 16 1 - 14/09/2006 12:30:44. From : (Busy, Unlocked) - NO REASON. To : (Disabled, Unlocked) - Remote Alarm OOS Timer. Config Tag 800133 10#0 - NOT APPL - *NONE*. stateChangeEvent - MMS-MSI - YC-BSS11(YC-BSS11:SITE-0:): 0 MMS-MSI 16 0 - 14/09/2006 13:04:47. From : (Busy, Unlocked) - MMS Providing Clock. To : (Disabled, Unlocked) - Remote Alm OOS Threshold. Config Tag 8001be - 113:38 BSC11 重启 7d FATAL SWFM ERROR: HARD RESET Routine: global_kill 7d Area: 0x00000042 Error: 0x0000001d PC: 0xc0001064 PID: 0x50 (Init) 7d BSS Release: 1.7.6.f0.71 Obj Version: 1.7.6.0.20 Exec Version: 1.7.6.21.3 7d 14-Sep-2006 13:38:47.290 Subsystem: 0x01 CPU: 0x0115 Board: GPROC2 RAM 7d Global Kill message received by IP from process 4b 7d The reset type is RESET_HARD_GENERIC13:53 BSC11开始循环重启。当时CBUS0上的CLKX端口光强很弱,怀疑存在硬件故障影响CBUS 0,进而影响cage 0的MCAP总线通信,导致cage 0 OOS，BSC11循环重启。 6 FATAL SWFM ERROR: SOFT RESET Routine: build_config 6 Area: 0x00000306 Error: 0x00000000 PC: 0xc000189c PID: 0x41 (MCAP DLSP) 6 BSS Release: 1.7.6.f0.71 Obj Version: 1.7.6.0.20 Exec Version: 1.7.6.21.3 6 14-Sep-2006 13:53:13.680 Subsystem: 0x01 CPU: 0x0115 Board: GPROC2 RAM 6 MCAP DLSP database bad (loaded_flag)3 -MCAP DLSP存在问题 8 FATAL SWFM ERROR: SOFT RESET Routine: cage_ksw_msi_transitions.c 8 Area: 0x00000300 Error: 0x00000011 PC: 0xc00458fc PID: 0x42 (CA) 8 BSS Release: 1.7.6.f0.71 Obj Version: 1.7.6.0.21 Exec Version: 1.7.6.21.3 8 14-Sep-2006 13:53:20.815 Subsystem: 0x01 CPU: 0x0115 Board: GPROC2 RAM 8 CA error, Line = 96, thread = 3, Config Tag = 0x00000001 8 ENABLE of Entity = CAGE0,0,0, State = DISABLED UNLOCKED NO REASON 8 Cage INS failure: Problems with KSW INS; 8 Check for correct KSW physical slot installation or bad KSW. -Cage0中的主备TDM存在问题而导致Cage 0 OOS a Nonfatal SWFM Error Routine: fw_audit_code.c a Area: 0x00000300 Error: 0x00000040 PC: 0xc00458fc PID: 0x42 (CA) a BSS Release: 1.7.6.f0.71 Obj Version: 1.7.6.0.21 Exec Version: 1.7.6.21.3 a 14-Sep-2006 14:00:58.480 Subsystem: 0x01 CPU: 0x0115 Board: GPROC2 RAM a CA error, Line = 134, thread = 41, Config Tag = 0x00000001 a ENABLE of Entity = KSW0,1,0, State = DISABLED UNLOCKED NO REASON a Call to function wait_for_msg returned bad value 0x22 c Nonfatal SWFM Error Routine: ksw_add_del_board.c c Area: 0x00000300 Error: 0x00000011 PC: 0xc00458fc PID: 0x42 (CA) c BSS Release: 1.7.6.f0.71 Obj Version: 1.7.6.0.21 Exec Version: 1.7.6.21.3 c 14-Sep-2006 14:01:12.115 Subsystem: 0x01 CPU: 0x0115 Board: GPROC2 RAM c CA error, Line = 231, thread = 41, Config Tag = 0x00000001 c ENABLE of E