虚拟机hang机处理最佳实践

全球支持服务GSSBangalore,Tokyo,

JapanCork,

IrelandBurlington,

CanadaBroomfield,

COSupport

officesLocal

language

supportSpanish,

Portuguese,

French,

German,

Japanese,

ChineseBeijing,

China全日制支持严重性为1的服务请求全球覆盖24x7,

365

天/年To

pro

utstanding

levels

of

technical

support

using

our

in-depthVirtualization

and

Cloud

expertise,

we

work

as

a

Global

team

to

makea

difference

for

VMware

and

our

Customers.关键任务支持Assigned

AccountManagementSenior

TSETeamEnhancedsResponseOnsite

SupportPriority

Queuing关键业务支持Shared

AccountManagementDesignated

TSETeam生产支持FasterResponse

TimesAfter

HoursSupport

(Sev

1)基本支持SubscriptionWeb

SupportUnlimited

SRsBasicResponse

TimesPhone

SupportVMware

SupportOfferings自助支持KnowledgebaseCommunitiesationSocial

SupportVMware

Support

Offering

MatrixKnowledgebase,ation,

CommunitiesYESYESYESYESUpgrades

&

UpdatesYESYESYESYESSupport

RequestsUnlimitedUnlimitedUnlimitedUnlimitedSupport

ChannelsLive

&

WebLive

&

WebLive

&

WebCallback

&

WebResponse

sMCS

(24x7)Production

(24x7)Production

(24x7)Basic

(12x5)24

x

7

Response

–

Sev1YES

(30

min)YES

(30

min)YES

(30

min)-Weekend

Response

–

Sev

2YES

(2

hr--12x7)---Priority

QueueYES---Senior

Support

TeamYESYES--#

of

Support

AdministratorsUnlimited*664CoverageGeography

lobalD enter

Team--Root

Cause

ysisYESYES--Onsite

EscalationSupportYES---Support

Account

ManagerYESProvided

by

TSE--Customized

Support

PlanYESYES--Regular

Support

MeetingsYESYES--Support

Request

ReportingYES---YES---Support

TrendysisQuarterly

Business

ReviewsYES---Environment

Best

PracticesDiscussionsDiscussions--Proactive

OnsiteYES---Environment

InventoryYES---Product

Updates,

FeaturesDiscussions---BasicProductionFeatureMission

CriticalBusiness

CriticalProactiveSupportReactiveSupportHelpSubscriptionVMware

SupportSeverityDefinitions紧急–服务器宕机服务器无法 .

可能的原因包括但不限于不能启动服务器，服务器频繁死机，和服务器失去连接等。关于文档的问题，如何操作等需要一些资料的支持要求htt

/support/policies/severity.htmlSeverity1Severity2Severity3Severity4严重–影响生产服务器正常但是故障影响到生产环境，有关键应用无法正常使用一般–普通问题有问题发生需要解决但是并不影响生产环境低–信息索取Important

Links议程虚拟机Hang机如何收集Hang机数据如何分析Hang机数据虚拟机Hang机快速判断虚拟机电源仍然处于开启状态，但不响应用户的任何请求；而在 其系统处于UNINTERRUPTABLE状态。常见现象通通虚拟机控制台操作无响应，但可以虚拟机控制台操作无响应，不可以虚拟机控制台显示蓝屏如何收集Hang机数据vSphere层面GuestOS层面如何收集虚拟机Hang的状态数据（vSphere层面）图形界面Web界面命令行方式如何收集虚拟机Hang的状态数据(图形方法)鼠标选中目标虚拟机如何收集虚拟机Hang的状态数据(图形方法)在菜单里选择导出系统日志如何收集虚拟机Hang的状态数据(图形方法)选中全部多选框如何收集虚拟机Hang的状态数据(图形方法)选择保存数据的目标如何收集虚拟机Hang的状态数据(图形方法)选择完成按钮如何收集虚拟机Hang的状态数据(图形方法)确认任务状态已经开始技巧：如果使用vSphereClient客户端连接vCenter导出系统日志不能正常工作，可以使用客户端直接连接到ESXi主机上进行操作。如何收集虚拟机Hang的状态数据(图形方法)虚拟机将自动电源挂起并生成二进制文件列表，收集程序在

将这些数据拷贝并打包。收集程序结束后，虚拟机电源状态继续保持挂起，需要手工恢复虚拟机电源；如果故障场景是虚拟机无响应，恢复电源后可再重置虚拟机，重新引导后恢复正常工作。虚拟机电源将被自动挂起如何收集虚拟机Hang的状态数据(WebClient)鼠标选中目标虚拟机如何收集虚拟机Hang的状态数据(WebClient)选中全部多选框1234虚拟机电源将被自动挂起举例：如果开机虚拟机较多可以使用再使用过滤命令得到目标虚拟机信息，比如要查询的虚拟机名称是TestVM#vm-support

-V

|

grep

TestVM/vmfs/volumes/52dcc163-ab24f115-9ede-005056a48eea/TestVM/TestVM.vmx

(Running)如何收集虚拟机Hang的状态数据(命令#1)1.

（ESXi5.x,ESXi6.x）首先运行如下命令列举开机的虚拟机信息#vm-support

-V如何收集虚拟机Hang的状态数据(命令#1)2a.(ESXi

5.0)将找到的目标虚拟机信息填入下面命令即可收集虚拟机dump数据。注意：如下是一条命令，在一条命令行里输入即可,多行输入需要在单行结尾以反斜杠结束。#

vm-support

--groups="Virtual"\--manifests="HungVM:Coredump_VM

HungVM:Suspend_VM

HungVM:Send_NMI_To_Guest

Virtual:Snapshots"

\--vm=/vmfs/volumes/52dcc163-ab24f115-9ede-005056a48eea/TestVM/TestVM.vmx2b.(ESXi

5.1及以后）将找到的目标虚拟机信息填入下面命令即可收集虚拟机dump数据。注意：如下是一条命令，在一条命令行里输入即可,多行输入需要在单行结尾以反斜杠结束。#

vm-support

--groups="VirtualMachines"

\--manifests="HungVM:Coredump_VM

HungVM:Suspend_VM

HungVM:Send_NMI_To_Guest“

\--vm=/vmfs/volumes/52dcc163-ab24f115-9ede-005056a48eea/TestVM/TestVM.vmx注：在收集过程中，目标虚拟机电源将被自动挂起；需要手工恢复电源状态。Crashing

a

virtual

machine

on

ESX/ESXi

to

collect

diagnosticinformation.英文版

ht

/kb/2005715中文版

ht

/kb/2097861如何收集虚拟机Hang的状态数据(命令#2)选择1：使用第命令收集并wget

--no-check-

' Fault

HardwareLogs

Network

Storage

System

Userworld

Virtual&vm=</vmfs/volumes/uuid/virtualmachine/virtualmachine.vmx>'选择2：使用vmdump命令分别提取VMX/VMM的DUMP信息/sbin/vmdumper

-l/sbin/vmdumper

WorldID

unsync/sbin/vmdumper

WorldID

sync/sbin/vmdumper

WorldID

vmx/sbin/vmdumper

WorldID

vmx_force参考：

/2014/07/quick-tip-how-to-bsodpanic-a-virtual-machine-in-esxi.htmlWindowsLinux如何收集虚拟机Hang的状态数据（操作系统层面）如何收集虚拟机Hang的状态数据(GuestOS-Win)能，则可以捕捉蓝屏画面，但业务恢复如果关闭windows自动重需要人工干预打开内存转储功能，使用最大内存转储选项如何收集虚拟机Hang的状态数据(GuestOS-Win)如何收集虚拟机Hang的状态数据(GuestOS-Win)如何收集虚拟机Hang的状态数据(GuestOS-Linux)使用KDUMP功能实现GuestOS内存自动转储注：仅Crash状态时激活；非Crash状态使用VM内存转储。如何分析Hang机数据a1)VM

vmware.log

人为操作生成dump的记录2015-04-02T01:20:02.562Z|

vcpu-0|

I120:

VMSAMPLE:

cs=0xea07,

rip=0x2571

halted

progress=2169512015-04-02T01:20:03.578Z|

vcpu-0|

I120:

VMSAMPLE:

cs=0xea07,

rip=0x2571

halted

progress=2005062015-04-02T01:20:03.619Z|

vmx|

I120:

Setting

log_vmsample:

02015-04-02T01:20:03.635Z|

vmx|

I120:

MonitorVMMCoreRequest:

********************************************2015-04-02T01:20:03.635Z|

vmx|

I120:

MonitorVMMCoreRequest:

Unsync

core

dump

requested;

not

a

real

fault2015-04-02T01:20:03.635Z|

vmx|

I120:

MonitorVMMCoreRequest:

********************************************2015-04-02T01:20:08.637Z|

vmx|

W110:

Writing

monitor

corefile

"/vmfs/volumes/53fafab5-de3a7795-41ca-b8ca3a660468/SR-VM-DUMP/vmmcores.gz”如何分析Hang机数据(日志分析)2015-04-02T01:18:58.575Z|

vmx|

W110:

VmkEventVmxTakeScreenShotCb:

saving

to

'vmdumper.png'2015-04-02T01:20:08.637Z|

vmx|

W110:

Writing

monitor

corefile

"/vmfs/volumes/53fafab5-de3a7795-41ca-b8ca3a660468/SR-VM-DUMP/vmmcores.gz"2015-04-02T01:21:03.666Z|

vcpu-0|

W110:

Writing

monitor

corefile

"/vmfs/volumes/53fafab5-de3a7795-41ca-b8ca3a660468/SR-VM-DUMP/vmmcores.gz"2015-04-02T01:22:07.846Z|

vmx|

W110:

A

core

file

is

available

in

"/vmfs/volumes/53fafab5-de3a7795-41ca-b8ca3a660468/SR-VM-DUMP/vmx-zdump.000"a2)VM

vmware.log

生成的文件列表信息如何分析Hang机数据(日志分析)如何分析Hang机数据(虚拟机DUMP文件确认列表)b1)ESXiHostd.log

dump任务记录2015-04-02T01:18:58.214Z

[363C2B70

verbose

'Vimsvc.HaCgiServiceManagerImpl']

Ticket

valid:

***6da24:

(GET/cgi-bin/vm-support.cgi?manifests=HungVM:Coredump_VM

HungVM:Send_NMI_To_Guest

HungVM:Suspend_VM&vm=/vmfs/volumes/53fafab5-de3a7795-41ca-b8ca3a660468/SR-VM-DUMP/SR-VM-DUMP.vmx)

==

((null)

/cgi-bin/vm-support.cgi?manifests=HungVM:Coredump_VMHungVM:Send_NMI_To_Guest

HungVM:Suspend_VM&vm=/vmfs/volumes/53fafab5-de3a7795-41ca-b8ca3a660468/SR-VM-DUMP/SR-VM-DUMP.vmx)mandPosix']

ForkExec(/usr/bin/sh)

17183231mandHandler

(/cgi-bin/vm-support.cgi)']

CGI

command

started.2015-04-02T01:18:58.217Z

[363C2B70

info

'2015-04-02T01:18:58.217Z

[363C2B70

verbose

'……2015-04-02T01:28:41.861Z

[36780B70

info

'vm:Vix:

[34385

foundryVMPowerOps.c:980]:

FoundryVMPowerStateChangeCallback:

/vmfs/volumes/53fafab5-de3a7795-41ca-b8ca3a660468/SR-VM-DUMP/SR-VM-DUMP.vmx']

vmx/execState/val

=

suspended.如何分析Hang机数据(日志分析)b2)ESXiHostd.log

dump事件记录2015-04-02T01:22:07.848Z[363C2B70

info

'Vimsvc.ha-eventmgr']

Event

1014

:

An

application

(/bin/vmx)

running

on

ESXihost

hascrashed

(1

time(s)

so

far).

A

core

file

might

have

been

created

at

/vmfs/volumes/53fafab5-de3a7795-41ca-b8ca3a660468/SR-VM-DUMP/vmx-zdump.000.2015-04-02T01:28:41.884Z

[378C1B70

verbose

'Vmsvc.vm:/vmfs/volumes/53fafab5-de3a7795-41ca-b8ca3a660468/SR-VM-DUMP/SR-VM-DUMP.vmx']

Suspendfile

'SR-VM-DUMP-c497fee6.vmss'

resolved

to'/vmfs/volumes/53fafab5-de3a7795-41ca-b8ca3a660468/SR-VM-DUMP/SR-VM-DUMP-c497fee6.vmss'2015-04-02T01:28:41.927Z

[38281B70

info

'Vimsvc.ha-eventmgr']

Event

1016

:

SR-VM-DUMP

on

cn-tse-d04.gsslab.bjin

ha-d

enteris

suspendedb3)ESXiHostd.log

虚拟机电源状态变更记录2015-04-02T01:18:54.324Z

[382C2B70

info

'Vmsvc.vm:/vmfs/volumes/53fafab5-de3a7795-41ca-b8ca3a660468/SR-VM-DUMP/SR-VM-DUMP.vmx'

opID=HB-host-393@95202-2272ae4c-38

user=vpxuser]

State

Transition

(VM_STATE_ON

->VM_STATE_RECONFIGURING)2015-04-02T01:18:54.326Z

[382C2B70

info

'Vmsvc.vm:/vmfs/volumes/53fafab5-de3a7795-41ca-b8ca3a660468/SR-VM-DUMP/SR-VM-DUMP.vmx'

opID=HB-host-393@95202-2272ae4c-38

user=vpxuser]

State

Transition

(VM_STATE_RECONFIGURING

->VM_STATE_ON)2015-04-02T01:28:41.921Z

[383F4B70

verbose

'Vmsvc.vm:/vmfs/volumes/53fafab5-de3a7795-41ca-b8ca3a660468/SR-VM-DUMP/SR-VM-DUMP.vmx']

ToolsNamespaceGenerationInfo

callback

received.

VM

state

=

VM_STATE_ON2015-04-02T01:28:41.926Z

[38281B70

info

'Vmsvc.vm:/vmfs/volumes/53fafab5-de3a7795-41ca-b8ca3a660468/SR-VM-DUMP/SR-VM-DUMP.vmx']

State

Transition(VM_STATE_ON

->

VM_STATE_SUSPENDED)2015-04-02T01:28:44.602Z

[38040B70

info

'Vmsvc.vm:/vmfs/volumes/53fafab5-de3a7795-41ca-b8ca3a660468/SR-VM-DUMP/SR-VM-DUMP.vmx'

opID=HB-host-393@95240-7c713e98-39

user=vpxuser]

State

Transition

(VM_STATE_SUSPENDED

->VM_STATE_RECONFIGURING)2015-04-02T01:28:44.623Z

[38040B70

info

'Vmsvc.vm:/vmfs/volumes/53fafab5-de3a7795-41ca-b8ca3a660468/SR-VM-DUMP/SR-VM-DUMP.vmx'

opID=HB-host-393@95240-7c713e98-39

user=vpxuser]

State

Transition

(VM_STATE_RECONFIGURING

->VM_STATE_SUSPENDED)如何分析Hang机数据(日志分析)Windows

BSOD2014-11-25T08:04:35.638Z|

svga|

I120:2014-11-25T08:04:35.638Z|

svga|

I120:2014-11-25T08:04:35.638Z|

svga|

I120:WinBSOD:WinBSOD:WinBSOD:(

1)(3)(6)`***

Hardware

Malfunction

'`Call

your

hardware

vendor

for

support'`***

The

system

has

halted

***如何分析Hang机数据(分析举例-WinBSOD)vmware.logLinux

Hangvmware.log1028

2014-01-25T16:01:17.155Z|

vcpu-1|

I120:

APIC

THERMLVTwrite:0x100001029

2014-01-25T16:01:17.159Z|

vcpu-0|

I120:

APIC

THERMLVTwrite:0x100001030

2014-01-25T16:01:17.160Z|

vcpu-2|

I120:

APIC

THERMLVTwrite:0x100001031

2014-01-25T16:01:17.160Z|

vcpu-0|

I120:

Vix:

[4779444 mands.c:7433]:

VMAutomation_HandleCLIHLTEvent.

Donothing.1032

2014-01-25T16:01:17.160Z|

vcpu-3|

I120:

APIC

THERMLVTwrite:0x100001033

2014-01-25T16:01:17.160Z|

vcpu-0|

I120:MsgHint:

msg.monitorevent.halt1034

2014-01-25T16:01:17.160Z|

vcpu-0|

I120+The

CPUhas

been

disabled

by

the

guestoperating

system.

Power

off

or

reset

the

virtual

machine.1035

2014-01-25T16:01:17.160Z|

vcpu-0|

I120+

------------------------10362014-01-25T16:01:37.039Z|vmx|

I120:GuestRpcSendTimedOut:

message

totoolbox-dnd

timedout.1037

2014-01-25T16:01:37.039Z|vmx|

I120:GuestRpcSendTimedOut:

message

to

toolbox

timed

out.1038

2014-01-25T16:01:40.305Z|vmx|

I120:

Tools:

Tools

heartbeattimeout.Understanding

the

message:

The

CPU

has

been

disabled

by

the

guest

operating

system

(2000542)了解消息：CPU已被客户机操作系统禁用(2076195)如何分析Hang机数据(分析举例-Linux

Hang)2855

2014-10-30T02:29:06.863Z|

vcpu-7|

I120:

VMSAMPLE:

cs=0x10,rip=0xffffffff814ef33e2856

2014-10-30T02:29:06.863Z|

vcpu-8|

I120:

VMSAMPLE:

cs=0x10,rip=0xffffffff814ef33e2857

2014-10-30T02:29:06.863Z|vcpu-10|

I120:

VMSAMPLE:

cs=0x10,

rip=0xffffffff814ef33e2858

2014-10-30T02:29:06.863Z|

vcpu-5|

I120:

VMSAMPLE:

cs=0x10,rip=0xffffffff814ef33e2859

2014-10-30T02:29:06.863Z|

vcpu-3|

I120:

VMSAMPLE:

cs=0x10,rip=0xffffffff814ef33e2860

2014-10-30T02:29:06.863Z|

vcpu-9|

I120:

VMSAMPLE:

cs=0x10,rip=0xffffffff814ef33e2861

2014-10-30T02:29:07.877Z|

vcpu-3|

I120:

VMSAMPLE:

cs=0x10,rip=0xffffffff814ef33e2862

2014-10-30T02:29:07.877Z|

vcpu-5|

I120:

VMSAMPLE:

cs=0x10,rip=0xffffffff814ef33e2863

2014-10-30T02:29:07.877Z|vcpu-10|

I120:

VMSAMPLE:

cs=0x10,

rip=0xffffffff814ef33e2864

2014-10-30T02:29:07.877Z|

vcpu-9|

I120:

VMSAMPLE:

cs=0x10,rip=0xffffffff814ef33e2865

2014-10-30T02:29:07.877Z|

vcpu-7|

I120:

VMSAMPLE:

cs=0x10,rip=0xffffffff814ef33e2866

2014-10-30T02:29:07.877Z|

vcpu-8|

I120:

VMSAMPLE:

cs=0x10,rip=0xffffffff814ef33e2867

2014-10-30T02:29:07.877Z|

vcpu-6|

I120:

VMSAMPLE:

cs=0x10,rip=0xffffffff814ef33e2868

2014-10-30T02:29:08.891Z|

vcpu-7|

I120:

VMSAMPLE:

cs=0x10,rip=0xffffffff814ef33eCPU

在等待一个事件的发生，需要vmss文件该地址具体存放的指令内容如何分析Hang机数据(分析举例-指令地址分析)CPU指令地址采样分析vmware.log2854

2014-10-30T02:29:06.863Z|

vcpu-6|

I120:

VMSAMPLE:

cs=0x10,rip=0xffffffff814ef33e$

cat

vmware.log

|

grep

SAMPLE|

sed

-e

's/.*rip=//'|cut

-d"

"-f1|grep

"0x"

|

sort

|

uniq

-c197

0xffffffff8100ae022124

0xffffffff810193251

0xffffffff8101ee7b54

0xffffffff81398109CPU

可能在执行一个死循环或者运行非常缓慢,需要vmss文件该地址具体存放的指令内容2014-02-18T09:19:56.817Z|

vcpu-1|VMSAMPLE:cs=0x10,rip=0xffffffff81019325rip=0xffffffff81019325rip=0xffffffff81019325rip=0xffffffff81019325rip=0xffffffff81019325rip=0xffffffff810193252014-02-18T09:19:56.817Z|

vcpu-3|

VMSAMPLE:

cs=0x10,2014-02-18T09:19:56.817Z|

vcpu-4|

VMSAMPLE:

cs=0x10,2014-02-18T09:19:56.817Z|

vcpu-2|

VMSAMPLE:

cs=0x10,2014-02-18T09:19:56.817Z|

vcpu-6|

VMSAMPLE:

cs=0x10,2014-02-18T09:19:56.817Z|

vcpu-5|

VMSAMPLE:

cs=0x10,2014-02-18T09:19:56.817Z|

vcpu-9|

VMSAMPLE:

cs=0x10,rip=0xffffffff8100ae022014-02-18T09:19:56.817Z|

vcpu-7|

VMSAMPLE:

cs=0x10,

rip=0xffffffff810193252014-02-18T09:19:56.817Z|

vcpu-8|

VMSAMPLE:

cs=0x10,

rip=0xffffffff810193252014-02-18T09:19:56.817Z|

vcpu-10|

VMSAMPLE:

cs=0x10,

rip=0xffffffff810193252014-02-18T09:19:56.817Z|

vcpu-11|

VMSAMPLE:cs=0x10,

rip=0xffffffff81019325如何分析Hang机数据(分析举例-指令地址分析)CPU指令地址采样分析vmware.log1.

虚拟机电源挂起后将生成的vmss文件,将vmss文件转换成vmcore文件如何分析Hang机数据(分析举例-LinuxDump步骤)#vmss2core

-N6

.vmss2.

使用命令获取该Linux系统的内核版本号3.debuginfo

包并安装从rpm包安装符

（symbol）#

strings

vmcore

|

grep

"Kernel"#

rpm2cpio

kernel-debuginfo-N.N.N-N.elN.x86_64.rpm

|

cpio

-idv4.使用crash命令和第3步安装的vmlinux符打开第1步生成的vmcore文件>#

crash

vmlinux

vmcore…..GNU

gdb

(GDB)

7.6Copyright

(C)

2013

Free

Software

Foundation,

Inc.License

GPLv3+:

GNU

GPL

version

3

or

later

<This

is

free

software:

you

are

free

to

change

and

redistribute

it.There

is

NO

WARRANTY,

to

the

extent

permitted

by

law.

Type

"showcopying"and

"show

warranty"

for

details.This

GDBwas

configuredas

"x86_64-unknown-linux-gnu"...…..crash>5.

使用ps命令查找UN状态的进程列表，并逐一检查具体的backtrace如何分析Hang机数据(分析举例-LinuxDump步骤)crash>

ps

|

grepUN60 2

2

ffff88043504c040

UN

0.00 0

[khugepaged]164012ffff880434e81540UN0.0

1851324332

vmtoolsd177312ffff8804358d3500UN0.0

2554242036

rs:main

Q:Reg197110ffff880434d97540UN0.0

418123952

tlmagent.bin199710ffff880434c7aaa0UN0.0

1325603624

pbx_exchange2218

2217

1

ffff88043812d540

UN

0.

vasd252211

ffff8804373d1540

UN0.0

917482648

login256311

ffff880434cc9540

UN0.0

27636832

auditd273013

ffff8804346f4ae0

UN0.0

1330966956

bgsagent273127303ffff880437222aa0

UN0.0

1298004104

bgscollect398539843ffff880436106ae0

UN0.0

1072921956

ksh482648251ffff880437473540

UN0.0

108132988

ps580623502ffff8801174ab500

UN0.0

1672763004

bpcd58761

3

ffff8801080cc040

UN

0.3

event_demon60082343

0

ffff880436107540

UN

0.0

156672

1760

vnetd60636062

3

ffff880434592040

UN

0.0

107260

1948

ksh61601

2

ffff880435b64040

UN0.0

173723088

csampmuxf64141

2

ffff880435bf2ae0

UN0.4

112944063768

igateway64251

1

ffff88043662f500

UN0.4

112944063768

igateway64906489

3

ffff880436db6aa0

UN

0.0

107224

1936

ksh0.4

514172

62712

cybAgent.bin0.4

514172

62712

cybAgent.bin0.4

514172

62712

cybAgent.bin4784

sshd716

sadc768

mv6571 1

2

ffff880436e3caa0

UN6625 1

3

ffff880436cf2040

UN6637 1

2

ffff88043758caa0

UN7800

2314

2

ffff880437099540

UN 0.0

1030928660

8650

0

ffff88043662e040

UN

0.0

41208663

8662

3

ffff8804366df540

UN 0.0

1678424236 1

0

ffff880437098ae0

UN

0.4

1129440

63768

igateway32626

2314

2

ffff88018d4f4ae0

UN

0.0

104304

5832

sshd32629 1

2

ffff88010060aaa0

UN

0.0

107228

1940

kshcrash>

set

ffff880437222aa0RU

:

Running

or

runnableIN

:

InterruptableUN

:un-interruptableZO

:

ZombieST

:

stoppedDE

:

deadSW

:

Swap6.

检查具体一个UN进程的backtrace如何分析Hang机数据(分析举例-LinuxDump步骤)crash>

set

ffff880437222aa0PID:

2731COMMAND:

"bgscollect"TASK:

ffff880437222aa0

[THREAD_INFO:

ffff880436518000]CPU:

3STATE:

TASK_UNINTERRUPTIBLEcrash>

btPID:

2731

TASK:

ffff880437222aa0

CPU:

3

COMMAND:

"bgscollect"#0

[ffff880436519bc8]schedule

at

ffffffff8150d692#1

[ffff880436519c90]

sb_start_write

at

ffffffff81182f4c#2

[ffff880436519d20]

generic_file_aio_write

at

ffffffff8111c379#3

[ffff880436519d70]

ext4_file_write

at

ffffffffa00d9fb1

[ext4]#4

[ffff880436519dc0]

do_sync_write

at

ffffffff81180c9a#5

[ffff880436519ef0]

vfs_write

at

ffffffff81180f98#6

[ffff880436519f30]

sys_write

at

ffffffff81181891#7

[ffff880436519f80]

system_call_fastpath

at

ffffffff8100b072…..crash>

files

list

open

filescrash>file

display

file

structcrash>

filesPID:

2731

TASK:

ffff880437222aa0

CPU:

3

COMMAND:

"bgscollect"ROOT:

/ CWD:

/tech/best1/Patrol3/Linux-2-6-x86-64/best1/9.0.00/bgs/monitor/logFD

FILE

DENTRY

INODE TYPE

PATH…..2

ffff880434630980

ffff880439f1c840

ffff88043a2b1108

CHR

/dev/null14ffff8804387ca9c0

ffff88042a21ccc0

ffff88042a22e8c0

REG

/tech/best1/Patrol3/Linux-2-6-x86-64/best1/9.0.00/bgs/log/asylvhp01-bgsagent_6767.log15

ffff880437d27d40

ffff88043312d140

ffff88042a0bc8c0

REG…..20ffff880434e51380

ffff88042a21e380

ffff88042a14dcc0

REG

/tech/best1/Patrol3/Linux-2-6-x86-64/best1/9.0.00/bgs/log/asylvhp01-bgscollect-noInstance.log21ffff880436b79ec0

ffff88019024d240

ffff88042a14d8c0

REG

/tech/best1/Patrol3/Linux-2-6-x86-64/best1/9.0.00/bgs/log/asylvhp01-bgscollect-noInstance-status.log…..如何分析Hang机数据(分析举例-LinuxDump步骤)crash>

file

ffff880436519ec8struct

file

{f_u