解决AIX下识别HDS磁盘阵列多路径问题.doc

解决AIX下识别HDS磁盘阵列多路径问题公司有台日立HDS AMS2003磁盘阵列，基本还没怎么使用，于是尝试分配了4个500G的LUN给一台测试服务器，都做了冗余，每块盘都有4条路径，分配好后在AIX识别出来了，但每块盘都重复了4次。因为是以前使用的都是IBM的阵列，没发现过有这样的问题，于是发贴求助 新划的磁盘阵列，lspv显示重复了4次/club/thread-78460-1-1.html 通过查询资料确定了是多路径的问题，需要用到HDLML软件，印象中听日立原厂的工程师提到过，于是QQ联系日立原厂工程师拿到了安装介质及license，并在指导下解决了问题，以下是我根据记录的日志整理的步骤。1 问题 磁盘阵列使用的是日立HDS AMS2003，RAID之前就做好了，现在我分配了4个500G的LUN给一台AIX服务器使用，但在AIX识别时居然重复了4次test2:root:/ lspvhdisk0 00c48a56795bc09d rootvg activehdisk1 00c48a566bbc5382 rootvg activehdisk2 00c48a562bbcd401 None hdisk3 00c48a562bbe131f None hdisk4 00c48a562bbe139b None hdisk5 00c48a562bbe1422 None hdisk6 00c48a562bbcd401 None hdisk7 00c48a562bbe131f None hdisk8 00c48a562bbe139b None hdisk9 00c48a562bbe1422 None hdisk10 00c48a562bbcd401 None hdisk11 00c48a562bbe131f None hdisk12 00c48a562bbe139b None hdisk13 00c48a562bbe1422 None hdisk14 00c48a562bbcd401 None hdisk15 00c48a562bbe131f None hdisk16 00c48a562bbe139b None hdisk17 00c48a562bbe1422 None test2:root:/ lspathEnabled hdisk0 sas0Enabled hdisk1 sas0Available ses0 sas0Available ses1 sas0 test2:root:/ lsdev -Cc diskhdisk0 Available 03-08-00 SAS Disk Drivehdisk1 Available 03-08-00 SAS Disk Drivehdisk2 Available 02-01-02 Other FC SCSI Disk Drivehdisk3 Available 02-01-02 Other FC SCSI Disk Drivehdisk4 Available 02-01-02 Other FC SCSI Disk Drivehdisk5 Available 02-01-02 Other FC SCSI Disk Drivehdisk6 Available 02-01-02 Other FC SCSI Disk Drivehdisk7 Available 02-01-02 Other FC SCSI Disk Drivehdisk8 Available 02-01-02 Other FC SCSI Disk Drivehdisk9 Available 02-01-02 Other FC SCSI Disk Drivehdisk10 Available 06-01-02 Other FC SCSI Disk Drivehdisk11 Available 06-01-02 Other FC SCSI Disk Drivehdisk12 Available 06-01-02 Other FC SCSI Disk Drivehdisk13 Available 06-01-02 Other FC SCSI Disk Drivehdisk14 Available 06-01-02 Other FC SCSI Disk Drivehdisk15 Available 06-01-02 Other FC SCSI Disk Drivehdisk16 Available 06-01-02 Other FC SCSI Disk Drivehdisk17 Available 06-01-02 Other FC SCSI Disk Drive2 解决过程2.1 license 上传license文件HDS HDLM AIX.plk到/var/tmp，并改名为hdlm_licensetest2:root:/ cd /var/tmptest2:root:/var/tmp ls.oracle dpid2.log oradiag_oracle snmpdv3.logHDS HDLM AIX.plk hostmibd.log slp_srvreg.lock snmpmibd.logaixmibd.log installpbx-2228732-031811161323.log snmpd.logtest2:root:/var/tmp mv HDS HDLM AIX.plk hdlm_licensetest2:root:/var/tmp ls.oracle hdlm_license oradiag_oracle snmpdv3.logaixmibd.log hostmibd.log slp_srvreg.lock snmpmibd.logdpid2.log installpbx-2228732-031811161323.log snmpd.log2.2 上传HDLM软件镜像文件并加载镜HDLM软件为一个ISO镜像文件，包含各个平台的HDLM软件，还从来没在AIX系统中加载过ISO文件，于是马上上网查询，终于找到了一个最简单的办法mount ISO镜像：#loopmount -i dlmglm_072001.iso -o -V cdrfs -o ro -m /mntunmount ISO镜像：#fuser -cuk /mnt #unmount /mnt2.3 安装ODMtest2:root:/var/tmp cd /mnttest2:root:/mnt lsAutorun.inf HDLM_AIX HDLM_Solaris HDLM_Windows PSP index.html rr_movedContents_list.txt HDLM_Linux HDLM_VMware HGLM hcmdslicense2 installux.shtest2:root:/mnt cd HDLM_AIXtest2:root:/mnt/HDLM_AIX ls.toc AIX_ODM DLManager.mpio.bff hdlmtool netmdm_sampletest2:root:/mnt/HDLM_AIX cd AIX_ODMtest2:root:/mnt/HDLM_AIX/AIX_ODM lsHTC_ODM XP_ODM readme.txttest2:root:/mnt/HDLM_AIX/AIX_ODM cd HTC*test2:root:/mnt/HDLM_AIX/AIX_ODM/HTC_ODM lsHTCODM3.tar HTC_readme.txttest2:root:/mnt/HDLM_AIX/AIX_ODM/HTC_ODM mkdir /tmp/hdlmtest2:root:/mnt/HDLM_AIX/AIX_ODM/HTC_ODM cp *.tar /tmp/hdlmtest2:root:/mnt/HDLM_AIX/AIX_ODM/HTC_ODM cd /tmp/hdlmtest2:root:/tmp/hdlm lsHTCODM3.tartest2:root:/tmp/hdlm tar xBpf *test2:root:/tmp/hdlm lsHTCODM3.tar HTC_FC_ODM_5001I HTC_FC_ODM_5005U HTC_FC_ODM_50521U HTC_FC_ODM_50522U HTC_FC_ODM_50523Utest2:root:/tmp/hdlm smitty installp通过以上步骤即完成了ODM的安装test2:root:/tmp/hdlm lslpp -l |grep -i dlmtest2:root:/tmp/hdlm lslpp -l |grep -i hitachiHitachi.aix.support.rte COMMITTED AIX Support for Hitachi Disk2.4 安装HDLMtest2:root:/tmp/hdlm cd /mnttest2:root:/mnt lsAutorun.inf HDLM_AIX HDLM_Solaris HDLM_Windows PSP index.html rr_movedContents_list.txt HDLM_Linux HDLM_VMware HGLM hcmdslicense2 installux.shtest2:root:/mnt cd HDLM_AIXtest2:root:/mnt/HDLM_AIX smitty installp通过以上步骤即完成HDLM的安装test2:root:/mnt/HDLM_AIX lslpp -l |grep -i dlmDLManager.mpio.rte COMMITTED Dynamic Link ManagerDLManager.mpio.rte COMMITTED Dynamic Link Manager2.5 重新识别磁盘test2:root:/mnt/HDLM_AIX cfgmgrtest2:root:/mnt/HDLM_AIX lsdev -Cc diskhdisk0 Available 03-08-00 SAS Disk Drivehdisk1 Available 03-08-00 SAS Disk Drivehdisk2 Available 02-01-02 Other FC SCSI Disk Drivehdisk3 Available 02-01-02 Other FC SCSI Disk Drivehdisk4 Available 02-01-02 Other FC SCSI Disk Drivehdisk5 Available 02-01-02 Other FC SCSI Disk Drivehdisk6 Available 02-01-02 Other FC SCSI Disk Drivehdisk7 Available 02-01-02 Other FC SCSI Disk Drivehdisk8 Available 02-01-02 Other FC SCSI Disk Drivehdisk9 Available 02-01-02 Other FC SCSI Disk Drivehdisk10 Available 06-01-02 Other FC SCSI Disk Drivehdisk11 Available 06-01-02 Other FC SCSI Disk Drivehdisk12 Available 06-01-02 Other FC SCSI Disk Drivehdisk13 Available 06-01-02 Other FC SCSI Disk Drivehdisk14 Available 06-01-02 Other FC SCSI Disk Drivehdisk15 Available 06-01-02 Other FC SCSI Disk Drivehdisk16 Available 06-01-02 Other FC SCSI Disk Drivehdisk17 Available 06-01-02 Other FC SCSI Disk Drivehdisk18 Available 02-01-02 Hitachi Disk Array (Fibre)hdisk19 Available 02-01-02 Hitachi Disk Array (Fibre)hdisk20 Available 02-01-02 Hitachi Disk Array (Fibre)hdisk21 Available 02-01-02 Hitachi Disk Array (Fibre)可以看到hdisk18至hdisk21已经识别为Hitachi的盘了2.6 检查HDLM相关信息test2:root:/mnt/HDLM_AIX cd /usr/D*/bintest2:root:/usr/DynamicLinkManager/bin ./dlnkmgr view -luProduct : AMSSerialNumber : 85013612 LUs : 4iLU HDevName OSPathID PathID Status 0011 hdisk21 00000 000000 Online 00001 000001 Online 00002 000002 Online 0012 hdisk18 00000 000007 Online 00001 000008 Online 00002 000009 Online 00003 000010 Online 0013 hdisk19 00000 000011 Online 00001 000012 Online 00002 000013 Online 00003 000014 Online 0014 hdisk20 00000 000003 Online 00001 000004 Online 00002 000005 Online 00003 000006 Online KAPL01001-I The HDLM command completed normally. Operation name = view, completion time = 2012/05/09 15:16:222.7 修改磁盘队列深度test2:root:/usr/DynamicLinkManager/bin chdev -l hdisk18 -a queue_depth=8hdisk18 changedtest2:root:/usr/DynamicLinkManager/bin chdev -l hdisk19 -a queue_depth=8hdisk19 changedtest2:root:/usr/DynamicLinkManager/bin r 19=20chdev -l hdisk20 -a queue_depth=8hdisk20 changedtest2:root:/usr/DynamicLinkManager/bin r 20=21chdev -l hdisk21 -a queue_depth=8hdisk21 changedtest2:root:/usr/DynamicLinkManager/bin lsdev -Cc diskhdisk0 Available 03-08-00 SAS Disk Drivehdisk1 Available 03-08-00 SAS Disk Drivehdisk2 Available 02-01-02 Other FC SCSI Disk Drivehdisk3 Available 02-01-02 Other FC SCSI Disk Drivehdisk4 Available 02-01-02 Other FC SCSI Disk Drivehdisk5 Available 02-01-02 Other FC SCSI Disk Drivehdisk6 Available 02-01-02 Other FC SCSI Disk Drivehdisk7 Available 02-01-02 Other FC SCSI Disk Drivehdisk8 Available 02-01-02 Other FC SCSI Disk Drivehdisk9 Available 02-01-02 Other FC SCSI Disk Drivehdisk10 Available 06-01-02 Other FC SCSI Disk Drivehdisk11 Available 06-01-02 Other FC SCSI Disk Drivehdisk12 Available 06-01-02 Other FC SCSI Disk Drivehdisk13 Available 06-01-02 Other FC SCSI Disk Drivehdisk14 Available 06-01-02 Other FC SCSI Disk Drivehdisk15 Available 06-01-02 Other FC SCSI Disk Drivehdisk16 Available 06-01-02 Other FC SCSI Disk Drivehdisk17 Available 06-01-02 Other FC SCSI Disk Drivehdisk18 Available 02-01-02 Hitachi Disk Array (Fibre)hdisk19 Available 02-01-02 Hitachi Disk Array (Fibre)hdisk20 Available 02-01-02 Hitachi Disk Array (Fibre)hdisk21 Available 02-01-02 Hitachi Disk Array (Fibre)test2:root:/usr/DynamicLinkManager/bin lspathEnabled hdisk0 sas0Enabled hdisk1 sas0Enabled hdisk18 fscsi1Enabled hdisk18 fscsi1Enabled hdisk18 fscsi3Enabled hdisk18 fscsi3Enabled hdisk19 fscsi1Enabled hdisk19 fscsi1Enabled hdisk19 fscsi3Enabled hdisk19 fscsi3Enabled hdisk20 fscsi1Enabled hdisk20 fscsi1Enabled hdisk20 fscsi3Enabled hdisk20 fscsi3Enabled hdisk21 fscsi1Enabled hdisk21 fscsi3Enabled hdisk21 fscsi3Available ses0 sas0Available ses1 sas0test2:root:/usr/DynamicLinkManager/bin lsattr -El fcs1bus_intr_lvl Bus interrupt level Falsebus_io_addr 0xffc00 Bus I/O address Falsebus_mem_addr 0xffe7f000 Bus memory address Falsebus_mem_addr2 0xffe7d000 Bus memory address Falseinit_link al INIT Link flags Trueintr_msi_1 534 Bus interrupt level Falseintr_priority 3 Interrupt priority Falselg_term_dma 0x800000 Long term DMA Truemax_xfer_size 0x100000 Maximum Transfer Size Truenum_cmd_elems 200 Maximum number of COMMANDS to queue to the adapter Truepref_alpa 0x1 Preferred AL_PA Truesw_fc_class 2 FC Class for Fabric Truetme no Target Mode Enabled Truetest2:root:/usr/DynamicLinkManager/bin lsatt -El fscsi1ksh: lsatt: not found.test2:root:/usr/DynamicLinkManager/bin lsattr -El fscsi1attach switch How this adapter is CONNECTED Falsedyntrk no Dynamic Tracking of FC Devices Truefc_err_recov delayed_fail FC Fabric Event Error RECOVERY Policy Truescsi_id 0x10500 Adapter SCSI ID Falsesw_fc_class 3 FC Class for Fabric Truetest2:root:/usr/DynamicLinkManager/bin lsattr -El fscsi3attach switch How this adapter is CONNECTED Falsedyntrk no Dynamic Tracking of FC Devices Truefc_err_recov delayed_fail FC Fabric Event Error RECOVERY Policy Truescsi_id 0x10300 Adapter SCSI ID Falsesw_fc_class 3 FC Class for Fabric Truetest2:root:/usr/DynamicLinkManager/bin rmdev -dl hdisk18hdisk18 deletedtest2:root:/usr/DynamicLinkManager/bin r 18=19rmdev -dl hdisk19hdisk19 deletedtest2:root:/usr/DynamicLinkManager/bin r 19=20rmdev -dl hdisk20hdisk20 deletedtest2:root:/usr/DynamicLinkManager/bin r 20=21rmdev -dl hdisk21hdisk21 deletedtest2:root:/usr/DynamicLinkManager/bin test2:root:/usr/DynamicLinkManager/bin test2:root:/usr/DynamicLinkManager/bin 2.8 修改HBA卡参数，以满足HDLM需要test2:root:/usr/DynamicLinkManager/bin rmdev -l fcs1 -Rfcnet1 Definedsfwcomm1 Definedhdisk2 Definedhdisk3 Definedhdisk4 Definedhdisk5 Definedhdisk6 Definedhdisk7 Definedhdisk8 Definedhdisk9 Definedfscsi1 Definedfcs1 Definedtest2:root:/usr/DynamicLinkManager/bin rmdev -l fcs3 -Rfcnet3 Definedsfwcomm3 Definedhdisk10 Definedhdisk11 Definedhdisk12 Definedhdisk13 Definedhdisk14 Definedhdisk15 Definedhdisk16 Definedhdisk17 Definedfscsi3 Definedfcs3 Definedtest2:root:/usr/DynamicLinkManager/bin chdev -l fscsi1 -a fc_err_recov=fast_failfscsi1 changedtest2:root:/usr/DynamicLinkManager/bin r 1=3chdev -l fscsi3 -a fc_err_recov=fast_failfscsi3 changedtest2:root:/usr/DynamicLinkManager/bin test2:root:/usr/DynamicLinkManager/bin test2:root:/usr/DynamicLinkManager/bin test2:root:/usr/DynamicLinkManager/bin cfgmgr test2:root:/usr/DynamicLinkManager/bin lsdev -Cc diskhdisk0 Available 03-08-00 SAS Disk Drivehdisk1 Available 03-08-00 SAS Disk Drivehdisk2 Available 02-01-02 Hitachi Disk Array (Fibre)hdisk3 Available 02-01-02 Hitachi Disk Array (Fibre)hdisk4 Available 02-01-02 Hitachi Disk Array (Fibre)hdisk5 Available 02-01-02 Hitachi Disk Array (Fibre)hdisk6 Defined 02-01-02 Other FC SCSI Disk Drivehdisk7 Defined 02-01-02 Other FC SCSI Disk Drivehdisk8 Defined 02-01-02 Other FC SCSI Disk Drivehdisk9 Defined 02-01-02 Other FC SCSI Disk Drivehdisk10 Defined 06-01-02 Other FC SCSI Disk Drivehdisk11 Defined 06-01-02 Other FC SCSI Disk Drivehdisk12 Defined 06-01-02 Other FC SCSI Disk Drivehdisk13 Defined 06-01-02 Other FC SCSI Disk Drivehdisk14 Defined 06-01-02 Other FC SCSI Disk Drivehdisk15 Defined 06-01-02 Other FC SCSI Disk Drivehdisk16 Define