new-sfrac简要操作手册.doc

VERITAS 大中国区顾问服务 舟山大宗商品交易所Storage Foundation for RAC操作手册VERITAS Consulting ServicesGreater China 大中国区顾问服务1 文件信息VersionDocument Title舟山大宗商品交易所Storage Foundation for RAC操作手册SubjectVERITAS 大中国区的顾问服务Version1.0Date2011/10AuthorTeller Filenameoperation.docComments作者这一份文件可以与下列作者取的联系Teller PanConsultantVERITAS Consulting Service Greater ChinaTeller_相关的文件IdTitleOrganisationAuthorVersion 2 目录1文件信息2Version2作者2相关的文件22目录33Storage Foundation for RAC简单操作43.1命令行操作简介43.1.0如何重启主机43.1.1检查cluster状态43.1.2 启动服务组53.1.3 停止服务组53.1.4 切换服务组53.1.5 停止某个资源53.1.6启动某个资源53.1.7发现某个资源出错后清除错误53.1.8如果资源无法启动或停止需要flush状态63.1.9查看心跳信息63.1.10查看cluster支持RAC的状态63.1.11查看磁盘状态63.1.12查看 disk group 状态73.1.13查看disk group 剩余空间73.1.14查看 volume 的状态83.1.15 添加一个disk group133.1.16在disk group中添加一块盘133.1.17创建stripe的volume133.1.18创建文件系统133.1.19 mount文件系统133.1.20查看cluster的日志143.1.21查看volume manager的日志143.1.22删除一个volume143.1.23删除一个disk group143.1.24如何扩大或缩小文件系统143.1.25如何禁止fencing153.1.26 shutdown说明153.1.27检查磁盘的serial number153.1.28如何查看license key163.1.29如何添加license key163.1.30如何查看多路径信息163.1.31如何让cluster两主机都停止后启动一台主机提供服务163.1.32如何停止一个HBA卡上的I/O（用于升级微码）163.1.33如何更换一个HBA卡173.1.34如何添加一个oracle使用的裸设备173.1.35如何停止cluster软件173.2 图形界面简介173.2.1 cluster图形界面173.2.2 Volume Manager图形界面283 Storage Foundation for RAC简单操作Storage Foundation for RAC的操作可以采用命令行、java图形界面和web界面进行。3.1命令行操作简介3.1.0如何重启主机SFRAC for AIX环境下要使用shutdown Fr命令来重启主机，不能使用 reboot命令shutdown Fr命令会执行/etc/rc.d下的K脚本，会正常地停止cluster软件，而reboot不会。如果使用reboot命令重启后，需要查看errpt,如果有fencing的报错，并且无法import disk group, 就要使用 vxfenclearpre命令后重启主机才能恢复正常。 3.1.1检查cluster状态jxysdb1:/#hastatus sum 查看总体的cluster状态# hastatusattempting to connect.attempting to connect.connectedgroup resource system message - - - - kadb1 RUNNING kadb2 RUNNING ClusterService kadb1 ONLINE ClusterService kadb2 OFFLINE -cvm kadb1 ONLINE cvm kadb2 ONLINE rac-sg kadb1 ONLINE rac-sg kadb2 ONLINE wac kadb1 ONLINE - wac kadb2 OFFLINE webip kadb1 ONLINE webip kadb2 OFFLINE csgnic kadb1 ONLINE csgnic kadb2 ONLINE - ocr-mnt-res kadb1 ONLINE ocr-mnt-res kadb2 ONLINE vxfsckd kadb1 ONLINE vxfsckd kadb2 ONLINE cvm_clus kadb1 ONLINE - cvm_clus kadb2 ONLINE ocrvotedg-res kadb1 ONLINE ocrvotedg-res kadb2 ONLINE cvm_vxconfigd kadb1 ONLINE cvm_vxconfigd kadb2 ONLINE - cssd kadb1 ONLINE cssd kadb2 ONLINE arch-mnt-res kadb1 ONLINE arch-mnt-res kadb2 ONLINE data-mnt-res kadb1 ONLINE - data-mnt-res kadb2 ONLINE datadg-res kadb1 ONLINE datadg-res kadb2 ONLINE 3.1.2 启动服务组#hagrp online -sys #hagrp online cvm sys kadb13.1.3 停止服务组#hagrp offline -sys #hagrp offline cvm sys kadb13.1.4 切换服务组这个操作相当于在一边停止服务组，在另一边驱动服务组#hagrp switch -to #hagrp switch ClusterService to kadb13.1.5 停止某个资源#hares offline -sys #hares offline cssd sys kadb13.1.6启动某个资源#hares online -sys #hares online cssd sys kadb13.1.7发现某个资源出错后清除错误#hares clear -sys # hares -clear cssd -sys kadb13.1.8如果资源无法启动或停止需要flush状态#hagrp flush -sys # hagrp -flush rac-sg -sys kadb13.1.9查看心跳信息jxysdb1:/#gabconfig -aGAB Port Memberships=Port a gen 705d01 membership 01Port b gen 705d07 membership 01Port d gen 705d06 membership 01Port f gen 705d0f membership 01Port h gen 705d05 membership 01Port o gen 705d04 membership 01Port v gen 705d0b membership 01Port w gen 705d0d membership 013.1.10查看cluster支持RAC的状态jxysdb1:/#vxdctl -c modemode: enabled: cluster active - SLAVEmaster: jxysdb2相当与oracle中的lsnodes3.1.11查看磁盘状态# vxdisk -o alldgs listDEVICE TYPE DISK GROUP STATUSdisk_0 auto:LVM - - LVMdisk_1 auto:LVM - - LVMibm_ds8x000_5000 auto:cdsdisk ibm_ds8x000_0 datadg online sharedibm_ds8x000_5001 auto:cdsdisk ibm_ds8x000_1 datadg online sharedibm_ds8x000_5100 auto:cdsdisk ibm_ds8x000_2 datadg online sharedibm_ds8x000_5101 auto:cdsdisk ibm_ds8x000_3 datadg online sharedibm_ds8x000_5200 auto:cdsdisk ibm_ds8x000_4 datadg online sharedibm_ds8x000_5300 auto:cdsdisk ibm_ds8x000_5 datadg online sharedibm_ds8x000_5301 auto:cdsdisk ibm_ds8x000_6 datadg online sharedibm_ds8x000_5400 auto:cdsdisk ibm_ds8x000_7 datadg online sharedibm_ds8x000_5401 auto:cdsdisk ibm_ds8x000_8 datadg online sharedibm_ds8x000_5500 auto:cdsdisk ibm_ds8x000_9 datadg online sharedibm_ds8x000_5501 auto:cdsdisk ibm_ds8x000_10 datadg online sharedibm_ds8x000_5600 auto:cdsdisk ibm_ds8x000_11 datadg online sharedibm_ds8x000_5601 auto:cdsdisk ibm_ds8x000_12 datadg online sharedibm_ds8x000_5700 auto:cdsdisk ibm_ds8x000_13 datadg online sharedibm_ds8x000_5701 auto:cdsdisk ibm_ds8x000_14 datadg online sharedibm_ds8x000_9004 auto:cdsdisk ibm_ds8x000_15 ocrvotedg online sharedibm_ds8x000_9104 auto:cdsdisk ibm_ds8x000_16 ocrvotedg online sharedibm_ds8x000_9204 auto:cdsdisk - (vxfencoorddg) onlineibm_ds8x000_9304 auto:cdsdisk - (vxfencoorddg) onlineibm_ds8x000_9404 auto:cdsdisk - (vxfencoorddg) online#3.1.12查看 disk group 状态# vxdg listNAME STATE IDdatadg enabled,shared,cds 1318391345.111.wlmhdb1ocrvotedg enabled,shared,cds 1318391230.109.wlmhdb13.1.13查看disk group 剩余空间# vxdg freeGROUP DISK DEVICE TAG OFFSET LENGTH FLAGSdatadg ibm_ds8x000_4 ibm_ds8x000_5200 ibm_ds8x000_5200 185440320 24199600 -datadg ibm_ds8x000_12 ibm_ds8x000_5601 ibm_ds8x000_5601 170920560 38719360 -datadg ibm_ds8x000_13 ibm_ds8x000_5700 ibm_ds8x000_5700 0 209639920 -datadg ibm_ds8x000_14 ibm_ds8x000_5701 ibm_ds8x000_5701 0 209639920 -ocrvotedg ibm_ds8x000_15 ibm_ds8x000_9004 ibm_ds8x000_9004 1946144 81120 -ocrvotedg ibm_ds8x000_16 ibm_ds8x000_9104 ibm_ds8x000_9104 1946144 81120 -3.1.14查看 volume 的状态# vxprint -vDisk group: datadgTY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0v archvol fsgen ENABLED 1638400000 - ACTIVE - -v datavol fsgen ENABLED 1024000000 - ACTIVE - -Disk group: ocrvotedgTY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0v ocrvotevol fsgen ENABLED 1945600 - ACTIVE - -v ocrvotevol_dcl gen ENABLED 544 - ACTIVE - -# vxprint -htfDisk group: datadgDG NAME NCONFIG NLOG MINORS GROUP-IDST NAME STATE DM_CNT SPARE_CNT APPVOL_CNTDM NAME DEVICE TYPE PRIVLEN PUBLEN STATERV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRLRL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNKCO NAME CACHEVOL KSTATE STATEVT NAME RVG KSTATE STATE NVOLUMEV NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPEPL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODESD NAME PLEX DISK DISKOFFS LENGTH COL/OFF DEVICE MODESV NAME PLEX VOLNAME NVOLLAYR LENGTH COL/OFF AM/NM MODESC NAME PLEX CACHE DISKOFFS LENGTH COL/OFF DEVICE MODEDC NAME PARENTVOL LOGVOLSP NAME SNAPVOL DCOEX NAME ASSOC VC PERMS MODE STATESR NAME KSTATEdg datadg default default 52000 1318391345.111.wlmhdb1dm ibm_ds8x000_0 ibm_ds8x000_5000 auto 65536 209639920 -dm ibm_ds8x000_1 ibm_ds8x000_5001 auto 65536 209639920 -dm ibm_ds8x000_2 ibm_ds8x000_5100 auto 65536 209639920 -dm ibm_ds8x000_3 ibm_ds8x000_5101 auto 65536 209639920 -dm ibm_ds8x000_4 ibm_ds8x000_5200 auto 65536 209639920 -dm ibm_ds8x000_5 ibm_ds8x000_5300 auto 65536 209639920 -dm ibm_ds8x000_6 ibm_ds8x000_5301 auto 65536 209639920 -dm ibm_ds8x000_7 ibm_ds8x000_5400 auto 65536 209639920 -dm ibm_ds8x000_8 ibm_ds8x000_5401 auto 65536 209639920 -dm ibm_ds8x000_9 ibm_ds8x000_5500 auto 65536 209639920 -dm ibm_ds8x000_10 ibm_ds8x000_5501 auto 65536 209639920 -dm ibm_ds8x000_11 ibm_ds8x000_5600 auto 65536 209639920 -dm ibm_ds8x000_12 ibm_ds8x000_5601 auto 65536 209639920 -dm ibm_ds8x000_13 ibm_ds8x000_5700 auto 65536 209639920 -dm ibm_ds8x000_14 ibm_ds8x000_5701 auto 65536 209639920 -v archvol - ENABLED ACTIVE 1638400000 SELECT - fsgenpl archvol-01 archvol ENABLED ACTIVE 1638400000 CONCAT - RWsd ibm_ds8x000_5-01 archvol-01 ibm_ds8x000_5 0 209639920 0 ibm_ds8x000_5300 ENAsd ibm_ds8x000_6-01 archvol-01 ibm_ds8x000_6 0 209639920 209639920 ibm_ds8x000_5301 ENAsd ibm_ds8x000_7-01 archvol-01 ibm_ds8x000_7 0 209639920 419279840 ibm_ds8x000_5400 ENAsd ibm_ds8x000_8-01 archvol-01 ibm_ds8x000_8 0 209639920 628919760 ibm_ds8x000_5401 ENAsd ibm_ds8x000_9-01 archvol-01 ibm_ds8x000_9 0 209639920 838559680 ibm_ds8x000_5500 ENAsd ibm_ds8x000_10-01 archvol-01 ibm_ds8x000_10 0 209639920 1048199600 ibm_ds8x000_5501 ENAsd ibm_ds8x000_11-01 archvol-01 ibm_ds8x000_11 0 209639920 1257839520 ibm_ds8x000_5600 ENAsd ibm_ds8x000_12-01 archvol-01 ibm_ds8x000_12 0 170920560 1467479440 ibm_ds8x000_5601 ENAv datavol - ENABLED ACTIVE 1024000000 SELECT - fsgenpl datavol-01 datavol ENABLED ACTIVE 1024000000 CONCAT - RWsd ibm_ds8x000_0-01 datavol-01 ibm_ds8x000_0 0 209639920 0 ibm_ds8x000_5000 ENAsd ibm_ds8x000_1-01 datavol-01 ibm_ds8x000_1 0 209639920 209639920 ibm_ds8x000_5001 ENAsd ibm_ds8x000_2-01 datavol-01 ibm_ds8x000_2 0 209639920 419279840 ibm_ds8x000_5100 ENAsd ibm_ds8x000_3-01 datavol-01 ibm_ds8x000_3 0 209639920 628919760 ibm_ds8x000_5101 ENAsd ibm_ds8x000_4-01 datavol-01 ibm_ds8x000_4 0 185440320 838559680 ibm_ds8x000_5200 ENADisk group: ocrvotedgDG NAME NCONFIG NLOG MINORS GROUP-IDST NAME STATE DM_CNT SPARE_CNT APPVOL_CNTDM NAME DEVICE TYPE PRIVLEN PUBLEN STATERV NAME RLINK_CNT KSTATE STATE PRIMARY DATAVOLS SRLRL NAME RVG KSTATE STATE REM_HOST REM_DG REM_RLNKCO NAME CACHEVOL KSTATE STATEVT NAME RVG KSTATE STATE NVOLUMEV NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPEPL NAME VOLUME KSTATE STATE LENGTH LAYOUT NCOL/WID MODESD NAME PLEX DISK DISKOFFS LENGTH COL/OFF DEVICE MODESV NAME PLEX VOLNAME NVOLLAYR LENGTH COL/OFF AM/NM MODESC NAME PLEX CACHE DISKOFFS LENGTH COL/OFF DEVICE MODEDC NAME PARENTVOL LOGVOLSP NAME SNAPVOL DCOEX NAME ASSOC VC PERMS MODE STATESR NAME KSTATEdg ocrvotedg default default 60000 1318391230.109.wlmhdb1dm ibm_ds8x000_15 ibm_ds8x000_9004 auto 65536 2027264 -dm ibm_ds8x000_16 ibm_ds8x000_9104 auto 65536 2027264 -v ocrvotevol - ENABLED ACTIVE 1945600 SELECT - fsgenpl ocrvotevol-01 ocrvotevol ENABLED ACTIVE 1945600 CONCAT - RWsd ibm_ds8x000_15-01 ocrvotevol-01 ibm_ds8x000_15 0 1945600 0 ibm_ds8x000_9004 ENApl ocrvotevol-02 ocrvotevol ENABLED ACTIVE 1945600 CONCAT - RWsd ibm_ds8x000_16-01 ocrvotevol-02 ibm_ds8x000_16 0 1945600 0 ibm_ds8x000_9104 ENAdc ocrvotevol_dco ocrvotevol ocrvotevol_dclv ocrvotevol_dcl - ENABLED ACTIVE 544 SELECT - genpl ocrvotevol_dcl-01 ocrvotevol_dcl ENABLED ACTIVE 544 CONCAT - RWsd ibm_ds8x000_15-02 ocrvotevol_dcl-01 ibm_ds8x000_15 1945600 544 0 ibm_ds8x000_9004 ENApl ocrvotevol_dcl-02 ocrvotevol_dcl ENABLED ACTIVE 544 CONCAT - RWsd ibm_ds8x000_16-02 ocrvotevol_dcl-02 ibm_ds8x000_16 1945600 544 0 ibm_ds8x000_9104 ENA3.1.15 添加一个disk group添加cluster的disk groupvxdg -s init 添加普通的disk groupvxdg init 3.1.16在disk group中添加一块盘#vxdg g adddisk 3.1.17创建stripe的volume#vxassist g make 500g layout=stripe stripeunit=64k ncol=23.1.18创建文件系统#mkfs V vxfs o largefiles /dev/vx/rdsk/ 3.1.19 mount文件系统mount cluster的文件系统:#mount V vxfs o cluster,largefiles /dev/vx/dsk/ /mount文件系统:#mount V vxfs o largefiles /dev/vx/dsk/ /3.1.20查看cluster的日志#tail f /var/VRTSvcs/log/engine_A.log2006/10/22 19:44:25 VCS NOTICE V-16-1-10447 Group ysdb_sg is online on system jxysdb12006/10/22 19:44:25 VCS INFO V-16-6-15004 (jxysdb1) hatrigger:Failed to send trigger for nfs_restart; script doesnt exist2006/10/22 19:44:25 VCS INFO V-16-6-15004 (jxysdb1) hatrigger:Failed to send trigger for postonline; script doesnt exist2006/10/22 20:55:53 VCS INFO V-16-1-10077 Received new cluster membership2006/10/22 20:55:53 VCS NOTICE V-16-1-10080 System (jxysdb1) - Membership: 0x3, Jeopardy: 0x22006/10/22 20:55:53 VCS ERROR V-16-1-10087 System jxysdb2 (Node 1) is in Regardy Membership - Membership: 0x3, Jeopardy: 0x22006/10/22 20:55:53 VCS INFO V-16-6-15004 (jxysdb2) hatrigger:Failed to send trigger for injeopardy; script doesnt exist2006/10/22 20:56:46 VCS INFO V-16-1-10077 Received new cluster membership2006/10/22 20:56:46 VCS NOTICE V-16-1-10080 System (jxysdb1) - Membership: 0x3, Jeopardy: 0x02006/10/22 20:56:46 VCS NOTICE V-16-1-10086 System jxysdb2 (Node 1) is in Regular Membership - Membership: 0x33.1.21查看volume manager的日志#errpt -a3.1.22删除一个volume#vxedit g -rf rm #vxedit g datadg rf rm test3.1.23删除一个disk group#vxdg destroy 3.1.24如何扩大或缩小文件系统To resize a file system and a volume that contains it, the vxresize command can beused. The command should be run from the cvm master node. If resizing the volume orthe file system independent of each other, then the command should be run from the cvmmaster node or the cfs primary node respectively.To determine the primary node for a file system in a cluster, type:# fsclustadm v showprimary mount_pointTo determine if the current node is the master CVM node, type:# vxdctl -c modemode: enabled: cluster active - SLAVEmaster: star33如果cvm的master和cfs的primary node在一边，可以使用如下命令# vxresize -g -b -F vxfs 10g (此命令只能用于mount起来的文件系统)3.1.25如何禁止fencingYou may have to disable fencing in the following cases: The cluster has been upgraded to the latest SFCFS stack and the storage does notsupport SCSI-3 PGR feature. During installation fencing was turned on but later you want to turn it off.By default, the VxFEN driver operates with I/O fencing enabled. To disable this feat