山西烟草云计算平台与集成整合项目_AIXHANFS实施方案.doc

山西烟草AIX+HA+NFS实施方案当前版本：Version 2.0编写单位：IBM 目 录山西烟草1AIX+HA+NFS实施方案1目 录2图 表4文档历史51.方案目的62.方案架构原理73.环境现状83.1服务器硬件环境83.2系统软件环境84.实施前准备工作104.1实施前准备工作104.1.1操作系统备份104.1.2应用数据备份105.HACMP5.4安装配置115.1设计规划115.1.1IP及拓扑设计115.1.2资源组命名115.1.3etc/hosts文件设计115.2实施前提125.2.1创建Concurrent VG125.2.2网络配置145.2.3主机名及IP地址145.2.4建立主机信任关系145.2.5测试信任关系155.3安装HACMP155.3.1安装HACMP5.5155.3.2修改文件16usr/es/sbin/cluster/etc/rhosts16usr/sbin/cluster/netmon.cf165.4配置HACMP165.4.1配置网络拓朴175.4.2配置Cluster Name175.4.3配置Cluster节点175.4.4配置HACMP网络创建IP网络创建磁盘网络和心跳设备185.4.5添加网络到Cluster创建IP通信接口195.4.6配置HACMP资源创建资源组配置资源组属性215.4.7校验并同步HACMP配置225.4.8启动/停止HACMP启动HACMP（快速路径：#smitty clstart）停止HACMP（快速路径：#smitty clstop）235.5测试HACMP（Cluster Simulator）235.5.1Start Simulated Cluster Services235.5.2Fail An Adapter235.5.3Fail An Volume Group235.5.4Fail A Node235.5.5Fail An Application245.5.6Re-start A Node245.5.7Stop Simulated Cluster Services246.存在的风险及应对措施256.1操作系统无法启动256.2NFS无法启动256.3257.附录267.1相关文件267.1.1etc/hosts267.1.2.rhosts277.1.3usr/es/sbin/cluster/etc/rhosts277.1.4usr/sbin/cluster/netmon.cf27图 表图表 1-1 XXX（字号：五号；字体：黑体；居中；单倍行距，无缩进，无段前段后）6图表 1-2 XXXX7图表 2-1 XXXX8图表 2-2 XXXX9Error! Reference source not found.Error! Reference source not found.Error! Reference source not found.文档历史文档位置这是一个在线文档的打印件。硬拷贝只在打印当天有效。如果对本文档有疑问，请与作者联系。修订历史版本号修订日期修订内容修订人1.02012-03-27创建初始设计文档。陈永红2.02012-05-24格式调整黄代军审批客户项目经理姓名XX客户项目经理签名(Signature)日期(Date)子包项目经理姓名XX子包项目经理签名(Signature)日期(Date)IBM项目经理姓名国际商业机器中国有限公司项目经理签名(Signature)日期(Date)分发本文档已被分发到如下人员姓名职务1. 方案目的本方案是为建立中间件云平台WVE在AIX系统上的管理服务DmanagerHA的底层共享存储上建立NFS服务的高可用性，以保障DmanagerHA能够达到设计要求的高可用。使得Dmanager服务在系统间切换时，能够不受底层NFS文件系统影响，顺利切换，持续提供服务。2. 方案架构原理HACMP V5.3 and V5.4 support on AIX V6.1The IBM High Availability Cluster Multi-Processing (HACMP) for AIX software product supports AIX V6.1 with HACMP V5.3 and V5.4.Please refer to the following table for solution offering details and following restrictions. It is recommended to obtain the latest service updates for this support.Note: TL = Technology LevelSoftwareHACMP V5.3HACMP V5.4AIX V6.1HACMP V5.3 SP 6 (#IZ07791+)HACMP V5.4.1 (#IZ02620)3. 环境现状3.1 服务器硬件环境服务器硬件环境要求具备配置并正常运行HACMP的条件。（崔工提供）服务器硬件已具备配置并运行HACMP的条件。3.2 系统软件环境3.2.1 操作系统：# oslevel -s6100-06-05-11153.2.2 HACMP软件安装情况：根据IBM官方声明，AIX6.1支持的HACMP版本有：HACMP V5.3 SP6、HACMP V5.4.1。检查两台服务器上HACMP安装情况如下：# lslpp -l cluster* Fileset Level State Description -Path: /usr/lib/objrepos cluster.adt.es.client.include COMMITTED ES Client Include Files cluster.adt.es.client.samples.clinfo COMMITTED ES Client CLINFO Samples cluster.adt.es.client.samples.clstat COMMITTED ES Client Clstat Samples cluster.adt.es.client.samples.libcl COMMITTED ES Client LIBCL Samples cluster.adt.es.java.demo.monitor COMMITTED ES Web Based Monitor Demo cluster.doc.en_US.es.html COMMITTED HAES Web-based HTML Documentation - U.S. English cluster.doc.en_US.es.pdf COMMITTED HAES PDF Documentation - U.S.English cluster.es.cfs.rte COMMITTED ES Cluster File System Support cluster.es.client.lib COMMITTED ES Client Libraries cluster.es.client.rte COMMITTED ES Client Runtime cluster.es.client.utils COMMITTED ES Client Utilities cluster.es.client.wsm COMMITTED Web based Smit cluster.es.cspoc.cmds COMMITTED ES CSPOC Commands cluster.es.cspoc.dsh COMMITTED ES CSPOC dsh cluster.es.cspoc.rte COMMITTED ES CSPOC Runtime Commands cluster.es.nfs.rte COMMITTED ES NFS Support cluster.es.plugins.dhcp COMMITTED ES Plugins - dhcp cluster.es.plugins.dns COMMITTED ES Plugins - Name Server cluster.es.plugins.printserver COMMITTED ES Plugins - Print Server cluster.es.server.cfgast COMMITTED ES Two-Node Configuration Assistant cluster.es.server.diag COMMITTED ES Server Diags cluster.es.server.events COMMITTED ES Server Events cluster.es.server.rte COMMITTED ES Base Server Runtime cluster.es.server.simulator COMMITTED ES Cluster Simulator cluster.es.server.testtool COMMITTED ES Cluster Test Tool cluster.es.server.utils COMMITTED ES Server Utilities cluster.es.worksheets COMMITTED Online Planning Worksheets cluster.license COMMITTED HACMP Electronic License cluster.msg.en_US.cspoc COMMITTED HACMP CSPOC Messages - U.S.English cluster.msg.en_US.es.client COMMITTED ES Client Messages - U.S.English cluster.msg.en_US.es.server COMMITTED ES Recovery Driver Messages - U.S.EnglishPath: /etc/objrepos cluster.es.client.lib COMMITTED ES Client Libraries cluster.es.client.rte COMMITTED ES Client Runtime cluster.es.cspoc.rte COMMITTED ES CSPOC Runtime Commands cluster.es.server.diag COMMITTED ES Server Diags cluster.es.server.events COMMITTED ES Server Events cluster.es.server.rte COMMITTED ES Base Server Runtime cluster.es.server.simulator COMMITTED ES Cluster Simulator cluster.es.server.utils COMMITTED ES Server UtilitiesPath: /usr/share/lib/objrepos cluster.man.en_US.es.data COMMITTED ES Man Pages - U.S. English确认已安装HACMP，版本为V5.4.1。3.2.3 存储(NFS)：6(db1)7(dbhost2)Oslevel -s6100-06-05-11156100-06-05-1115pvHdisk2Hdisk2vgwvevglv/dev/wvelv(wvelv)Pathname of directory to export/ opt/IBM/WebSphere/WVEmountpoint/opt/IBM/WebSphere/WVEPathname of mount point/opt/IBM/WebSphere/WVEPathname of remote directory/opt/IBM/WebSphere/WVEHost where remote directory residesdb1HACMP软件安装情况：4. 实施前准备工作4.1 操作系统备份4.2 应用数据备份5. Db2数据库规划和安装实施5.1 数据库参数规划5.1.1 建立共享VG、LV、FS建立3个vg db2inesb，db2inbpm，db2inwps在相对于的vg上建立3个lv db2esb，db2bpm，db2wps建立相对应的FS：/app/db2inesb #db2inesb用户的主目录/app/db2inbpm #db2inbpm 用户的主目录/app/db2inwps #db2inst1的主目录 创建用户和组user userid group groupiddb2inesb 1001 db2iadm1999db2fenc1 1004 db2fadm1998db2inbpm 1002 db2iadm1999db2inst1 1003 db2iadm1999db2fenc2 1005 db2fadm1998启动脚本/home/start：cp /ap/db2inst1/sqllib/db2nodes.cfg.97 /home/db2inst1/sqllib/db2nodes.cfgsu - db2inst1 -c db2start关闭脚本/home/stop：su - db2inst1 -c db2stop forcesu - db2inst1 -c db2_kill开始安装 进入安装目录，运行./db2setupu 点击“Install a Product”u Install Newu Acceptu Typical：u Nextu 选择路径后Next最后完成点击finish创建实例：实例名：db2inesb，db2inbpm，db2inst1cd /opt/IBM/db2/V9.7/instance./db2icrt -a SERVER_ENCRYPT -p 51000 -u db2fenc1 db2inesb./db2icrt -a SERVER_ENCRYPT -p 52000 -u db2fenc1 db2inbpmcd /opt/IBM/db2/V9.5/instance./db2icrt -a SERVER_ENCRYPT -p 50000 -u db2fenc2 db2inst1创建数据库测试数据库名：samplesu - db2inst1db2startdb2 create database sampledb2 connect to sampledb2 list tables for system手动切换在主机上卸载共享VGumount /app/db2inesbumount /app/db2inbpmumount /app/db2inwpsvaryoffvg db2inesbvaryoffvg db2inbpmvaryoffvg db2inwps在备机上导入vg导入共享VGimportvg -y db2inesb hdisk4importvg -y db2inbpm hdisk3importvg -y db2inwps hdisk5varyonvg db2inesbvaryonvg db2inbpmvaryonvg db2inwpsmount /app/db2inesbmount /app/db2inbpmmount /app/db2inwps6. HACMP5.4安装配置6.1 设计规划6.1.1 IP及拓扑设计cluster namewveClsnode namepecsdb1ip labelpecsdb1pecsdb1_privjfdb1_tty0network typeethernetethernetrs232network attributePublicpublicserialnetwork namepecsdb1pecsdb1_privnet_rs232_01ip address36/dev/tty0adapter functionstandbyserialadapter nameen0en1cluster namewveClsnode namepecsdb1ip labelpecsdb1pecsdb1_privjfdb1_tty0network typeethernetethernetrs232network attributePublicpublicserialnetwork namepecsdb1pecsdb1_privnet_rs232_01ip address36/dev/tty0adapter functionstandbyserialadapter nameen0en16.1.2 资源组命名Res.Grp.NameNode Rel.Volume GroupPart.Node NamepecsresConcurrentoradatavg（61）pecsdb1 pecsdb26.1.3 /etc/hosts文件设计loopback localhost# loopback (lo0) name/address#pecsdb1#36 pecsdb39 pecsdb1_vip pecsdb1_priv#pecsdb2#38 pecsdb40 pecsdb2_vip pecsdb2_priv6.2 实施前提6.2.1 创建Concurrent VGpecsdb1做如下操作：# lspvhdisk0 00cb8f047f5d9ca2 rootvg activehdisk1 none None hdisk2 none None hdisk3 none None hdisk4 none None hdisk5 none None hdisk6 none None hdisk7 none None hdisk8 none None hdisk9 none None hdisk10 none None hdisk11 none None hdisk12 none None hdisk13 none None hdisk14 none None hdisk15 none None hdisk16 none None vpath4 00cb8f0495287ff6 appvg activevpath5 00cb8f04a9d97c8f oradatavg vpath6 none None vpath7 none None # smitty mkvg Add a Big Volume Group Type or select values in entry fields. Press Enter AFTER making all desired changes. Entry Fields VOLUME GROUP name oradatavg + Physical partition SIZE in megabytes 64* PHYSICAL VOLUME names vpath5,vpath7Force the creation of a volume group no + Activate volume group AUTOMATICALLY no + at system restart? Volume Group MAJOR NUMBER 61 Create VG Concurrent Capable? enhanced concurrent + 在另一台机器上要激活 PV正确的操作：chdev -l vpath5 -a pv=yeschdev -l vpath7 -a pv=yes激活卷组# varyonvg oradatavg 使用以下命令应该能够看到oradatavg # lsvg -ooradatavgappvgrootvg使用以下命令查看oradatavg的详细信息# lsvg oradatavgVOLUME GROUP: oradatavg VG IDENTIFIER: 00cb8f0400004c0000000132b327ee55VG STATE: active PP SIZE: 128 megabyte(s)VG PERMISSION: read/write TOTAL PPs: 1598 (204544 megabytes)MAX LVs: 512 FREE PPs: 1598 (204544 megabytes)LVs: 0 USED PPs: 0 (0 megabytes)OPEN LVs: 0 QUORUM: 2 (Enabled)TOTAL PVs: 2 VG DESCRIPTORS: 3STALE PVs: 0 STALE PPs: 0ACTIVE PVs: 2 AUTO ON: noConcurrent: Enhanced-Capable Auto-Concurrent: DisabledVG Mode: Non-Concurrent MAX PPs per VG: 130048 MAX PPs per PV: 1016 MAX PVs: 128LTG size (Dynamic): 1024 kilobyte(s) AUTO SYNC: noHOT SPARE: no BB POLICY: relocatable将卷组改为未激活状态# varyoffvg oradatavg# lsvg orootvgappvg两台机器不能同时varyonvg,只有使用HACMP才能同时激活pecsdb2做如下操作：# smitty vg Import a Volume GroupType or select values in entry fields. Press Enter AFTER making all desired changes. Entry Fields VOLUME GROUP name oradatavg * PHYSICAL VOLUME name vpath5 + Volume Group MAJOR NUMBER 61 +# 执行：chdev -l vpath5 -a pv=yes两台主机的VG major number要保持一致！6.2.2 网络配置6.2.3 主机名及IP地址主机内容备注pecsdb1pecsdb36en0pecsdb1-vip39pecsdb1_priven1pecsdb2pecsdb38en0pecsdb2-vip40pecsdb2_priven16.2.4 建立主机信任关系在两台主机的/目录下，建立.rhosts文件，注意改变文件权限：chmod 600 /.rhosts文件内容如下：pecsdb1pecsdb1_vippecsdb1_privpecsdb2pecsdb2_vippecsdb2_priv6.2.5 测试信任关系在两台主机上做以下操作，看结果是否正常：ping pecsdb1ping pecsdb1_privping pecsdb2ping pecsdb2_privrlogin pecsdb1rlogin pecsdb26.3 安装HACMP6.3.1 安装HACMP5.5除了cluster.hativoli,不安lslpp -l |grep clustercluster.es.cfs.rte COMMITTED ES Cluster File System Supportcluster.es.client.clcomd COMMITTED ES Cluster Communicationcluster.es.client.lib COMMITTED ES Client Librariescluster.es.client.rte COMMITTED ES Client Runtimecluster.es.client.utils COMMITTED ES Client Utilitiescluster.es.client.wsm COMMITTED Web based Smitcluster.es.cspoc.cmds COMMITTED ES CSPOC Commandscluster.es.cspoc.dsh COMMITTED ES CSPOC dshcluster.es.cspoc.rte COMMITTED ES CSPOC Runtime Commandscluster.es.server.cfgast COMMITTED ES Two-Node Configurationcluster.es.server.diag COMMITTED ES Server Diagscluster.es.server.events COMMITTED ES Server Eventscluster.es.server.rte COMMITTED ES Base Server Runtimecluster.es.server.simulatorcluster.es.server.testtoolcluster.es.server.utils COMMITTED ES Server Utilitiescluster.license COMMITTED HACMP Electronic Licensecluster.msg.en_US.cspoc COMMITTED HACMP CSPOC Messages - U.S.cluster.msg.en_US.es.clientcluster.msg.en_US.es.servercluster.es.client.clcomd COMMITTED ES Cluster Communicationcluster.es.client.lib COMMITTED ES Client Librariescluster.es.client.rte COMMITTED ES Client Runtimecluster.es.client.wsm COMMITTED Web based Smitcluster.es.cspoc.rte COMMITTED ES CSPOC Runtime Commandscluster.es.server.diag COMMITTED ES Server Diagscluster.es.server.events COMMITTED ES Server Eventscluster.es.server.rte COMMITTED ES Base Server Runtimecluster.es.server.simulatorcluster.es.server.utils COMMITTED ES Server Utilities6.3.2 修改文件/usr/es/sbin/cluster/etc/rhosts该文件的权限为/usr/es/sbin/cluster/etc/rhosts 600pecsdb1pecsdb1_vippecsdb1_privpecsdb2pecsdb2_vippecsdb2_priv/usr/sbin/cluster/netmon.cf/usr/sbin/cluster/netmon.cfpecsdb1pecsdb1_vippecsdb1_privpecsdb2pecsdb2_vippecsdb2_priv6.4 配置HACMP 以下操作只需在一个节点上操作即可。6.4.1 配置网络拓朴6.4.2 配置Cluster Name # smitty hacmp Extended Configuration Extended Topology Configuration Configure an HACMP Cluster Add/Change/Show an HACMP Cluster * Cluster Name pecsclu 6.4.3 配置Cluster节点 # smitty hacmp Extended Configuration Extended Topology Configuration Configure HACMP Nodes Add a Node to the HACMP Cluste *NodeName pecsdb1 Communication Path to Node pecsdb1 +以同样方法将commdb2添加到Cluster中 *NodeName pecsdb2 Communication Path to Node pecsdb2 +6.4.4 配置HACMP网络 创建IP网络 # smitty hacmp Extended Configuration Extended Topology Configuration Configure HACMP Networks Add a Network to the HACMP Cluster * NetworkName net_ether_01 * NetworkType ether * Netmask 28 +* Enable IP Address Takeover via IP Aliases yes + IP Address Offse