HACMP Oracle Rac安装设置手册

HACMPOracleRac安装设置手册版本v1.0二零一二年十月北京华胜天成科技股份文档控制更改记录版本创立/修改时间编制/修改者文件/修改内容审批人V1.02023.10审阅姓名职位发布姓名职位目录TOC\o\h\z\u目录3第1章HACMPv5.4安装配置41.1、IP规划41.2、安装HACMP软件41.2.1、操作系统版本41.2.2、安装依赖的软件包51.2.3、安装HACMPv5.461.3、HACMP配置规划81.3.1、配置IP和网络81.3.2、编辑/etc/hosts文件91.3.3、创立共享卷组和文件系统91.4、HACMPStandard配置101.4.1、添加cluster和节点111.4.2、创立并配置资源组111.4.3、同步HACMP的配置131.5、HACMP启动131.5.1、启动HACMP13第2章OracleRAC安装配置132.1、规划132.2、安装检查152.3、安装前配置172.4、安装CRS202.5、安装数据库412.6、创立数据库472.7、数据测试59HACMPv5.4安装配置IP规划NodeInterfaceNameTypeIPAddressRegisteredInibm03ibm03Public8Hostsfileibm03ibm03-vipVirtual10Hostsfileibm03ibm03-privPrivate8Hostsfileibm04ibm04Public9Hostsfileibm04ibm04-vipVirtual11Hostsfileibm04ibm04-privPrivate9Hostsfile安装HACMP软件完成HACMP安装配置规划后，可以开始安装HACMP软件，安装版本为HACMPv5.4。操作系统版本#oslevel-r5300-06安装依赖的软件包检查系统的软件安装包，请参照IBM红皮书中?HighAvailabilityClusterMulti-ProcessingforAIX5LInstallationGuide?。这里我们建议安装软件束APP-DEV和SERVER，其中包括的红皮书要求的大局部软件包。安装rsct软件包：rsct.basic〔可选〕安装其它软件包：bos.clvm〔使用并行资源组时需要，RAC下必须安装此包〕bos.data#smittyinstall_latestInstallSoftwareTypeorselectvaluesinentryfields.PressEnterAFTERmakingalldesiredchanges.[EntryFields]INPUTdevice/directoryforsoftware/dev/cd0SOFTWAREtoinstall[_all_latest]PREVIEWonly?(installoperationwillNOToccur)noCOMMITsoftwareupdates?yesSAVEreplacedfiles?noAUTOMATICALLYinstallrequisitesoftware?yesEXTENDfilesystemsifspaceneeded?yesOVERWRITEsameornewerversions?noVERIFYinstallandcheckfilesizes?noIncludecorrespondingLANGUAGEfilesets?yesDETAILEDoutput?noProcessmultiplevolumes?yesACCEPTnewlicenseagreements?yesPREVIEWnewLICENSEagreements?no在“SOFTWAREtoinstall〞一栏中选择要安装的软件包，“ACCEPTnewlicenseagreements〞中接受许可协议，即完成安装软件包。或参考以下脚本命令：#安装软件包/usr/lib/instl/sm_instinstallp_cmd-a-Q-d'/dev/cd0'-f'bos.clvmALL@@I:bos.clvm_all_fileagerm_all_filesets''-c''-N''-g''-X''-G''-Y'#安装HACMPv5.4除了haview，netwiew(Tivoli),的包以外，所有的HACMP的软件包都要安装，rm为可选安装包，对于根本包在安装光盘或Expansion光盘上可以查找到。#smittyinstall_latestInstallSoftwareTypeorselectvaluesinentryfields.PressEnterAFTERmakingalldesiredchanges.[EntryFields]INPUTdevice/directoryforsoftware/dev/cd0SOFTWAREtoinstall[_all_latest]PREVIEWonly?(installoperationwillNOToccur)noCOMMITsoftwareupdates?yesSAVEreplacedfiles?noAUTOMATICALLYinstallrequisitesoftware?yesEXTENDfilesystemsifspaceneeded?yesOVERWRITEsameornewerversions?noVERIFYinstallandcheckfilesizes?noIncludecorrespondingLANGUAGEfilesets?yesDETAILEDoutput?noProcessmultiplevolumes?yesACCEPTnewlicenseagreements?yesPREVIEWnewLICENSEagreements?no在“SOFTWAREtoinstall〞一栏中选择要安装的软件包，“ACCEPTnewlicenseagreements〞中接受许可协议，即完成安装软件包，安装过程大约在10~30分钟完成。或参考以下脚本命令：#安装HACMP/usr/lib/instl/sm_instinstallp_cmd-a-Q-d'/dev/cd0'-f'cluster.adt.esALL@@I:cluster.adt.es_all_filesets,cluster.doc.en_US.esALL@@I:cluster.doc.en_US.es_all_filesets,cluster.esALL@@I:cluster.es_all_filesets,cluster.es.cfsALL@@I:cluster.es.cfs_all_filesets,cluster.es.cspocALL@@I:cluster.es.cspoc_all_filesets,cluster.es.nfsALL@@I:cluster.es.nfs_all_filesets,cluster.es.pluginsALL@@I:cluster.es.plugins_all_filesets,cluster.es.worksheetsALL@@I:cluster.es.worksheets_all_filesets,cluster.licenseALL@@I:cluster.license_all_filesets,cluster.man.en_US.esALL@@I:cluster.man.en_US.es_all_filesets''-c''-N''-g''-X''-G''-Y'系统提示成功后，完成HACMP补丁升级，将HACMP补丁上传至系统目录下，我们假设上传目录为/tmp/hav54，本地目录为d:\hav54，上传命令如下：Windows：“开始〞->“运行〞，输入“cmd〞，根据以下提示完成补丁上传：MicrosoftWindowsXP[版本5.1.2600](C)版权所有1985-2001MicrosoftCorp.C:\DocumentsandSettings\Administrator>ftpremote-host-ip-address输入用户名和密码〔此处需要给用户设置密码，如果为空，AIX拒绝登录〕，成功登录后：ftp>cd/tmp/hav54ftp>lcdd:\hav54ftp>promptftp>binftp>mput*.bff上传完成后退出ftp程序ftp>quit升级HACMPv5.4补丁#smittyupdate_allUpdateInstalledSoftwaretoLatestLevel(UpdateAll)Typeorselectvaluesinentryfields.PressEnterAFTERmakingalldesiredchanges.[EntryFields]INPUTdevice/directoryforsoftware/tmp/hav54SOFTWAREtoupdate_update_allPREVIEWonly?(updateoperationwillNOToccur)noCOMMITsoftwareupdates?yesSAVEreplacedfiles?noAUTOMATICALLYinstallrequisitesoftware?yesEXTENDfilesystemsifspaceneeded?yesVERIFYinstallandcheckfilesizes?noDETAILEDoutput?noProcessmultiplevolumes?yesACCEPTnewlicenseagreements?yesPREVIEWnewLICENSEagreements?no回车确认以上命令，即可完成升级工作。注意：如果安装程序提示“没有找到相关的fileset〞等类似的提示，可在/tmp/hav54目录下执行命令“#inutoc.〞，重新生成软件包安装列表。重新启动AIX安装完成HACMP后，为防止各种问题，要求将AIX系统重新启动，至此我们完成了所有的安装工作，可以开始配置HACMP双机集群了。#shutdown–FrHACMP配置规划配置IP和网络AIX网卡IP地址配置：#smiittymktcpipAvailableNetworkInterfacesMovecursortodesireditemandpressEnter.en008-08StandardEthernetNetworkInterfaceen108-09StandardEthernetNetworkInterfaceen20A-08StandardEthernetNetworkInterfaceen30J-08StandardEthernetNetworkInterface选择想要配置IP地址的网卡，选择“en0〞，回车确认，进入下一步配置：MinimumConfiguration&StartupToDeleteexistingconfigurationdata,pleaseuseFurtherConfigurationmenusTypeorselectvaluesinentryfields.PressEnterAFTERmakingalldesiredchanges.[EntryFields]HOSTNAME[ora01]InternetADDRESS(dotteddecimal)[]NetworkMASK(dotteddecimal)[]NetworkINTERFACEen0NAMESERVERInternetADDRESS(dotteddecimal)[]DOMAINName[]DefaultGatewayAddress(dotteddecimalorsymbolicname)[]Cost[0]DoActiveDeadGatewayDetection?noYourCABLETypeN/ASTARTTCP/IPdaemonsNowno使用同样的方法配置完成网卡的IP地址。编辑/etc/hosts文件对于两个节点ibm03和ibm04要配置相同的/etc/hosts文件，使用vi配置hosts文件，建议首先对原有的hosts文件进行备份，配置如下：#cp/etc/hosts/etc/hosts.bak#vi/etc/hosts8ibm039ibm048ibm03-priv9ibm04-priv10ibm03-vip11ibm04-vip创立共享卷组和文件系统创立卷组#smitty_mkbvgAddaBigVolumeGroupTypeorselectvaluesinentryfields.PressEnterAFTERmakingalldesiredchanges.[EntryFields]VOLUMEGROUPname[vgdata]PhysicalpartitionSIZEinmegabytesPHYSICALVOLUMEnames[hdisk2]FORCEthecreationofvolumegroup?yesActivatevolumegroupAUTOMATICALLYnoatsystemrestart?VolumegroupMAJORNUMBER[100]CreateVGConcurrentCapable?no我们在建立HACMP共享卷组时，要注意的参数“ActivatevolumegroupAUTOMATICALLYatsystemrestart?〞该项，设置为“no〞，是使用HACMP的方式将卷组varyon，而不是系统启动时由操作系统varyon。MajorNUMBER是指该设备〔orvg〕的主设备号，此处设置为系统唯一即可，可通过命令查询可用的majornumber：#lvlstmajor43..59,62...或可通过ls–l/dev/命令查看当前所有设备的主设备号，第五列即是。创立逻辑卷和文件系统在ibm03节点上按照以下操作：#varyoffvgvgdata需将vgdatavaryoff下来，在ora02节点上才能正确的导入。在ibm04节点上按照一下操作：#lspvhdisk000c2ef1fa38df28arootvgactivehdisk100c2ef1f38b81facrootvgactivehdisk200c2ecef3846c20eNone查看hdisk2是否为物理卷，否那么需要使用命令将其属性更改：#chdev–lhdisk2–apv=yes导入卷组和文件系统的定义#smittyimportvgImportaVolumeGroupTypeorselectvaluesinentryfields.PressEnterAFTERmakingalldesiredchanges.[EntryFields]VOLUMEGROUPname[vgdata]*PHYSICALVOLUMEname[hdisk2]VolumegroupMAJORNUMBER[100]#varyonvgvgdata#varyoffvgvgdataHACMPStandard配置HACMP配置可采用Standard和Extended两种方式，其中Standard配置可以配置Cluter名称，节点，根本网络，应用效劳器，各种资源和资源组等根本的内容，如果要配置永久IP，串口心跳，对资源组进行详细的配置等工作那么需要在Extended中完成，可以说Standard菜单是Extended菜单的一个子集，可以在Extended菜单中完成HACMP的所有配置工作。下面我们以Standard菜单完成HACMP的根本配置工作：添加cluster和节点配置cluster资源创立并配置资源组同步HACMP的配置HACMP配置菜单如下：#smittyhacmpHACMPforAIXMovecursortodesireditemandpressEnter.InitializationandStandardConfigurationExtendedConfigurationSystemManagement(C-SPOC)ProblemDeterminationTools选择“InitializationandStandardConfiguration〞，如下菜单：InitializationandStandardConfigurationMovecursortodesireditemandpressEnter.ConfigurationAssistantsConfigureanHACMPClusterandNodesConfigureResourcestoMakeHighlyAvailableConfigureHACMPResourceGroupsVerifyandSynchronizeHACMPConfigurationDisplayHACMPConfigurationHACMPClusterTestTool添加cluster和节点#smittyhacmp->InitializationandStandardConfigurationConfigureanHACMPClusterandNodesConfigureNodestoanHACMPCluster(standard)Typeorselectvaluesinentryfields.PressEnterAFTERmakingalldesiredchanges.[EntryFields]*ClusterName[ora_cluster]NewNodes(viaselectedcommunicationpaths)[ibm03ibm04]CurrentlyConfiguredNode(s)创立并配置资源组#smittyhacmp->InitializationandStandardConfigurationConfigureHACMPResourceGroupsConfigureHACMPResourceGroupsMovecursortodesireditemandpressEnter.AddaResourceGroupChange/ShowaResourceGroupRemoveaResourceGroupChange/ShowResourcesforaResourceGroup(standard)AddaResourceGroupTypeorselectvaluesinentryfields.PressEnterAFTERmakingalldesiredchanges.[EntryFields]*ResourceGroupName[ora_rg]*ParticipatingNodes(DefaultNodePriority)[ibm03ibm04]StartupPolicyOnlineOnAllAvailabFalloverPolicyBringOffline(OnErr>FallbackPolicyNeverFallback添加资源至资源组中#smittyhacmp->InitializationandStandardConfigurationConfigureHACMPResourceGroupsConfigureHACMPResourceGroupsMovecursortodesireditemandpressEnter.AddaResourceGroupChange/ShowaResourceGroupRemoveaResourceGroupChange/ShowResourcesforaResourceGroup(standard)Change/ShowAllResourcesandAttributesforaResourceGroupTypeorselectvaluesinentryfields.PressEnterAFTERmakingalldesiredchanges.[EntryFields]ResourceGroupNameora_rgParticipatingNodes(DefaultNodePriority)ibm03ibm04StartupPolicyOnlineOnHomeNodeOnlyFalloverPolicyFalloverToNextPriorityNodeInTheListFallbackPolicyFallbackToHigherPriorityNodeInTheListConcurrentVolumeGroups[vgdata]+Useforcedvaryonofvolumegroups,ifnecessaryfalse+AutomaticallyImportVolumeGroupsfalse+ApplicationServers[]+TapeResources[]+至此，我们完成了资源和资源组的关联。同步HACMP的配置#smittyhacmp->InitializationandStandardConfigurationVerifyandSynchronizeHACMPConfiguration回车确认该操作，这样HACMP的配置已经同步到两个节点中了，也就是ibm03和ibm04拥有同样的配置了。HACMP启动启动HACMP在节点ibm03上执行smittyclstart，回车确认即可启动HACMP*Startnow,onsystemrestartorbothnow+StartClusterServicesonthesenodes[ibm03]+BROADCASTmessageatstartup?true+StartupClusterInformationDaemon?false+Reacquireresourcesafterforceddown?false+OracleRAC安装配置规划数据库配置规划ORACLE_BASE/dataCRS_HOME/data/product//crsORACLE_HOME/data/product//dbDB_NAMEorclORACLE_SIDorcl1，orcl2TNS_ADMIN$ORACLE_HOME/network/adminOracle管理帐户的口令sysSystemoracleoracle数据库存放位置裸设备是否使用归档方式运行数据库Yes备份方式说明RMAN/TSM裸设备规划设备名称数据库设施存储空间/dev/rorcl_ocrOcr信息存放500Mb/dev/rorcl_vdiskvdisk信息存放500Mb/dev/rorcl_spfilespfile信息存放100Mb/dev/rorcl_srvconfigsrvconfig信息存放100Mb/dev/rorcl_systemSystem表空间10G/dev/rorcl_sysauxSysaux表空间10G/dev/rorcl_temp1系统临时表空间20G/dev/rorcl_temp2系统临时表空间20G/dev/rorcl_undo1撤销表空间20G/dev/rorcl_undo2撤销表空间20G/dev/rorcl_undo3撤销表空间20G/dev/rorcl_undo4撤销表空间20G/dev/rorcl_usersUser表空间10G/dev/rorcl_toolsTools表空间10G/dev/rorcl_ctl1控制文件1G/dev/rorcl_ctl2控制文件1G/dev/rorcl_ctl3控制文件1G/dev/rorcl_log1a日志文件500Mb/dev/rorcl_log1b日志文件500Mb/dev/rorcl_log2a日志文件500Mb/dev/rorcl_log2b日志文件500Mb/dev/rorcl_log3a日志文件500Mb/dev/rorcl_log3b日志文件500Mb/dev/rorcl_log4a日志文件500Mb/dev/rorcl_log4b日志文件500Mb/dev/rorcl_data1用户数据表空间20G/dev/rorcl_data2用户数据表空间20G/dev/rorcl_data3用户数据表空间20G/dev/rorcl_data4用户数据表空间20G安装检查操作系统版本：#oslevel-r5300-06物理内存：#lsattr-Elmem0goodsize7744AmountofusablephysicalmemoryinMbytesFalsesize7744TotalamountofphysicalmemoryinMbytesFalse交换区：#lsps-aPageSpacePhysicalVolumeVolumeGroupSize%UsedActiveAutoTypehd6hdisk0rootvg512MB3yesyeslv系统空间：#df-gFilesystemGBblocksFree%UsedIused%IusedMountedon/dev/hd40.500.4413%47885%//dev/hd25.003.1038%412406%/usr/dev/hd9var1.000.956%7471%/var/dev/hd31.000.6536%10821%/tmp/dev/fwdump0.380.371%41%/var/adm/ras/platform/dev/hd10.500.501%711%/home/proc-----/proc/dev/hd10opt2.001.867%34611%/opt/dev/fslv0015.004.4271%292433%/data/dev/lv000.120.124%181%/var/adm/csd文件集：#lslpp-lbos.adt.basebos.adt.libbos.adt.libmbos.perf.libperfstatbos.perf.perfstatctoolsrsct.basic.rtersctpat.clients.rtexlC.aix50.rtexlC.rteFilesetLevelPath:/usr/lib/objreposbos.adt.base.0COMMITTEDBaseApplicationDevelopmentToolkitbos.adt.lib.0COMMITTEDBaseApplicationDevelopmentLibrariesbos.adt.libm.0COMMITTEDBaseApplicationDevelopmentMathLibrarybos.perf.libperfstat.0COMMITTEDPerformanceStatisticsLibraryInterfacebos.perf.perfstat.1COMMITTEDPerformanceStatisticsInterfacectools.0COMMITTEDProcFilesystemToolsrsct.basic.rte.0COMMITTEDRSCTBasicFunctionrsctpat.clients.rte.0COMMITTEDRSCTEventManagementClientFunctionxlC.aix50.rte.1COMMITTEDXLC/C++RuntimeforAIX5.2xlC.rte.1COMMITTEDXLC/C++RuntimePath:/etc/objreposbos.perf.libperfstat.0COMMITTEDPerformanceStatisticsLibraryInterfacebos.perf.perfstat.1COMMITTEDPerformanceStatisticsInterfacersct.basic.rte.0COMMITTEDRSCTBasicFunction安装前配置为确保oraclerac安装成功，系统管理员root用户需创立相应的系统用户、系统组，以便安装管理oracle软件。Root用户还要修改相关系统参数，打破默认值对实际应用的限制。调试rac节点之间的高级功能：rcp/rsh/rlogin，以确定两节点之间可以通信正常，可以在rac环境下进行相关同步操作和心跳监控。创立dba组，id为500#smittymkgroup创立oracle用户，id为601#smittymkuser修改oracle用户密码#passwdoracle在oracle用户下创立.rhosts文件，内容为“+〞。#su–oracle$vi.rhosts+修改oracle用户限制#smittychuserSoftFILEsize[-1]SoftCPUtime[-1]SoftDATAsegment[-1]SoftSTACKsize[-1]SoftCOREfilesize[-1]修改系统配置参数，增加系统进程数上限#smittychgsysMaximumnumberofPROCESSESallowedperuser[2048]修改网络参数no-p-oudp_sendspace=65536no-p-oudp_recvspace=65536no-p-otcp_sendspace=65536no-p-otcp_recvspace=65536no-p-orfc1323=1no-p-osb_max=1310720no-r-oipqmaxlen=512创立裸设备mklv-yrac_system_raw_500m-TO-wn-sn-rnvgdata8依次创立以下裸设备：/dev/rorcl_ocr/dev/rorcl_vdisk/dev/rorcl_spfile/dev/rorcl_srvconfig/dev/rorcl_system/dev/rorcl_sysaux/dev/rorcl_temp1/dev/rorcl_temp2/dev/rorcl_undo1/dev/rorcl_undo2/dev/rorcl_undo3/dev/rorcl_undo4/dev/rorcl_users/dev/rorcl_tools/dev/rorcl_ctl1/dev/rorcl_ctl2/dev/rorcl_ctl3/dev/rorcl_log1a/dev/rorcl_log1b/dev/rorcl_log2a/dev/rorcl_log2b/dev/rorcl_log3a/dev/rorcl_log3b/dev/rorcl_log4a/dev/rorcl_log4b/dev/rorcl_data1/dev/rorcl_data2/dev/rorcl_data3/dev/rorcl_data4更改裸设备权限#cd/dev#chownoracle:dba*orcl*修改oracle用户环境变量#vi/home/oracle/.profileumask022exportORACLE_BASE=/dataexportORACLE_HOME=$ORACLE_BASE/product//dbexportORA_CRS_HOME=$ORACLE_BASE/product//crsexportLD_LIBRARY_PATH=$ORACLE_HOME/lib:$ORACLE_HOME/lib32exportORACLE_SID=orclexportPATH=$ORACLE_HOME/bin:$ORA_CRS_HOME/bin:$PATH安装CRS在ibm03节点上进入CRS安装盘，执行runInstaller启动CRS的安装；按提示在每个安装节点上以root用户执行rootpre.sh，该脚本在CRS安装盘上的rootpre目录下；#./rootpre.sh./rootpre.shoutputwillbeloggedin/tmp/rootpre.out_06-04-13.12:33:04Savingtheoriginalfilesin/etc/ora_save_06-04-13.12:33:04Copyingnewkernelextensionto/etcLoadingthekernelextensionfrom/etcOracleKernelExtensionLoaderforAIXCopyright(c)1998,1999OracleCorporationSuccessfullyloaded/etc/pw-syscall.64bit_kernelwithkmid:0x441e300Successfullyconfigured/etc/pw-syscall.64bit_kernelwithkmid:0x441e300Thekernelextensionwassuccessfulyloaded.ConfiguringAsynchronousI/OAsynchronousI/OisalreadydefinedConfiguringPOSIXAsynchronousI/OCheckingifgroupservicesshouldbeconfiguredGroup"hagsuser"doesnotexist.Creatingrequiredgroupforgroupservices:hagsuserPleaseaddyourOracleuseridtothegroup:hagsuserConfiguringHACMPgroupservicessocketforpossibleusebyOracle.Thegrouporpermissionsofthegroupservicessockethavechanged.PleasestopandrestartHACMPbeforetryingtouseOracle.在两个节点都执行root.sh脚本：ibm03#./root.shWARNING:directory'/oracle/product/'isnotownedbyrootWARNING:directory'/oracle/product'isnotownedbyrootCheckingtoseeifOracleCRSstackisalreadyconfiguredCheckingtoseeifany9iGSDisupSettingthepermissionsonOCRbackupdirectorySettingupNSdirectoriesOracleClusterRegistryconfigurationupgradedsuccessfullyWARNING:directory'/oracle/product/'isnotownedbyrootWARNING:directory'/oracle/product'isnotownedbyrootSuccessfullyaccumulatednecessaryOCRkeys.Usingports:CSS=49895CRS=49896EVMC=49898andEVMR=49897.node<nodenumber>:<nodename><privateinterconnectname><hostname>node1:ibm03ibm03-privibm03node2:ibm04ibm04-privibm04CreatingOCRkeysforuser'root',privgrp'system'..Operationsuccessful.Nowformattingvotingdevice:/dev/rora_voteFormatof1votingdevicescomplete.Startupwillbequeuedtoinitwithin30seconds.Addingdaemonsto