积成电子RAC集群.doc

方正世纪公司 MC/SG安装报告积成电子Oracle RAC 集群版权所有者 赵帅本篇仅为项目记录，供大家学习之用1安装准备及注意事项1 确认电是否已就绪。2 无终端，需自带笔记本。最好自备一根两端均为母头的九孔串口线使笔记本与服务器直接相连。使用Reflection X 来操作，否则只能用Web Console。3 确认是否有网络设备(Switch)及网线。MC/SG需4根直连(Server-Switch)，一根互连(Server-Server)，Web Console（lan Console ）需2根。要有备用网线或可当场做线的条件。4 配置为两台RP5405 主机，但软件Mirror Disk及MC/SG 各买了两个，工厂分开预装。所以需安装Mirror Disk、MC/SG各一。B2491BA Mirror DiskB5161FA MC/SG5 准备 MC/SG Patch:GOLDQPK11i_11.11.depot2 用户及主机配置信息 1主机信息Gateway为*.*.*.254No.Host nameHost IPPackage NameHeart Beat IPnetmaskLan Console IP1SMSDB15SMS2SMSDB26SMS2 MC/SG 11.14 patch bundles执行以下命令，给主机添加相应补丁：swinstall s /tmp/GOLDQPK11i_11.11.depot按提示安装软件B5161FA 即可安装补丁结束时，系统要求重新启动主机，重启主机即可。3系统安装步骤1 配 hostname、IP及Gateway。（见上表）2 扩 /tmp 到500M。3 安装Mirror Disk，MC/SG各一。4 系统部分：Ignitemake_recovery -C -A5 打系统Patch：(GOLDQPK11i_11.11.depot)6 打MC Patch：同上7 配置ROOT盘Mirror，方法如下：pvcreate -B /dev/rdsk/cxtxdxvgextend /dev/vg00 /dev/dsk/cxtxdxmkboot /dev/rdsk/cxtxdxmkboot -a hpux -lq /dev/rdsk/cxtxdxlvextend -m 1 /dev/vg00/lvol1 /dev/dsk/cxtxdx.lvlnboot -Rlvlnboot -v8 修改BOOT ALT路径到Mirror盘。9 配 Lan Console IP地址分别为，。10 连网线，配心跳IP地址。hosta：hostb：注意：心跳地址信息不要出现在hosts文件中。4 VG、LV划分1 VG、LV信息由用户提供，划分由用户工程师完成。2 以下为一个例子，可作参考。LV划分：3 注意：确认vglock盘已划分，一般空间为100M。4主机参数调整，由用户工程师完成。5 安装 ORACLE 数据库1 由Oracle工程师完成。2 数据库用户名与组的建立由Oracle工程师完成。3 oracle的start,stop,monitor脚本由 oracle工程师提供。以下为配置实例，可作参考。Db_startsu - oracle -c /oracle/oracle/app/oracle/product/.0/bin/gsdctl startsu - oracle -c /oracle/oracle/app/oracle/product/.0/bin/gsdctl statsu - oracle -c /oracle/oracle/app/oracle/product/.0/bin/lsnrctl startsu - oracle -c /oracle/oracle/app/oracle/product/.0/bin/lsnrctl statsu - oracle -c /oracle/oracle/app/oracle/product/.0/bin/sqlplus / as sysdba EOFstartup;exit;EOFDb_shutsu - oracle -c /oracle/oracle/app/oracle/product/.0/bin/lsnrctl stopsu - oracle -c /oracle/oracle/app/oracle/product/.0/bin/lsnrctl statsu - oracle -c /oracle/oracle/app/oracle/product/.0/bin/sqlplus / as sysdba EOFshutdown immediate;exit;EOFsu - oracle -c /oracle/oracle/app/oracle/product/.0/bin/gsdctl stopsu - oracle -c /oracle/oracle/app/oracle/product/.0/bin/gsdctl stat6 MC/ServiceGuard 配置步骤1心跳线确认连好，且心跳IP最好和业务IP不同网段。2询问第三块网卡是否要给集群做备用，如果是，需要连上网线但不设置IP。3安装RAC.DEPOT，如果是FTP，用bin格式传。swinstall s /tmp/RAC.DEPOT4 swlist 查看集群软件版本，通常情况下V3操作系统是11.18/11.195 #vi /.profile 最后一行加：export PS1=hostname#$PWDvi /.rhosts-两个主机都作smsdb1rootsmsdb2rootsmsdb1oracle#oracle 用户smsdb2oracle#oracle 用户vi /etc/hosts-两个主机都作（如果有心跳IP，要删掉）CRCT1: 0CRCT2: 1CRCT: 5写在最低部cp /.rhosts /etc/hosts.equivcp /etc/hosts.equiv /etc/cmcluster/cmclnodelistrcp /etc/hosts.equiv CRCT1:/etc/hosts.equivrcp /etc/hosts.equiv CRCT1:/etc/cmcluster/cmclnodelist1 在HOST A主机上 mkdir /dev/vglockmkdir /dev/vgdatamknod /dev/vglock/group c 64 0x010000mknod /dev/vgdata/group c 64 0x020000ioscan m dsf (ioscan m lun) （diskinfo 裸设备号）（strings /etc/lvmtab）pvcreate f /dev/rdsk/c4t1d2pvcreate f /dev/rdsk/c4t1d3vgcreate s 64vgcreate vglock /dev/dsk/c4t1d2vgcreate vgdata /dev/dsk/c4t1d3lvcreate -L 5000M -n system01.dbf vgdata（不建FS）2 在主配置机(hosta)上输出(vgexport)卷组#vgchange -a n /dev/vgdata#vgchange -a n /dev/vglock#vgexport -v -p -s -m /tmp/vgdata1.map /dev/vgdata#vgexport -v -p -s -m /tmp/vglock.map /dev/vglock#vgexport -v -p -m /tmp/*.map /dev/vg*-第三方盘柜（HDS）#rcp /tmp/vgdata.map smsdb2:/tmp#rcp /tmp/vglock.map smsdb2:/tmp3 在非配置机(hostb)上输入(vgimport)卷组#mkdir /dev/vgdata#mkdir /dev/vglock#mknod /dev/vglock/group c 64 0x010000(1,2,3要与hosta一致)#mknod /dev/vgdata1/group c 64 0x020000#vgimport -v -s -m /tmp/vgdata.map /dev/vgdata#vgimport -v -s -m /tmp/vglock.map /dev/vglock#vgimport -v -m /tmp/*.map /dev/vg* /dev/dsk/c6t0d1-该机器的统一虚盘的设备号#vgchange -a y /dev/vgdata#vgchange -a y /dev/vglock#vgcfgbackup /dev/vgdata#vgcfgbackup /dev/vglock# vgchange -a n /dev/vgdata#vgchange -a n /dev/vglock#chmod 777 /dev/vgdata# #chmod 660 /dev/vgdata/r*#chown oracle:oracle /dev/vgdata/r*(oracle用户名、组依实际建立情况而定)2、 将VG设置为非自动激活-两个主机都作#vi /etc/lvmrc其中AUTO_VG_ACTIVATE=1 修改为0确定CRCT1和CRCT2双机上的VG01、VG02、VG03、VGLOCK处于非活动状态#vgdisplaystrings /etc/lvmtab 查看LV是否正确4 在HOSTA 创立集群配置文件#cmquerycl -n hosta -n hostb -v -C /etc/cmcluster/cluster.ascii5 编辑集群配置文件#vi /etc/cmcluster/cluster.ascii# *# * HIGH AVAILABILITY CLUSTER CONFIGURATION FILE *# * For complete details about cluster parameters and how to *# * set them, consult the ServiceGuard manual. *# *# Enter a name for this cluster. This name will be used to identify the# cluster when viewing or manipulating it.CLUSTER_NAMEcluster1# Cluster Lock Parameters# The cluster lock is used as a tie-breaker for situations# in which a running cluster fails, and then two equal-sized# sub-clusters are both trying to form a new cluster. The# cluster lock may be configured using either a lock disk# or a quorum server. # You can use either the quorum server or the lock disk as# a cluster lock but not both in the same cluster.# Consider the following when configuring a cluster.# For a two-node cluster, you must use a cluster lock. For# a cluster of three or four nodes, a cluster lock is strongly# recommended. For a cluster of more than four nodes, a# cluster lock is recommended. If you decide to configure# a lock for a cluster of more than four nodes, it must be# a quorum server.# Lock Disk Parameters. Use the FIRST_CLUSTER_LOCK_VG and# FIRST_CLUSTER_LOCK_PV parameters to define a lock disk.# The FIRST_CLUSTER_LOCK_VG is the LVM volume group that# holds the cluster lock. This volume group should not be# used by any other cluster as a cluster lock device. # Quorum Server Parameters. Use the QS_HOST, QS_POLLING_INTERVAL,# and QS_TIMEOUT_EXTENSION parameters to define a quorum server.# The QS_HOST is the host name or IP address of the system# that is running the quorum server process. The# QS_POLLING_INTERVAL (microseconds) is the interval at which# ServiceGuard checks to make sure the quorum server is running.# The optional QS_TIMEOUT_EXTENSION (microseconds) is used to increase# the time interval after which the quorum server is marked DOWN.# The default quorum server timeout is calculated from the# ServiceGuard cluster parameters, including NODE_TIMEOUT and# HEARTBEAT_INTERVAL. If you are experiencing quorum server# timeouts, you can adjust these parameters, or you can include# the QS_TIMEOUT_EXTENSION parameter.# For example, to configure a quorum server running on node# qshost with 120 seconds for the QS_POLLING_INTERVAL and to# add 2 seconds to the system assigned value for the quorum server# timeout, enter:# QS_HOST qshost# QS_POLLING_INTERVAL 120000000# QS_TIMEOUT_EXTENSION 2000000# newFIRST_CLUSTER_LOCK_VG/dev/vglock# Definition of nodes in the cluster.# Repeat node definitions as necessary for additional nodes.NODE_NAMEsmsdb1 NETWORK_INTERFACElan0 HEARTBEAT_IP NETWORK_INTERFACElan1 HEARTBEAT_IP5 NETWORK_INTERFACElan2# new FIRST_CLUSTER_LOCK_PV/dev/dsk/c4t0d1# List of serial device file names# For example:# SERIAL_DEVICE_FILE/dev/tty0p0# Warning: There are no standby network interfaces for lan0.# Possible standby Network Interfaces for lan1: lan2.NODE_NAMEsmsdb2 NETWORK_INTERFACElan0 HEARTBEAT_IP NETWORK_INTERFACElan1 HEARTBEAT_IP6 NETWORK_INTERFACElan2 FIRST_CLUSTER_LOCK_PV/dev/dsk/c4t0d1# List of serial device file names# For example:# SERIAL_DEVICE_FILE/dev/tty0p0# Warning: There are no standby network interfaces for lan0.# Possible standby Network Interfaces for lan1: lan2.# Cluster Timing Parameters (microseconds).# The NODE_TIMEOUT parameter defaults to 2000000 (2 seconds).# This default setting yields the fastest cluster reformations.# However, the use of the default value increases the potential# for spurious reformations due to momentary system hangs or# network load spikes.# For a significant portion of installations, a setting of# 5000000 to 8000000 (5 to 8 seconds) is more appropriate.# The maximum value recommended for NODE_TIMEOUT is 30000000# (30 seconds).HEARTBEAT_INTERVAL1000000NODE_TIMEOUT20000000# Configuration/Reconfiguration Timing Parameters (microseconds).AUTO_START_TIMEOUT600000000NETWORK_POLLING_INTERVAL2000000# Package Configuration Parameters.# Enter the maximum number of packages which will be configured in the cluster.# You can not add packages beyond this limit.# This parameter is required.MAX_CONFIGURED_PACKAGES5# List of cluster aware LVM Volume Groups. These volume groups will# be used by package applications via the vgchange -a e command.# Neither CVM or VxVM Disk Groups should be used here.# For example: # VOLUME_GROUP/dev/vgdatabase# VOLUME_GROUP/dev/vg02VOLUME_GROUP/dev/vglock# List of OPS Volume Groups.# Formerly known as DLM Volume Groups, these volume groups# will be used by OPS cluster applications via# the vgchange -a s command. (Note: the name DLM_VOLUME_GROUP# is also still supported for compatibility with earlier versions.)# For example: # OPS_VOLUME_GROUP/dev/vgdatabase# OPS_VOLUME_GROUP/dev/vg02# new# OPS_VOLUME_GROUP/dev/vglockOPS_VOLUME_GROUP/dev/vglockOPS_VOLUME_GROUP/dev/vgdata6 HOSTA检查集群配置文件#cmcheckconf -v -C /etc/cmcluster/cluster.ascii HOSTA做：#vgchange a y /dev/vglock7 分发到集群中所有结点#cmapplyconf -v -C /etc/cmcluster/cluster.ascii执行完本步骤后，集群中所有结点都已生成 /etc/cmcluster/cmclconfig二进制的配置文件HOSTA做：#vgchange a n /dev/vglock8 在两台主机上创立包目录#mkdir /etc/cmcluster/rac 9 启动Cluster#cmruncl vVgchange c y /dev/vglockVgchange c y /dev/vgdata*vgchange -S y c y /dev/vgdata*vgchange -S y c y /dev/vglock*如要单独操作卷组：Vgchange c n /dev/vglockVgchange c n /dev/vgdata*vgchange -a s /dev/vgdata*注：*为装RAC 时用10 创立包配置文件#cmmakepkg -p /etc/cmcluster/rac.ascii 13编辑包配置文件（以库尔勒为例）#vi /etc/cmcluster/rac.ascii# *# * HIGH AVAILABILITY PACKAGE CONFIGURATION FILE (template) *# *# * Note: This file MUST be edited before it can be used. *# * For complete details about package parameters and how to set them, *# * consult the MC/ServiceGuard ServiceGuard OPS Edition manuals *# *# Enter a name for this package. This name will be used to identify the# package when viewing or manipulating it. It must be different from# the other configured package names.PACKAGE_NAMErac# Enter the package type for this package. PACKAGE_TYPE indicates# whether this package is to run as a FAILOVER or SYSTEM_MULTI_NODE# package.# FAILOVER package runs on one node at a time and if a failure# occurs it can switch to an alternate node.# SYSTEM_MULTI_NODE# package runs on multiple nodes at the same time.# It can not be started and halted on individual nodes.# Both NODE_FAIL_FAST_ENABLED and AUTO_RUN must be set# to YES for this type of package. All SERVICES must# have SERVICE_FAIL_FAST_ENABLED set to YES.# NOTE: Packages which have a PACKAGE_TYPE of SYSTEM_MULTI_NODE are# not failover packages and should only be used for applications# provided by Hewlett-Packard.# Since SYSTEM_MULTI_NODE packages run on multiple nodes at# one time, following parameters are ignored:# FAILOVER_POLICY# FAILBACK_POLICY# Since an IP address can not be assigned to more than node at a# time, relocatable IP addresses can not be assigned in the# package control script for multiple node packages. If# volume groups are assigned to multiple node packages they must# activated in a shared mode and data integrity is left to the# application. Shared access requires a shared volume manager.# Examples : PACKAGE_TYPE FAILOVER (default)# PACKAGE_TYPE SYSTEM_MULTI_NODE#PACKAGE_TYPESYSTEM_MULTI_NODE# Enter the failover policy for this package. This policy will be used# to select an adoptive node whenever the package needs to be started.# The default policy unless otherwise specified is CONFIGURED_NODE.# This policy will select nodes in priority order from the list of# NODE_NAME entries specified below.# The alternative policy is MIN_PACKAGE_NODE. This policy will select# the node, from the list of NODE_NAME entries below, which is# running the least number of packages at the time this package needs# to start.FAILOVER_POLICYCONFIGURED_NODE# Enter the failback policy for this package. This policy will be used# to determine what action to take when a package is not running on# its primary node and its primary node is capable of running the# package. The default policy unless otherwise specified is MANUAL.# The MANUAL policy means no attempt will be made to move the package# back to its primary node when it is running on an adoptive node.# The alternative policy is AUTOMATIC. This policy will attempt to# move the package back to its primary node whenever the primary node# is capable of running the package.FAILBACK_POLICYMANUAL# Enter the names of the nodes configured for this package. Repeat# this line as necessary for additional adoptive nodes.# NOTE: The order is relevant.# Put the second Adoptive Node after the first one.# Example : NODE_NAME original_node# NODE_NAME adoptive_node# If all nodes in cluster is to be specified and order is not# important, NODE_NAME * may be specified.# Example : NODE_NAME *NODE_NAMEsmsdb1NODE_NAMEsmsdb2# Enter the value for AUTO_RUN. Possible values are YES and NO.# The default for AUTO_RUN is YES. When the cluster is started the# package will be automatically started. In the event of a failure the# package will be started on an adoptive node. Adjust as necessary.# AUTO_RUN replaces obsolete PKG_SWITCHING_ENABLED.AUTO_RUNYES# Enter the value for LOCAL_LAN_FAILOVER_ALLOWED.# Possible values are YES and NO.# The default for LOCAL_LAN_FAILOVER_ALLOWED is YES. In the event of a# failure, this permits the cluster software to switch LANs locally # (transfer to a standby LAN card). Adjust as necessary.# LOCAL_LAN_FAILOVER_ALLOWED replaces obsolete NET_SWITCHING_ENABLED.LOCAL_LAN_FAILOVER_ALLOWEDYES# Enter the value for NODE_FAIL_FAST_ENABLED.# Possible values are YES and NO.# The default for NODE_FAIL_FAST_ENABLED is NO. If set to YES,# in the event of a failure, the cluster software will halt the node # on which the package is running. All SYSTEM_MULTI_NODE packages must have# NODE_FAIL_FAST_ENABLED set to YES. Adjust as necessary.NODE_FAIL_FAST_ENABLEDYES# Enter the complete path for the run and halt scripts. In most cases# the run script and halt script specified here will be the same script,# the package control script generated by the cmmakepkg command. This# control script handles the run(ning) and halt(ing) of the package.# Enter the timeout, specified in seconds, for the run and halt scripts.# If the script has not completed by the specified timeout value,# it will be terminated. The default for each script timeout is# NO_TIMEOUT. Adjust the timeouts as necessary to permit full # execution of each script.# Note: The HALT_SCRIPT_TIMEOUT should be greater than the sum of# all SERVICE_HALT_TIMEOUT specified for all services.RUN_SCRIPT/etc/cmcluster/rac/control.shRUN_SCRIPT_TIMEOUTNO_TIMEOUTHALT_SCRIPT/etc/cmcluster/rac/control.shHALT_SCRIPT_TIMEOUTNO_TIMEOUT# Enter the names of the storage groups configured for this package.# Repeat this line as necessary for additional storage groups.# Storage groups are only used with CVM disk groups. Neither# VxVM disk groups or LVM volume groups should be listed here.# By specifying a CVM disk group with the STORAGE_GROUP keyword# this package will not run until the VxVM-CVM-pkg package is# running and thus the CVM shared disk groups are ready for# activation.# NOTE: Should only be used by applications provided by# Hewlett-Packard.# Example : STORAGE_GROUP dg01# STORAGE_GROUP dg02# STORAGE_GROUP dg03# STORAGE_GROUP dg04# Enter the SERVICE_NAME, the SERVICE_FAIL_FAST_ENABLED and the# SERVICE_HALT_TIMEOUT values for this package. Repeat these # three lines as necessary for additional service n