数据库RAC添加新节点的操作步骤(门户数据库PDSDB).docx

数据库RAC添加新节点的操作步骤安装pdsrac的新节点1.在两个节点分别安装vncwget /dist/vnc-4.0-x86_linux.tar.gz tar -xzvf vnc-4.0-x86_linux.tar.gz ./vncinstall /usr/local/bin mkdir -p /usr/local/vnc/classes cp java/* /usr/local/vnc/classes vncserver2.添加网卡参数说明：DEVICE=物理设备名 IPADDR=IP地址 NETMASK=掩码值 NETWORK=网络地址 BROADCAST=广播地址 GATEWAY=网关地址 ONBOOT=yes|no（引导时是否激活设备） USERCTL=yes|no（非root用户是否可以控制该设备） BOOTPROTO=none|static|bootp|dhcp（引导时不使用协议|静态分配|BOOTP协议|DHCP协议） HWADDR = 你的MAC地址 在/etc/sysconfig/network-scripts/下创建虚拟网卡bond0的配置文件ifcfg-bond0cd /etc/sysconfig/network-scripts/vi ifcfg-bond1DEVICE=bond1BOOTPROTO=noneBORADCAST=55IPADDR=2NETMASK=NETWORK=ONBOOT=yesTYPE=Ethernet分别修改原来网卡配置文件的信息，删除其中的IP 地址、子网掩码等信息vi ifcfg-eth0DEVICE=eth0ONBOOT=yesBOOTPROTO=noneMASTER=bond0SLAVE=yesUSERCTL=yesvi ifcfg-eth1DEVICE=eth1ONBOOT=yesBOOTPROTO=noneMASTER=bond0SLAVE=yesUSERCTL=yesvi ifcfg-eth2# Broadcom Corporation NetXtreme II BCM5708 Gigabit EthernetDEVICE=eth2HWADDR=00:24:E8:56:4E:88ONBOOT=yesBOOTPROTO=noneMASTER=bond1SLAVE=yesUSERCTL=yesHOTPLUG=novi ifcfg-eth3# Broadcom Corporation NetXtreme II BCM5708 Gigabit EthernetDEVICE=eth3HWADDR=00:24:E8:56:4E:8AONBOOT=yesMASTER=bond1USERCTL=yesSLAVE=yesBOOTPROTO=noneHOTPLUG=no3.配置 驱动模块参数：/etc/modprobe.confalias bond0 bondingoptions bond0 miimon=100 mode=1alias eth0 bnx2xalias eth1 bnx2xalias bond1 bondingoptions bond1 miimon=100 mode=1alias eth2 bnx2xalias eth3 bnx2xalias scsi_hostadapter ccissalias usb-controller uhci-hcdalias usb-controller1 ehci-hcdalias scsi_hostadapter1 qla2xxxoptions hangcheck-timer hangcheck_tick=30 hangcheck_margin=180说明：（1）、miimon 是链路监测的时间间隔单位是毫秒，miimon=100的意思就是，每100毫秒检测网卡和交换机之间是否连通，如不通则使用另外的链路。（2)、mode=0 表示负载均衡方式，两块网卡都工作，需要交换机作支持 mode=1 表示冗余方式，网卡只有一个工作，一个出问题启用另外的 mode=6 表示负载均衡方式，两块网卡都工作，不需要交换机作支持 设置开机绑定 rootBJ-FT-1F-120-12 etc# cat /etc/rc.local #!/bin/sh# This script will be executed *after* all the other init scripts.# You can put your own initialization stuff in here if you dont# want to do the full Sys V style init stuff.touch /var/lock/subsys/local/usr/sbin/ntpdate &/sbin/modprobe hangcheck_timer/usr/local/sbin/oraraw.shservice network restartifenslave bond eth2 eth3modprobe hangcheck-timer hangcheck_tick=30 hangcheck_margin=180重启网络service network restartcat /proc/net/bonding/bond0cat /proc/net/bonding/bond14.创建用户groupadd -g 200 dbagroupadd -g 201 oinstalluseradd -u 300 -g oinstall -G dba -d /home/oracle -s /bin/bash -m oraclevi /home/oracle/.bash_profileexport ORACLE_BASE=/apps/oracleexport ORACLE_HOME=$ORACLE_BASE/product/10.2.0export CRS_HOME=$ORACLE_BASE/crsexport PATH=$CRS_HOME/bin:$ORACLE_HOME/jdk/bin:/usr/kerberos/bin:/usr/local/bin:/bin:$PATH:/usr/bin:/usr/X11R6/bin:/home/oracle/bin:$ORACLE_HOME/binexport ORACLE_OWNER=oracleexport LD_LIBRARY_PATH=:$ORACLE_HOME/Slibexport NLS_LANG=American_america.zhs16gbkexport ORACLE_TERM=xtermexport AGENT_HOME=$ORACLE_BASE/agent10galias msh=cd $ORACLE_BASE/admin/mshexport ORACLE_SID=pdsdb15.修改hostsvi /etc/hosts1 BJ-FT-1F-120-112 BJ-FT-1F-120-123 BJ-FT-1F-120-134 BJ-FT-1F-120-141 ftpds_vip12 ftpds_vip23 ftpds_vip34 ftpds_vip41 ftpds_priv12 ftpds_priv23 ftpds_priv34 ftpds_priv46.配置信任关系oracleBJ-FT-1F-120-12 $ mkdir /.ssh chmod 755 /.ssh /usr/bin/ssh-keygen -t rsa /usr/bin/ssh-keygen -t dsa rootBJ-FT-1F-120-11 # cp /home/oracle/.ssh/authorized_keys rac1.authorized_keys scp rac1.authorized_keys shexiang0:/2/root/home/oracle rootBJ-FT-1F-120-12 #chown oracle:dba rac1.authorized_keys oracleBJ-FT-1F-120-12 $mv rac1.authorized_keys .ssh/cd .ssh/mv rac1.authorized_keys authorized_keyscat id_rsa.pub authorized_keyscat id_dsa.pub authorized_keyscp authorized_keys rac2.authorized_keysrootBJ-FT-1F-120-12 .ssh# scp /home/oracle/.ssh/rac2.authorized_keys shexiang0:/4/root/home/oracleoracleBJ-FT-1F-120-14 #chown oracle:dba /home/oracle/rac2.authorized_keys oracleBJ-FT-1F-120-14 $ mkdir /.ssh chmod 755 /.ssh /usr/bin/ssh-keygen -t rsa /usr/bin/ssh-keygen -t dsa mv rac2.authorized_keys .ssh/ cd .ssh/ mv rac2.authorized_keys authorized_keyscat id_rsa.pub authorized_keyscat id_dsa.pub authorized_keyscp authorized_keys rac4.authorized_keysoracleBJ-FT-1F-120-14 # scp /home/oracle/.ssh/rac2.authorized_keys shexiang0:/1/root/home/oracle scp /home/oracle/.ssh/rac2.authorized_keys shexiang0:/2/root/home/oracle scp /home/oracle/.ssh/rac2.authorized_keys shexiang0:/3/root/home/oracle rootBJ-FT-1F-120-11 # chown oracle:dba rac4.authorized_keys mv rac2.authorized_keys .ssh/ cd .ssh/ cp authorized_keys authorized_keys.old mv rac2.authorized_keys authorized_keys在每一个节点执行 ssh BJ-FT-1F-120-11 date ssh BJ-FT-1F-120-12 date ssh BJ-FT-1F-120-13 date ssh BJ-FT-1F-120-14 date ssh ftpds_priv1 date ssh ftpds_priv2 date ssh ftpds_priv3 date ssh ftpds_priv4 date 7.修改核心参数vi /etc/sysctl.conf# Controls the maximum shared segment size, in byteskernel.shmmax = 68719476736# Controls the maximum number of shared memory segments, in pageskernel.shmall = 4294967296kernel.core_uses_pid = 1kernel.panic = 1fs.file-max = 6815744net.ipv4.ip_local_port_range = 9000 65000# For 11g, recommended value for file-max is 6815744# For 10g, uncomment fs.file-max 327679, comment other entries for this parameter and re-run sysctl -p# fs.file-max:327679kernel.msgmni = 2878kernel.sem = 250 32000 100 142kernel.shmmni = 4096net.core.rmem_default = 1048576# For 11g, recommended value for net.core.rmem_max is 4194304net.core.rmem_max = 4194304# For 10g, uncomment net.core.rmem_max 2097152, comment other entries for this parameter and re-run sysctl -p# net.core.rmem_max=2097152net.core.wmem_default = 262144# For 11g, recommended value for wmem_max is 1048576net.core.wmem_max = 1048576# For 10g, uncomment net.core.wmem_max 262144, comment other entries for this parameter and re-run sysctl -p# net.core.wmem_max:262144fs.aio-max-nr = 3145728# For 11g, recommended value for ip_local_port_range is 9000 65500# For 10g, uncomment net.ipv4.ip_local_port_range 1024 65000, comment other entries for this parameter and re-run sysctl -p# net.ipv4.ip_local_port_range:1024 65000 sysctl -p 立刻生效 vi /etc/security/limits.conf oracle soft nproc 2047oracle hard nproc 16384oracle soft nofile 1024oracle hard nofile 65536 vi /etc/pam.d/login session required pam_limits.so vi /etc/rc.d/rc.local /sbin/modprobe hangcheck_timer 设置hangcheck内核模块参数：vi /etc/modprobe.confoptions hangcheck-timer hangcheck_tick=30 hangcheck_margin=180设置hangcheck内核模块开机加载vi /etc/rc.d/rc.local/sbin/modprobe hangcheck-timer 要立即加载模块，执行：modprobe -v hangcheck-timer检查加载情况：rootzhh1 # lsmod | grep hangcheck_timerhangcheck_timer 5593 08.安装ASM包，配置ASM，并搜索ASM-在rac3上安装 ASMLibs, tools, support 三个rpm文件# rpm -ivh *.rpm -nodeps -force- 运行 /etc/init.d/oracleasm configure回答 oracle , dba, y, y 就可以了vi /usr/local/sbin/oraraw.sh # Oracle Cluster Registry (OCR) devices#chown root:oinstall /dev/mapper/ocrchmod 660 /dev/mapper/*raw /dev/raw/raw1 /dev/mapper/ocrsleep 2chown root:oinstall /dev/raw/raw1chmod 660 /dev/raw/raw1# Oracle Cluster Voting disks#chown oracle:oinstall /dev/mapper/vtdskraw /dev/raw/raw2 /dev/mapper/vtdsksleep 2chown oracle:oinstall /dev/raw/raw2chown oracle:oinstall /dev/mapper/archchown oracle:oinstall /dev/mapper/data*chown oracle:oinstall /dev/mapper/redo*9.安装CRScd /apps/software/clusterware/cluvfy./cluvfy stage -post hwos -n BJ-FT-1F-120-11,BJ-FT-1F-120-13 ./cluvfy stage -post hwos -n BJ-FT-1F-120-11,BJ-FT-1F-120-12,BJ-FT-1F-120-13,BJ-FT-1F-120-14 ./runcluvfy.sh stage -pre crsinst -n BJ-FT-1F-120-11,BJ-FT-1F-120-12,BJ-FT-1F-120-13,BJ-FT-1F-120-14 ./runInstaller -updateNodeList CLUSTER_NODES=BJ-FT-1F-120-11,BJ-FT-1F-120-13 LOCAL_NODE=BJ-FT-1F-120-11 ORACLE_HOME=$ORACLE_HOME -local. 为新节点添加CRS Home启动vnc 3:5801/xhost+ su - oracleexport LANG=en在【BJ-FT-1F-120-13】上运行CRS Home下的addNode.sh，启动OUI添加节点。原计划在BJ-FT-1F-120-11上执行，但是上面报错，报错内容为SEVERE: OUI-25023: The local node is not selected for installing this product. Include the local node in the cluster list or perform the installation on the nodes on which the install is to be performed.未找到解决办法，在节点13上执行通过。cd /apps/oracle/crs/oui/bin./addNode.sh按照提示填写节点3公网私网虚拟名称执行安装分别执行3个脚本在节点BJ-FT-1F-120-12运行orainstRoot.sh/apps/oracle/oraInventory/orainstRoot.sh在节点BJ-FT-1F-120-13运行rootaddnode.sh /apps/oracle/crs/install/rootaddnode.sh 最后会报错 /apps/oracle/crs/bin/srvctl add nodeapps -n bj-ft-1f-120-12 -A BJ-FT-1F-120-12-vip//bond0 -o /apps/oracle/crsPRKO-2109 : 地址字符串无效: BJ-FT-1F-120-12-vip//bond0手动执行/apps/oracle/crs/bin/srvctl add nodeapps -n bj-ft-1f-120-12 -A ftpds_vip2//bond0 -o /apps/oracle/crs最后在节点BJ-FT-1F-120-12运行 /apps/oracle/crs/root.shrootBJ-FT-1F-120-12 raw# /apps/oracle/crs/root.sh WARNING: directory /apps/oracle is not owned by rootNo value set for the CRS parameter CRS_OCR_LOCATIONS. Using Values in paramfile.crsChecking to see if Oracle CRS stack is already configured/etc/oracle does not exist. Creating it now.OCR LOCATIONS = /dev/raw/raw1OCR backup directory /apps/oracle/crs/cdata/crs does not exist. Creating nowSetting the permissions on OCR backup directorySetting up NS directoriesOracle Cluster Registry configuration upgraded successfullyWARNING: directory /apps/oracle is not owned by rootclscfg: EXISTING configuration version 3 detected.clscfg: version 3 is 10G Release 2.Successfully accumulated necessary OCR keys.Using ports: CSS=49895 CRS=49896 EVMC=49898 and EVMR=49897.node : node 1: bj-ft-1f-120-11 ftpds_priv1 bj-ft-1f-120-11node 2: bj-ft-1f-120-13 ftpds_priv3 bj-ft-1f-120-13clscfg: Arguments check out successfully.NO KEYS WERE WRITTEN. Supply -force parameter to override.-force is destructive and will destroy any previous clusterconfiguration.Oracle Cluster Registry for cluster has already been initializedStartup will be queued to init within 30 seconds.Adding daemons to inittabExpecting the CRS daemons to be up within 600 seconds.CSS is active on these nodes. bj-ft-1f-120-11 bj-ft-1f-120-13 bj-ft-1f-120-12CSS is active on all nodes.Waiting for the Oracle CRSD and EVMD to startOracle CRS stack installed and running under init(1M) 检查su - oracle crs_stat -tName Type Target State Host -ora.SM1.asm application ONLINE ONLINE bj-f.0-11 ora.11.lsnr application ONLINE ONLINE bj-f.0-11 ora.-11.gsd application ONLINE ONLINE bj-f.0-11 ora.-11.ons application ONLINE ONLINE bj-f.0-11 ora.-11.vip application ONLINE ONLINE bj-f.0-11 ora.-12.gsd application ONLINE ONLINE bj-f.0-12 ora.-12.ons application ONLINE ONLINE bj-f.0-12 ora.-12.vip application ONLINE ONLINE bj-f.0-12 ora.SM2.asm application ONLINE ONLINE bj-f.0-13 ora.13.lsnr application ONLINE ONLINE bj-f.0-13 ora.-13.gsd application ONLINE ONLINE bj-f.0-13 ora.-13.ons application ONLINE ONLINE bj-f.0-13 ora.-13.vip application ONLINE ONLINE bj-f.0-13 ora.pdsdb.db application ONLINE ONLINE bj-f.0-11 ora._taf.cs application ONLINE ONLINE bj-f.0-13 ora.db1.srv application ONLINE ONLINE bj-f.0-11 ora.db2.srv application ONLINE ONLINE bj-f.0-13 ora.b1.inst appli