Oracle11g搭建RAC步骤.doc

一、网卡绑定1、修改配置文件/etc/modprobe.confalias bond0 bondingalias bond1 bonding/etc/sysconfig/network-scripts/ifcfg-bond0DEVICE=bond0BOOTPROTO=staticONBOOT=yesUSERCTL=noIPADDR=192.168.56.101BROADCAST=192.168.56.255NETWORK=192.168.56.0NETMASK=255.255.255.0TYPE=EthernetBONDING_OPTS=mode=1 miimon=100/etc/sysconfig/network-scripts/ifcfg-bond1DEVICE=bond1BOOTPROTO=staticONBOOT=yesUSERCTL=noIPADDR=192.168.136.101BROADCAST=192.168.136.255NETWORK=192.168.136.0NETMASK=255.255.255.0TYPE=EthernetBONDING_OPTS=mode=1 miimon=100/etc/sysconfig/network-scripts/ifcfg-eth0# Advanced Micro Devices AMD 79c970 PCnet32 LANCEDEVICE=eth0BOOTPROTO=noneONBOOT=yesUSERCTL=noHWADDR=00:0c:29:9e:87:44MASTER=bond0SLAVE=yes/etc/sysconfig/network-scripts/ifcfg-eth1# Advanced Micro Devices AMD 79c970 PCnet32 LANCEDEVICE=eth1BOOTPROTO=noneONBOOT=yesHWADDR=00:0c:29:9e:87:4eMASTER=bond0SLAVE=yes/etc/sysconfig/network-scripts/ifcfg-eth2# Advanced Micro Devices AMD 79c970 PCnet32 LANCEDEVICE=eth2BOOTPROTO=noneONBOOT=yesUSERCTL=noHWADDR=00:0c:29:9e:87:58MASTER=bond1SLAVE=yes/etc/sysconfig/network-scripts/ifcfg-eth3# Advanced Micro Devices AMD 79c970 PCnet32 LANCEDEVICE=eth3BOOTPROTO=noneONBOOT=yesHWADDR=00:0c:29:9e:87:62USERCTL=noMASTER=bond1SLAVE=yes命令：ifensalve bond0 eth0 eth1 ifensalve bond1 eth2 eth3service network restart2、检查配置cat /proc/net/bonding/bond0Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)Bonding Mode: fault-tolerance (active-backup)Primary Slave: NoneCurrently Active Slave: eth0MII Status: upMII Polling Interval (ms): 100Up Delay (ms): 0Down Delay (ms): 0Slave Interface: eth0MII Status: upLink Failure Count: 0Permanent HW addr: 00:0c:29:9e:87:44Slave queue ID: 0Slave Interface: eth1MII Status: upLink Failure Count: 0Permanent HW addr: 00:0c:29:9e:87:4eSlave queue ID: 0cat /proc/net/bonding/bond1Ethernet Channel Bonding Driver: v3.6.0 (September 26, 2009)Bonding Mode: fault-tolerance (active-backup)Primary Slave: NoneCurrently Active Slave: eth2MII Status: upMII Polling Interval (ms): 100Up Delay (ms): 0Down Delay (ms): 0Slave Interface: eth2MII Status: upLink Failure Count: 0Permanent HW addr: 00:0c:29:9e:87:58Slave queue ID: 0Slave Interface: eth3MII Status: upLink Failure Count: 0Permanent HW addr: 00:0c:29:9e:87:62Slave queue ID: 03、测试自动切换在vmware workstation界面上将相应的网卡(cat /proc/met/bonding/bondN)disconnect，依次测试。用ifdown、ifup方式不行不会自动切换/etc/modprobe.conf (redhat6.0+:/etc/modprobe.d/dist.conf)需要添加fail_over_mac=1 =切换的时候改MAC地址，不加不会切换成功(virtualbox、vmware workstation下测试)二、建用户和组groupadd oinstall -g 1000groupadd dba -g 1001groupadd asmadmin -g 1101groupadd asmdba -g 1102groupadd asmoper -g 1103useradd-m -u 1001 -g oinstall -G asmadmin,asmdba,asmoper -d /home/grid griduseradd -m -u 1002 -g dba -G oinstall,asmdba,asmoper -d /home/oracle oracle三、创建磁盘ocr、data .KERNEL=sd?,BUS=scsi,PROGRAM=/sbin/scsi_id -g -u /dev/$name,RESULT=36000c293bc7e9c8139a42cb118aa8913,NAME=asm-ocr,OWNER=grid,GROUP=asmadmin,MODE=0660 RUN+=/bin/raw /dev/raw/raw1 /dev/asm-ocrKERNEL=sd?,BUS=scsi,PROGRAM=/sbin/scsi_id -g -u /dev/$name,RESULT=36000c2910f9afeae89aa3ef976f95c50,NAME=asm-data,OWNER=grid,GROUP=asmadmin,MODE=0660 RUN+=/bin/raw /dev/raw/raw2 /dev/asm-dataACTION=add,KERNEL=raw1, OWNER=grid, GROUP=asmadmin, MODE=660ACTION=add,KERNEL=raw2, OWNER=grid, GROUP=asmadmin, MODE=660四、安装Grid Infrastructure这一步老是报错(报diskgroup相关错误，忘记记录了)，改成预分配harddisk空间后成功node1:rootoel501 # /opt/grid/app/11.2.0/grid/root.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /opt/grid/app/11.2.0/gridEnter the full pathname of the local bin directory: /usr/local/bin: The file dbhome already exists in /usr/local/bin. Overwrite it? (y/n) n: The file oraenv already exists in /usr/local/bin. Overwrite it? (y/n) n: The file coraenv already exists in /usr/local/bin. Overwrite it? (y/n) n: Creating /etc/oratab file.Entries will be added to the /etc/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root script.Now product-specific root actions will be performed.Using configuration parameter file: /opt/grid/app/11.2.0/grid/crs/install/crsconfig_paramsCreating trace directoryUser ignored Prerequisites during installationOLR initialization - successful root wallet root wallet cert root cert export peer wallet profile reader wallet pa wallet peer wallet keys pa wallet keys peer cert request pa cert request peer cert pa cert peer root cert TP profile reader root cert TP pa root cert TP peer pa cert TP pa peer cert TP profile reader pa cert TP profile reader peer cert TP peer user cert pa user certCRS-2728: A resource type with the name ora.daemon.type is already registeredCRS-4000: Command Add failed, or completed with errors.CRS-2728: A resource type with the name ora.haip.type is already registeredCRS-4000: Command Add failed, or completed with errors.CRS-2728: A resource type with the name ora.mdns.type is already registeredCRS-4000: Command Add failed, or completed with errors.Failed to register Grid Infrastructure type ora.mdns.type at /opt/grid/app/11.2.0/grid/crs/install/crsconfig_lib.pm line 7777./opt/grid/app/11.2.0/grid/perl/bin/perl -I/opt/grid/app/11.2.0/grid/perl/lib -I/opt/grid/app/11.2.0/grid/crs/install /opt/grid/app/11.2.0/grid/crs/install/rootcrs.pl execution failedrootoel501 # /opt/grid/app/11.2.0/grid/root.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /opt/grid/app/11.2.0/gridEnter the full pathname of the local bin directory: /usr/local/bin: The file dbhome already exists in /usr/local/bin. Overwrite it? (y/n) n: The file oraenv already exists in /usr/local/bin. Overwrite it? (y/n) n: The file coraenv already exists in /usr/local/bin. Overwrite it? (y/n) n: Entries will be added to the /etc/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root script.Now product-specific root actions will be performed.Using configuration parameter file: /opt/grid/app/11.2.0/grid/crs/install/crsconfig_paramsUser ignored Prerequisites during installationCINT at /opt/grid/app/11.2.0/grid/crs/install/crsconfig_lib.pm line 1379./opt/grid/app/11.2.0/grid/perl/bin/perl -I/opt/grid/app/11.2.0/grid/perl/lib -I/opt/grid/app/11.2.0/grid/crs/install /opt/grid/app/11.2.0/grid/crs/install/rootcrs.pl execution failedOracle root script execution aborted!rootoel501 # /opt/grid/app/11.2.0/grid/root.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /opt/grid/app/11.2.0/gridEnter the full pathname of the local bin directory: /usr/local/bin: The file dbhome already exists in /usr/local/bin. Overwrite it? (y/n) n: The file oraenv already exists in /usr/local/bin. Overwrite it? (y/n) n: The file coraenv already exists in /usr/local/bin. Overwrite it? (y/n) n: Entries will be added to the /etc/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root script.Now product-specific root actions will be performed.Using configuration parameter file: /opt/grid/app/11.2.0/grid/crs/install/crsconfig_paramsUser ignored Prerequisites during installationCRS-2672: Attempting to start ora.mdnsd on oel501CRS-2676: Start of ora.mdnsd on oel501 succeededCRS-2672: Attempting to start ora.gpnpd on oel501CRS-2676: Start of ora.gpnpd on oel501 succeededCRS-2672: Attempting to start ora.cssdmonitor on oel501CRS-2672: Attempting to start ora.gipcd on oel501CRS-2676: Start of ora.cssdmonitor on oel501 succeededCRS-2676: Start of ora.gipcd on oel501 succeededCRS-2672: Attempting to start ora.cssd on oel501CRS-2672: Attempting to start ora.diskmon on oel501CRS-2676: Start of ora.diskmon on oel501 succeededCRS-2676: Start of ora.cssd on oel501 succeededASM created and started successfully.Disk Group OCR created successfully.clscfg: -install mode specifiedSuccessfully accumulated necessary OCR keys.Creating OCR keys for user root, privgrp root.Operation successful.CRS-4256: Updating the profileSuccessful addition of voting disk 71e7f2f09e7d4f1bbfa5e6eebf203d8f.Successfully replaced voting disk group with +OCR.CRS-4256: Updating the profileCRS-4266: Voting file(s) successfully replaced# STATE File Universal Id File Name Disk group- - - - -1. ONLINE 71e7f2f09e7d4f1bbfa5e6eebf203d8f (/dev/raw/raw1) OCRLocated 1 voting disk(s).CRS-2672: Attempting to start ora.asm on oel501CRS-2676: Start of ora.asm on oel501 succeededCRS-2672: Attempting to start ora.OCR.dg on oel501CRS-2676: Start of ora.OCR.dg on oel501 succeededConfigure Oracle Grid Infrastructure for a Cluster . succeedednode2:rootoel502 # /opt/grid/app/oraInventory/orainstRoot.shrootoel502 # /opt/grid/app/11.2.0/grid/root.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /opt/grid/app/11.2.0/gridEnter the full pathname of the local bin directory: /usr/local/bin: The file dbhome already exists in /usr/local/bin. Overwrite it? (y/n) n: The file oraenv already exists in /usr/local/bin. Overwrite it? (y/n) n: The file coraenv already exists in /usr/local/bin. Overwrite it? (y/n) n: Creating /etc/oratab file.Entries will be added to the /etc/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root script.Now product-specific root actions will be performed.Using configuration parameter file: /opt/grid/app/11.2.0/grid/crs/install/crsconfig_paramsCreating trace directoryUser ignored Prerequisites during installationOLR initialization - successfulCRS-2728: A resource type with the name ora.daemon.type is already registeredCRS-4000: Command Add failed, or completed with errors.CRS-2728: A resource type with the name ora.haip.type is already registeredCRS-4000: Command Add failed, or completed with errors.CRS-2728: A resource type with the name ora.mdns.type is already registeredCRS-4000: Command Add failed, or completed with errors.Failed to register Grid Infrastructure type ora.mdns.type at /opt/grid/app/11.2.0/grid/crs/install/crsconfig_lib.pm line 7777./opt/grid/app/11.2.0/grid/perl/bin/perl -I/opt/grid/app/11.2.0/grid/perl/lib -I/opt/grid/app/11.2.0/grid/crs/install /opt/grid/app/11.2.0/grid/crs/install/rootcrs.pl execution failedrootoel502 # /opt/grid/app/11.2.0/grid/root.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /opt/grid/app/11.2.0/gridEnter the full pathname of the local bin directory: /usr/local/bin: The file dbhome already exists in /usr/local/bin. Overwrite it? (y/n) n: The file oraenv already exists in /usr/local/bin. Overwrite it? (y/n) n: The file coraenv already exists in /usr/local/bin. Overwrite it? (y/n) n: Entries will be added to the /etc/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root script.Now product-specific root actions will be performed.Using configuration parameter file: /opt/grid/app/11.2.0/grid/crs/install/crsconfig_paramsUser ignored Prerequisites during installationCRS-2672: Attempting to start ora.mdnsd on oel502CRS-2676: Start of ora.mdnsd on oel502 succeededCRS-2672: Attempting to start ora.gpnpd on oel502CRS-2676: Start of ora.gpnpd on oel502 succeededCRS-2672: Attempting to start ora.cssdmonitor on oel502CRS-2672: Attempting to start ora.gipcd on oel502CRS-2676: Start of ora.gipcd on oel502 succeededCRS-2676: Start of ora.cssdmonitor on oel502 succeededCRS-2672: Attempting to start ora.cssd on oel502CRS-2672: Attempting to start ora.diskmon on oel502CRS-2676: Start of ora.diskmon on oel502 succeededCRS-2674: Start of ora.cssd on oel502 failedCRS-2679: Attempting to clean ora.cssd on oel502CRS-2681: Clean of ora.cssd on oel502 succeededCRS-2673: Attempting to stop ora.gipcd on oel502CRS-2677: Stop of ora.gipcd on oel502 succeededCRS-2673: Attempting to stop ora.cssdmonitor on oel502CRS-2677: Stop of ora.cssdmonitor on oel502 succeededCRS-2673: Attempting to stop ora.gpnpd on oel502CRS-2677: Stop of ora.gpnpd on oel502 succeededCRS-2673: Attempting to stop ora.mdnsd on oel502CRS-2677: Stop of ora.mdnsd on oel502 succeededCRS-4000: Command Start failed, or completed with errors.CSS startup failed with return code 1The exlusive mode cluster start failed, see Grid Infrastructure alert log for more informationInitial cluster configuration failed. See /opt/grid/app/11.2.0/grid/cfgtoollogs/crsconfig/rootcrs_oel502.log for details/opt/grid/app/11.2.0/grid/perl/bin/perl -I/opt/grid/app/11.2.0/grid/perl/lib -I/opt/grid/app/11.2.0/grid/crs/install /opt/grid/app/11.2.0/grid/crs/install/rootcrs.pl execution failedrootoel502 # /opt/grid/app/11.2.0/grid/root.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /opt/grid/app/11.2.0/gridEnter the full pathname of the local bin directory: /usr/local/bin: The file dbhome already exists in /usr/local/bin. Overwrite it? (y/n) n: The file oraenv already exists in /usr/local/bin. Overwrite it? (y/n) n: The file coraenv already exists in /usr/local/bin. Overwrite it? (y/n) n: Entries will be added to the /etc/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root script.Now product-specific root actions will be performed.Using configuration parameter file: /opt/grid/app/11.2.0/grid/crs/install/crsconfig_paramsUser ignored Prerequisites during installationCRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node oel501, number 1, and is terminatingAn active cluster was found during exclusive startup, restarting to join the clusterrootoel502 grid# /opt/grid/app/11.2.0/grid/crs/install/rootcrs.pl -verbose -deconfig -forceUsing configuration parameter file: /opt/grid/app/11.2.0/grid/crs/install/crsconfig_paramsNetwork exists: 1/192.168.56.0/255.255.255.0/bond1, type staticVIP exists: /oel501-vip/192.168.56.201/192.168.56.0/255.255.255.0/bond1, hosting node oel501GSD existsONS exists: Local port 6100, remote port 6200, EM port 2016PRKO-2439 : VIP does not exist.PRKO-2313 : VIP oel502 does not exist.CLSU-00100: Operating System function: failed failed with error data: 2CLSU-00101: Operating System error message: No such file or directoryCLSU-00103: error location: scrsearch3CLSU-00104: additional error information: id doesnt exist scls_scr_setvalCRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on oel502CRS-2673: Attempting to stop ora.crsd on oel502CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on oel502CRS-2673: Attempting to stop ora.OCR.dg on oel502CRS-2677: Stop of ora.OCR.dg on oel502 succeededCRS-2673: Attempting to stop ora.asm on oel502CRS-2677: Stop of ora.asm on oel502 succeededCRS-2792: Shutdown of Cluster Ready Services-managed resources on oel502 has completedCRS-2677: Stop of ora.crsd on oel502 succeededCRS-2673: Attempting to stop ora.crf on oel502CRS-2673: Attempting to stop ora.ctssd on oel502CRS-2673: Attempting to stop ora.evmd on oel502CRS-2673: Attempting to stop ora.asm on oel502CRS-2673: Attempting to stop ora.mdnsd on oel502CRS-2677: Stop of ora.evmd on oel502 succeededCRS-2677: Stop of ora.crf on oel502 succeededCRS-2677: Stop of ora.ctssd on oel502 succeededCRS-2677: Stop of ora.mdnsd on oel502 succeededCRS-2677: Stop of ora.asm on oel502 succeededCRS-2673: Attempting to stop ora.cluster_interconnect.haip on oel502CRS-2677: Stop of ora.cluster_interconnect.haip on oel502 succeededCRS-2673: Attempting to stop ora.cssd on oel502CRS-2677: Stop of ora.cssd on oel502 succeededCRS-2673: Attempting to stop ora.gipcd on oel502CRS-2677: Stop of ora.gipcd on oel502 succeededCRS-2673: Attempting to stop ora.gpnpd on oel502CRS-2677: Stop of ora.gpnpd on oel502 succeededCRS-2793: Shutdown of Oracle High Availability Services-managed resources on oel502 has completedCRS-4133: Oracle High Availability Services has been stopped.Successfully deconfigured Oracle clusterware stack on this noderootoel502 grid# /opt/grid/app/11.2.0/grid/root.sh Performing root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /o