VMware搭建Oracle-11g-RAC测试环境-For-Linux.docx

VMware搭建Oracle 11g RAC测试环境 For Linux2015-01-11 17:1615359人阅读评论(0)收藏举报分类：Oracle RAC（1）版权声明：本文为博主原创文章，未经博主允许不得转载。环境如下：Linux操作系统：Centos 6.5 64bit （这个版本的redhat 6内核等OS在安装grid最后执行root.sh时会出现crs-4124，是oracle的bug）VMware version：Workstation 8.0.3 build-703057Oracle版本：11G R2 grid/oracle 64位 .0 for linux步骤规划：1，配置虚拟机、安装Linux操作系统2，配置OS3，配置ASM4，安装Oracle grid5，安装Oracle database6，创建ASM磁盘组7，建立数据库实例1，配置虚拟机、安装Linux操作系统1.1 新建虚拟机：RAC1和RAC2，并安装OS下载Centos的iso后新建虚拟机后安装即可，这里略。注意内存分配，每个虚拟机至少1.5G内存比较好，我的机器8G内存，给每个虚拟机分了2G，给每个虚拟机分配50GB硬盘容量，配置2个桥接网卡。1.2 配置Share Disks新建共享硬盘，在DOS VM安装目录下执行：cd D F:/Software/vm8/vmware-vdiskmanager.exe -c -s 1000Mb -a lsilogic -t 2 K:VMDiskSharediskocr.vmdkvmware-vdiskmanager.exe -c -s 1000Mb -a lsilogic -t 2 K:VMDiskSharediskocr2.vmdkvmware-vdiskmanager.exe -c -s 1000Mb -a lsilogic -t 2 K:VMDiskSharediskvotingdisk.vmdkvmware-vdiskmanager.exe -c -s 20000Mb -a lsilogic -t 2 K:VMDiskSharediskdata.vmdkvmware-vdiskmanager.exe -c -s 10000Mb -a lsilogic -t 2 K:VMDiskSharediskbackup.vmdk配置虚拟机RAC1 RAC2目录下的vmx虚拟机配置文件，在行最后添加如下：scsi1.present = TRUEscsi1.virtualDev = lsilogicscsi1.sharedBus = virtualscsi1:1.present = TRUEscsi1:1.mode = independent-persistentscsi1:1.filename = K:VMDiskSharediskocr.vmdkscsi1:1.deviceType = plainDiskscsi1:2.present = TRUEscsi1:2.mode = independent-persistentscsi1:2.filename = K:VMDiskSharediskvotingdisk.vmdkscsi1:2.deviceType = plainDiskscsi1:3.present = TRUEscsi1:3.mode = independent-persistentscsi1:3.filename = K:VMDiskSharediskdata.vmdkscsi1:3.deviceType = plainDiskscsi1:4.present = TRUEscsi1:4.mode = independent-persistentscsi1:4.filename = K:VMDiskSharediskbackup.vmdkscsi1:4.deviceType = plainDiskscsi1:5.present = TRUEscsi1:5.mode = independent-persistentscsi1:5.filename = K:VMDiskSharediskocr2.vmdkscsi1:5.deviceType = plainDiskdisk.locking = falsediskLib.dataCacheMaxSize = 0diskLib.dataCacheMaxReadAheadSize = 0diskLib.DataCacheMinReadAheadSize = 0diskLib.dataCachePageSize = 4096diskLib.maxUnsyncedWrites = 0保存退出。最后虚拟机图大致如下：再次提醒注意内存、硬盘和网卡配置。配置虚拟机、安装Linux操作系统完成。2，配置OS以下没有特别说明都需要同时在两个节点执行。2.1 关闭防火墙、selinux(考虑到后面安装时会碰到一些情况，所以需要执行这步骤）关闭selinuxsetenforce 0 rootrac1 # vim /etc/selinux/configSELINUX=disabled关闭防火墙service iptables stopchkconfig iptables off2.2 配置网络修改节点主机名为rac1，rac2hostname rac1hostname rac2或修改/etc/sysconfig/networkRAC1eth0: 30eth1: 21RAC2eth0: 40eth1: 22host文件，两个节点都配置Vim /etc/hosts30 rac140 rac231 rac1-vip41 rac2-vip21 rac1-priv22 rac2-priv00 scan-ip保存退出。配置完之后互相ping看看是否连通。2.3 配置CTSS服务、节点SSH等效CTSS 需要把ntp服务停止和删除配置文件及pid文件 service ntpd stop chkconfig ntpd off rm -rf /etc/ntpd.conf rm -rf /var/run/ntpd.pid 查看ctss服务资源 crsctl check ctssSSH在grid安装时会提供配置步骤，所以不需要手工配置。ssh具体配置rac1 rac2mkdir /.sshchmod 700 /.sshssh-keygen -t rsassh-keygen -t dsarac1 rac2cat /.ssh/id_rsa.pub /.ssh/authorized_keyscat /.ssh/id_dsa.pub /.ssh/authorized_keys测试SSHssh rac2ssh rac12.4 配置Oracle软件安装用户、组、环境变量、安装目录#组groupadd -g 1300 dbagroupadd -g 1301 opergroupadd -g 1000 oinstallgroupadd -g 1200 asmadmingroupadd -g 1201 asmdbagroupadd -g 1202 asmoper#用户useradd -m -u 1100 -g oinstall -G asmadmin,asmdba,asmoper -s /bin/bash griduseradd -m -u 1101 -g oinstall -G dba,oper,asmdba -s /bin/bash oracle#用户环境变量修改home下的 .bash_profileORACLEexport TMP=/tmpexport TMPDIR=$TMPexport ORACLE_SID=RAC1 # RAC1export ORACLE_SID=RAC2 # RAC2export ORACLE_BASE=/u01/app/oracleexport ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1export TNS_ADMIN=$ORACLE_HOME/network/adminexport PATH=/usr/sbin:$PATHexport PATH=$ORACLE_HOME/bin:$PATHexport LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/libexport CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlibumask 022GRIDexport TMP=/tmpexport TMPDIR=$TMPexport ORACLE_SID=+ASM1 # RAC1export ORACLE_SID=+ASM2 # RAC2export ORACLE_BASE=/u01/app/gridexport ORACLE_HOME=/u01/app/11.2.0/gridexport PATH=/usr/sbin:$PATHexport PATH=$ORACLE_HOME/bin:$PATHexport LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/libexport CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlibumask 022配置目录mkdir -p /u01/app/11.2.0/gridmkdir -p /u01/app/gridmkdir -p /u01/app/oraclechown grid:oinstall /u01/app/11.2.0/gridchown grid:oinstall /u01/app/gridchown oracle:oinstall /u01/app/oraclechmod -R 775 /u01/chown -R grid:oinstall /u012.5 配置系统文件配置oracle、grid用户的shell限制vim /etc/security/limits.conf添加grid soft nproc 2047grid hard nproc 16384grid soft nofile 1024grid hard nofile 65536oracle soft nproc 2047oracle hard nproc 16384oracle soft nofile 1024oracle hard nofile 65536修改/etc/pam.d/login配置文件Vim /etc/pam.d/login添加 session required pam_limits.so修改内核配置文件/etc/sysctl.conffs.aio-max-nr = 1048576fs.file-max = 6815744kernel.shmall = 2097152kernel.shmmax = 1054472192kernel.shmmni = 4096kernel.sem = 250 32000 100 128net.ipv4.ip_local_port_range = 9000 65500net.core.rmem_default = 262144net.core.rmem_max = 4194304net.core.wmem_default = 262144net.core.wmem_max = 1048586net.ipv4.tcp_wmem = 262144 262144 262144net.ipv4.tcp_rmem = 4194304 4194304 4194304保存退出后执行 sysctl -p 立即生效。2.6 配置共享磁盘 （在一个节点执行即可）查看分区ls /dev/sd*使用fdisk进行分区，其他区按照自己的规划划分就可以了，例：fdisk /dev/sdbrootrac1 # fdisk /dev/sdbCommand (m for help): nCommand action e extended p primary partition (1-4)pPartition number (1-4): 1First cylinder (1-1000, default 1):Using default value 1Last cylinder, +cylinders or +sizeK,M,G (1-1000, default 1000): +500MCommand (m for help): nCommand action e extended p primary partition (1-4)pPartition number (1-4): 2First cylinder (502-1000, default 502):Using default value 502Last cylinder, +cylinders or +sizeK,M,G (502-1000, default 1000):Using default value 1000Command (m for help): wThe partition table has been altered!最后配置完后查看分区fidks -lrootrac1 # ls /dev/sd*/dev/sda /dev/sda1 /dev/sda2 /dev/sdb /dev/sdb1 /dev/sdc /dev/sdc1 /dev/sdd /dev/sdd1 /dev/sdd2 /dev/sde /dev/sde1 /dev/sde2 /dev/sdf /dev/sdf12.7 安装Oracle grid等需要的包binutils-.2-5.11.el6 (x86_64)compat-libcap1-1.10-1 (x86_64)compat-libstdc+-33-3.2.3-69.el6 (x86_64)compat-libstdc+-33-3.2.3-69.el6.i686gcc-4.4.4-13.el6 (x86_64)gcc-c+-4.4.4-13.el6 (x86_64)glibc-2.12-1.7.el6 (i686)glibc-2.12-1.7.el6 (x86_64)glibc-devel-2.12-1.7.el6 (x86_64)glibc-devel-2.12-1.7.el6.i686kshlibgcc-4.4.4-13.el6 (i686)libgcc-4.4.4-13.el6 (x86_64)libstdc+-4.4.4-13.el6 (x86_64)libstdc+-4.4.4-13.el6.i686libstdc+-devel-4.4.4-13.el6 (x86_64)libstdc+-devel-4.4.4-13.el6.i686libaio-0.3.107-10.el6 (x86_64)libaio-0.3.107-10.el6.i686libaio-devel-0.3.107-10.el6 (x86_64)libaio-devel-0.3.107-10.el6.i686make-3.81-19.el6sysstat-9.0.4-11.el6 (x86_64)3，配置ASM3.1 查看操作系统内核版本对应下载uname -a下载oracleasm/technetwork/topics/linux/asmlib/index-101839.html安装yum -y install kmod-oracleasm.x86_64rpm -ivh oracleasmlib-2.0.4-1.el6.x86_64.rpmrpm -ivh oracleasm-support-2.1.8-1.el6.x86_64.rpm3.2 配置ASM磁盘在两个节点上执行rootrac1 oracle# /etc/init.d/oracleasm configureConfiguring the Oracle ASM library driver.This will configure the on-boot properties of the Oracle ASM librarydriver. The following questions will determine whether the driver isloaded on boot and what permissions it will have. The current valueswill be shown in brackets (). Hitting without typing ananswer will keep that current value. Ctrl-C will abort.Default user to own the driver interface : gridDefault group to own the driver interface : asmdbaStart Oracle ASM library driver on boot (y/n) n: yScan for Oracle ASM disks on boot (y/n) y:Writing Oracle ASM library driver configuration: doneInitializing the Oracle ASMLib driver: OK Scanning the system for Oracle ASMLib disks: OK rootrac1 oracle# /etc/init.d/oracleasm scandisksScanning the system for Oracle ASMLib disks: OK rootrac1 # /etc/init.d/oracleasm listdisksVOL1VOL2VOL3VOL4VOL5VOL6VOL7注：如果没有查看到asm磁盘组需要查看之前的selinux是否有禁用。4，安装Oracle grid安装前可以执行Oracle提供的检查环境工具./runcluvfy.shstage-precrsinst-nrac1,rac2-fixup-verbose4.1 在安装目录下执行./runinstaller，如果出现安装grid提示must be configured to display at least 256 colors Failed，就执行如下root 下先执行#xhost +su - gridexport DISPLAY=:0.0注意在此建立SSH等效。执行setup后点test测试。如果不成功就手工配置SSH再进行测试。这个地方忘记截图了，就是类似这个图。定义磁盘组名字，这里我选择normal冗余，需要选择3个VOL。这里是把OCR和votingdisk安装在所选磁盘。后面还需要新建asm磁盘组来安装其他数据文件。这里的software location应该是 /u01/app/11.2.0/grid通过Oracle的系统环境检查，安装缺失的包和修改系统环境参数等。安装65%的时候可能会出现卡顿的情况，这个时候需要检查防火墙和selinux是否关闭，详细情况可以点击detail查看安装进度，如出现拷贝文件到RAC2节点的信息则正常。根据提示执行orainstRoot.sh和root.sh脚本，先执行rac1后再执行rac2。在执行root .sh时可能在的时候会提示出现crs-4124，这个是oracle的bug，当在redhat 6等OS下安装版本Oracle的时候会出现此错误。解决如下：在执行root.sh时，在出现Adding daemon to inittab时，需要不停地执行命令：/bin/dd if=/var/tmp/.oracle/npohasd of=/dev/null bs=1024 count=1rootrac1 grid# ./root.shRunning Oracle 11g root.sh script.The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /u01/app/11.2.0/gridEnter the full pathname of the local bin directory: /usr/local/bin: Copying dbhome to /usr/local/bin . Copying oraenv to /usr/local/bin . Copying coraenv to /usr/local/bin .Entries will be added to the /etc/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root.sh script.Now product-specific root actions will be performed.2015-01-06 23:46:35: Parsing the host name2015-01-06 23:46:35: Checking for super user privileges2015-01-06 23:46:35: User has super user privilegesUsing configuration parameter file: /u01/app/11.2.0/grid/crs/install/crsconfig_paramsCreating trace directoryLOCAL ADD MODECreating OCR keys for user root, privgrp root.Operation successful. root wallet root wallet cert root cert export peer wallet profile reader wallet pa wallet peer wallet keys pa wallet keys peer cert request pa cert request peer cert pa cert peer root cert TP profile reader root cert TP pa root cert TP peer pa cert TP pa peer cert TP profile reader pa cert TP profile reader peer cert TP peer user cert pa user certAdding daemon to inittabCRS-4123: Oracle High Availability Services has been started.ohasd is startingADVM/ACFS is not supported on centos-release-6-6.el6.centos.12.2.x86_64CRS-2672: Attempting to start ora.gipcd on rac1CRS-2672: Attempting to start ora.mdnsd on rac1CRS-2676: Start of ora.gipcd on rac1 succeededCRS-2676: Start of ora.mdnsd on rac1 succeededCRS-2672: Attempting to start ora.gpnpd on rac1CRS-2676: Start of ora.gpnpd on rac1 succeededCRS-2672: Attempting to start ora.cssdmonitor on rac1CRS-2676: Start of ora.cssdmonitor on rac1 succeededCRS-2672: Attempting to start ora.cssd on rac1CRS-2672: Attempting to start ora.diskmon on rac1CRS-2676: Start of ora.diskmon on rac1 succeededCRS-2676: Start of ora.cssd on rac1 succeededCRS-2672: Attempting to start ora.ctssd on rac1CRS-2676: Start of ora.ctssd on rac1 succeededASM created and started successfully.DiskGroup DATA created successfully.clscfg: -install mode specifiedSuccessfully accumulated necessary OCR keys.Creating OCR keys for user root, privgrp root.Operation successful.CRS-2672: Attempting to start ora.crsd on rac1CRS-2676: Start of ora.crsd on rac1 succeededCRS-4256: Updating the profileSuccessful addition of voting disk 60b5c5168a574f3fbfabf7d93a17c18f.Successful addition of voting disk a59fab87e8634f00bf5b44656d6caaf1.Successful addition of voting disk 228f0986e7c84f20bf66ed6905b69e43.Successfully replaced voting disk group with +DATA.CRS-4256: Updating the profileCRS-4266: Voting file(s) successfully replaced# STATE File Universal Id File Name Disk group- - - - -1. ONLINE 60b5c5168a574f3fbfabf7d93a17c18f (ORCL:VOL1) DATA2. ONLINE a59fab87e8634f00bf5b44656d6caaf1 (ORCL:VOL2) DATA3. ONLINE 228f0986e7c84f20bf66ed6905b69e43 (ORCL:VOL4) DATALocated 3 voting disk(s).CRS-2673: Attempting to stop ora.crsd on rac1CRS-2677: Stop of ora.crsd on rac1 succeededCRS-2673: Attempting to stop ora.asm on rac1CRS-2677: Stop of ora.asm on rac1 succeededCRS-2673: Attempting to stop ora.ctssd on rac1CRS-2677: Stop of ora.ctssd on rac1 succeededCRS-2673: Attempting to stop ora.cssdmonitor on rac1CRS-2677: Stop of ora.cssdmonitor on rac1 succeededCRS-2673: Attempting to stop ora.cssd on rac1CRS-2677: Stop of ora.cssd on rac1 succeededCRS-2673: Attempting to stop ora.gpnpd on rac1CRS-2677: Stop of ora.gpnpd on rac1 succeededCRS-2673: Attempting to stop ora.gipcd on rac1CRS-2677: Stop of ora.gipcd on rac1 succeededCRS-2673: Attempting to stop ora.mdnsd on rac1CRS-2677: Stop of ora.mdnsd on rac1 succeededCRS-2672: Attempting to start ora.mdnsd on rac1CRS-2676: Start of ora.mdnsd on rac1 succeededCRS-2672: Attempting to start ora.gipcd on rac1CRS-2676: Start of ora.gipcd on rac1 succeededCRS-2672: Attempting to start ora.gpnpd on rac1CRS-2676: Start of ora.gpnpd on rac1 succeededCRS-2672: Attempting to start ora.cssdmonitor on rac1CRS-2676: Start of ora.cssdmonitor on rac1 succeededCRS-2672: Attempting to start ora.cssd on rac1CRS-2672: Attempting to start ora.diskmon on rac1CRS-2676: Start of ora.diskmon on rac1 succeededCRS-2676: Start of ora.cssd on rac1 succeededCRS-2672: Attempting to start ora.ctssd on rac1CRS-2676: Start of ora.ctssd on rac1 succeededCRS-2672: Attempting to start ora.asm on rac1CRS-2676: Start of ora.asm on rac1 succeededCRS-2672: Attempting to start ora.crsd on rac1CRS-2676: Start of ora.crsd on rac1 succeededCRS-2672: Attempting to start ora.evmd on rac1CRS-2676: Start of ora.evmd on rac1 succeededCRS-2672: Attempting to start ora.asm on rac1CRS-2676: Start of ora.asm on rac1 succeededCRS-2672: Attempting to start ora.DATA.dg on rac1CRS-2676: Start of ora.DATA.dg on rac1 succeededrac1 2015/01/06 23:52:15 /u01/app/11.2.0/grid/cdata/rac1/backup_20150106_235215.olrPreparing packages for installation.cvuqdisk-1.0.7-1Configure Oracle Grid Infrastructure for a Cluster . succeededUpdating inventory properties for clusterwareStarting Oracle Universal Installer.Checking swap space: must be greater than 500 MB. Actual 3906 MB PassedThe inventory pointer is located at /etc/oraInst.locThe inventory is located at /u01/app/oraInventoryUpdateNodeList was successful.root.sh执行完毕。在rac2节点执行也需要此步骤。检查grid组件启动情况gridrac1 $ crsctl check crsCRS-4638: Oracle High Availability Services is onlineCRS-4537: Cluster Ready Services is onlineCRS-4529: Cluster Synchronization Services is onlineCRS-4533: Event Manager is onlinegridrac1 $ crs_stat -v -tName Type R/RA F/FT Target State Host-ora.DATA.dg ora.up.type 0/5 0/ ONLINE ONLINE rac1ora.ER.lsnr ora.er.type 0/5 0/ ONLINE ONLINE rac1ora.N1.lsnr ora.er.type 0/5 0/0 ONLINE ONLINE rac1ora.asm ora.asm.typ