vmware workstation 9 安装rdht6 and oracle11g rac.docx

Vmware workstation9（redhat6 64）安装oracle rac第一章 环境准备一 创建虚拟机11grac1二 创建操作系统 Redhat 6.2 64 磁盘20G 自动分区（swap 4G） 选择包时除了BASE自带的包外，选中以下选项：Compatibility librariesftp servergnome-desktopx windows system Development toolsChinese support关闭selinux：/etc/selinux/configSELINUX=disabled安装vmtoolsrootrac1 # cd /media/VMware Tools/rootrac1 # cp VMwareTools-9.2.0-799703.tar.gz /rootrootrac1 # cd /rootrootrac1 # tar -xvf VMwareTools-9.2.0-799703.tar.gzrootrac1 # cd vmware-tools-distrib/rootrac1 vmware-tools-distrib# lsbin doc etc FILES INSTALL installer lib vmware-install.plrootrac1 vmware-tools-distrib# ./vmware-install.pl三 添加共享磁盘E:vmware 9vmware-vdiskmanager.exe -c -s 2g -a lsilogic -t 2 D:xunijishareocr.vmdkCreating disk D:xunijishareocr.vmdk Create: 100% done.Virtual disk creation successful.E:vmware 9vmware-vdiskmanager.exe -c -s 5g -a lsilogic -t 2 D:xunijisharedata.vmdkCreating disk D:xunijisharedata.vmdk Create: 100% done.Virtual disk creation successful.E:vmware 9vmware-vdiskmanager.exe -c -s 2g -a lsilogic -t 2 D:xunijisharefra.vmdkCreating disk D:xunijisharefra.vmdk Create: 100% done.Virtual disk creation successful.注：-a 指定磁盘类型 t 2表示直接划分一个预分配空间的文件。用记事本编辑两个虚拟机的vmx文件,例如，11grac2.vmx:disk.EnableUUID = TRUE #否则UUID出不来#shared disks configure disk.locking = FALSEdiskLib.dataCacheMaxSize = 0diskLib.dataCacheMaxReadAheadSize = 0 diskLib.dataCacheMinReadAheadSize = 0diskLib.maxUnsyncedWrites = 0scsi1.present = TRUEscsi1.virtualDev = lsilogicscsil.sharedBus = VIRTUALscsi1:0.present = TRUEscsi1:0.mode = independent-persistentscsi1:0.fileName = D:xunijishareocr.vmdkscsi1:0.deviceType = diskscsi1:0.redo = scsi1:1.present = TRUEscsi1:1.mode = independent-persistentscsi1:1.fileName = D:xunijisharedata.vmdkscsi1:1.deviceType = diskscsi1:1.redo = scsi1:2.present = TRUEscsi1:2.mode = independent-persistentscsi1:2.fileName = D:xunijisharefra.vmdkscsi1:2.deviceType = diskscsi1:2.redo = 重启vmware软件,查看虚拟机配置，确认共享磁盘文件加载成功：四 配置网卡Rac1：1、rootrac1 network-scripts# cat ifcfg-eth0 DEVICE=eth0NM_CONTROLLED=yesONBOOT=noHWADDR=00:0C:29:5D:1F:AATYPE=EthernetBOOTPROTO=noneIPADDR=85PREFIX=16GATEWAY=54DEFROUTE=yesIPV4_FAILURE_FATAL=yesIPV6INIT=noNAME=System eth0UUID=5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e032、增加一个host only的网卡用于私有网络rootrac1 network-scripts# cat ifcfg-Auto_eth1 TYPE=EthernetBOOTPROTO=noneIPADDR=28PREFIX=24GATEWAY=DEFROUTE=yesIPV4_FAILURE_FATAL=yesIPV6INIT=noNAME=Auto eth1UUID=6b905981-d8c3-46a4-b9ea-5a48fa88b522ONBOOT=yesHWADDR=00:0C:29:5D:1F:B4*本例中二节点是通过复制一节点的虚拟机文件创建的，在rac2启动后需要修改2个网卡的ip*启动后发现eth0和eth1变成了eth2、eth3，删除/etc/udev/ rules.d/70-persistent-net.rules,然后重启rac2，可以解决此问题。第二章 安装grid一 检查包*本例中二节点是通过复制一节点的虚拟机文件创建的，因此检查系统包的过程在二节点中可以不用再执行。binutils-.2-5.11.el6 (x86_64)compat-libcap1-1.10-1 (x86_64)compat-libstdc+-33-3.2.3-69.el6 (x86_64)compat-libstdc+-33-3.2.3-69.el6.i686gcc-4.4.4-13.el6 (x86_64)gcc-c+-4.4.4-13.el6 (x86_64)glibc-2.12-1.7.el6 (i686)glibc-2.12-1.7.el6 (x86_64)glibc-devel-2.12-1.7.el6 (x86_64)glibc-devel-2.12-1.7.el6.i686kshlibgcc-4.4.4-13.el6 (i686)libgcc-4.4.4-13.el6 (x86_64)libstdc+-4.4.4-13.el6 (x86_64)libstdc+-4.4.4-13.el6.i686libstdc+-devel-4.4.4-13.el6 (x86_64)libstdc+-devel-4.4.4-13.el6.i686libaio-0.3.107-10.el6 (x86_64)libaio-0.3.107-10.el6.i686libaio-devel-0.3.107-10.el6 (x86_64)libaio-devel-0.3.107-10.el6.i686make-3.81-19.el6sysstat-9.0.4-11.el6 (x86_64)rpm -qa binutils compat-libcap1 compat-libstdc+-33 gcc gcc-c+ glibc glibc-devel ksh gcc-4.4.6-3.el6.x86_64gcc-c+-4.4.6-3.el6.x86_64glibc-devel-2.12-1.47.el6.x86_64glibc-2.12-1.47.el6.i686glibc-2.12-1.47.el6.x86_64binutils-.2-5.28.el6.x86_64compat-libcap1-1.10-1.x86_64rpm -qa libgcc libstdc+ libstdc+-devel libaio libaio-devel make sysstatlibgcc-4.4.6-3.el6.i686sysstat-9.0.4-18.el6.x86_64make-3.81-19.el6.x86_64libgcc-4.4.6-3.el6.x86_64libstdc+-4.4.6-3.el6.x86_64libstdc+-devel-4.4.6-3.el6.x86_64libaio-0.3.107-10.el6.x86_64rootrac1 Packages# rpm -ivh compat-libstdc+-33-3.2.3-69.el6.i686.rpm Preparing. # 100% 1:compat-libstdc+-33 # 100%rootrac1 Packages# rpm -ivh compat-libstdc+-33-3.2.3-69.el6.x86_64.rpm Preparing. # 100% package compat-libstdc+-33-3.2.3-69.el6.x86_64 is already installedrootrac1 Packages# rpm -ivh glibc-devel-2.12-1.47.el6.i686.rpm Preparing. # 100% 1:glibc-devel # 100% rootrac1 Packages# rpm -ivh ksh-20100621-12.el6.x86_64.rpm Preparing. # 100% 1:ksh # 100%rootrac1 Packages# rpm -ivh libstdc+-4.4.6-3.el6.i686.rpm Preparing. # 100% 1:libstdc+ # 100%rootrac1 Packages# rpm -ivh libstdc+-4.4.6-3.el6.x86_64.rpm Preparing. # 100% package libstdc+-4.4.6-3.el6.x86_64 is already installedrootrac1 Packages# rpm -ivh libstdc+-devel-4.4.6-3.el6.i686.rpm Preparing. # 100% 1:libstdc+-devel # 100%rootrac1 Packages# rpm -ivh libstdc+-devel-4.4.6-3.el6.x86_64.rpm Preparing. # 100% package libstdc+-devel-4.4.6-3.el6.x86_64 is already installed rootrac1 Packages# rpm -ivh libaio-0.3.107-10.el6.i686.rpm Preparing. # 100% 1:libaio # 100%rootrac1 Packages# rpm -ivh libaio-0.3.107-10.el6.x86_64.rpm Preparing. # 100% package libaio-0.3.107-10.el6.x86_64 is already installedrootrac1 Packages# rootrac1 Packages# rpm -ivh libaio-devel-0.3.107-10.el6.i686.rpm Preparing. # 100% 1:libaio-devel # 100%rootrac1 Packages# rpm -ivh libaio-devel-0.3.107-10.el6.x86_64.rpm Preparing. # 100% 1:libaio-devel # 100%二 创建用户和目录*本例中二节点是通过复制一节点的虚拟机文件创建的，因此创建用户和目录的过程在二节点中可以不用再执行，但是需要修改用户的环境变量。* 用户等效性（ssh）在11.2的安装过程中由oracle完成，当然如果安装前手工配置也可以。groupadd -g 501 oinstallgroupadd -g 502 dbagroupadd -g 504 asmadmingroupadd -g 505 asmdbauseradd -g oinstall -u 201 -G asmadmin,asmdba griduseradd -g oinstall -u 202 -G dba,asmdba oraclepasswd gridpasswd oraclemkdir -p /u01/app/gridmkdir -p /u01/app/gridbasemkdir -p /u01/app/oraclechown -R grid:oinstall /u01chown oracle:oinstall /u01/app/oraclechmod -R 775 /u01grid环境变量：export ORACLE_SID=+ASM1export ORACLE_BASE=/u01/app/gridbaseexport ORACLE_HOME=/u01/app/gridexport PATH=$ORACLE_HOME/bin:$PATHexport NLS_LANG=AMERICAN_AMERICA.ZHS16GBKexport DISPLAY=84:0.0umask 022oracle环境变量：export ORACLE_SID=xue1export ORACLE_BASE=/u01/app/oracleexport ORACLE_HOME=$ORACLE_BASE/product//db_1export PATH=$ORACLE_HOME/bin:$PATHexport LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/libexport NLS_LANG=AMERICAN_AMERICA.ZHS16GBKexport DISPLAY=84:0.0umask 022三 检查内核参数等*本例中二节点是通过复制一节点的虚拟机文件创建的，因此检查内核参数等过程在二节点中可以不用再执行。1、vi /etc/sysctl.conffs.aio-max-nr = 1048576fs.file-max = 6815744kernel.shmall = 2097152kernel.shmmax = 536870912kernel.shmmni = 4096kernel.sem = 250 32000 100 128net.ipv4.ip_local_port_range = 9000 65500net.core.rmem_default = 262144net.core.rmem_max = 4194304net.core.wmem_default = 262144net.core.wmem_max = 1048576sysctl p2、vi /etc/security/limits.conforacle soft nproc 2047oracle hard nproc 16384oracle soft nofile 1024oracle hard nofile 65536oracle soft stack 10240oracle hard stack 32768grid soft nproc 2047grid hard nproc 16384grid soft nofile 1024grid hard nofile 65536grid soft stack 10240grid hard stack 32768四 禁用NTP*本例中二节点是通过复制一节点的虚拟机文件创建的，因此禁用NTP的过程在二节点中可以不用再执行。rootrac1 u01# service ntpd statusntpd is stoppedrootrac1 u01# chkconfig ntpd offrootrac1 u01# mv /etc/ntp.conf /etc/五 配置/etc/hosts*本例中二节点是通过复制一节点的虚拟机文件创建的，因此禁用NTP的过程在二节点中可以不用再执行。#public:85 rac186 rac2#vip:87 rac1-vip88 rac2-vip#priv28 rac1-priv29 rac2-priv#SCAN00 rac-scan六 Asm磁盘准备1、在rac1进行分区，每个磁盘都使用全部空间分配一个主分区：rootrac1 # fdisk l。Disk /dev/sdc: 2147 MB, 2147483648 bytes255 heads, 63 sectors/track, 261 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x00000000Disk /dev/sdc doesnt contain a valid partition tableDisk /dev/sde: 2147 MB, 2147483648 bytes255 heads, 63 sectors/track, 261 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x00000000Disk /dev/sde doesnt contain a valid partition tableDisk /dev/sdd: 5368 MB, 5368709120 bytes255 heads, 63 sectors/track, 652 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x00000000Disk /dev/sdd doesnt contain a valid partition table分区后：Disk /dev/sdc: 2147 MB, 2147483648 bytes255 heads, 63 sectors/track, 261 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0xbf6914ac Device Boot Start End Blocks Id System/dev/sdc1 1 261 2096451 83 LinuxDisk /dev/sde: 2147 MB, 2147483648 bytes255 heads, 63 sectors/track, 261 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x63ae23b2 Device Boot Start End Blocks Id System/dev/sde1 1 261 2096451 83 LinuxDisk /dev/sdd: 5368 MB, 5368709120 bytes255 heads, 63 sectors/track, 652 cylindersUnits = cylinders of 16065 * 512 = 8225280 bytesSector size (logical/physical): 512 bytes / 512 bytesI/O size (minimum/optimal): 512 bytes / 512 bytesDisk identifier: 0x09ca7da2 Device Boot Start End Blocks Id System/dev/sdd1 1 652 5237158+ 83 Linux2、在rac2上使用fdisk l可以查看到与rac1一样的分区情况后，再继续。3、对比两个节点的共享磁盘的UID，确定两边完全一致：rootrac1 # /sbin/scsi_id -g -u /dev/sdc36000c2985a40d9d6f071e60f4bd2097brootrac1 # /sbin/scsi_id -g -u /dev/sdc136000c2985a40d9d6f071e60f4bd2097b rootrac1 # /sbin/scsi_id -g -u /dev/sdd36000c2978d043ca2f8889e948c185df5rootrac1 # /sbin/scsi_id -g -u /dev/sde36000c2920fe493134e24de56d184317crootrac2 # scsi_id -g -u /dev/sdc36000c2985a40d9d6f071e60f4bd2097b rootrac2 # scsi_id -g -u /dev/sdd36000c2978d043ca2f8889e948c185df5rootrac2 # scsi_id -g -u /dev/sde36000c2920fe493134e24de56d184317c4、udev绑定磁盘for i in c d e ;doecho KERNEL=sd*, BUS=scsi, PROGRAM=/sbin/scsi_id -whitelisted -replace-whitespace -device=/dev/$name, RESULT=/sbin/scsi_id -whitelisted -replace-whitespace -device=/dev/sd$i, NAME=asm-disk$i, OWNER=grid, GROUP=asmadmin, MODE=0660 /etc/udev/rules.d/99-oracle-asmdisk.rulesdonerootrac2 # cat /etc/udev/rules.d/99-oracle-asmdisk.rulesKERNEL=sd*, BUS=scsi, PROGRAM=/sbin/scsi_id -whitelisted -replace-whitespace -device=/dev/$name, RESULT=36000c2985a40d9d6f071e60f4bd2097b, NAME=asm-diskc, OWNER=grid, GROUP=asmadmin, MODE=0660KERNEL=sd*, BUS=scsi, PROGRAM=/sbin/scsi_id -whitelisted -replace-whitespace -device=/dev/$name, RESULT=36000c2978d043ca2f8889e948c185df5, NAME=asm-diskd, OWNER=grid, GROUP=asmadmin, MODE=0660KERNEL=sd*, BUS=scsi, PROGRAM=/sbin/scsi_id -whitelisted -replace-whitespace -device=/dev/$name, RESULT=36000c2920fe493134e24de56d184317c, NAME=asm-diske, OWNER=grid, GROUP=asmadmin, MODE=0660七 安装cvuqdisk包上传介质：rootrac1 oracle# ls -rtltotal 3664216-rw-r-r- 1 oracle oinstall 1395582860 Feb 17 13:12 p13390677_112040_Linux-x86-64_1of7.zip-rw-r-r- 1 oracle oinstall 1151304589 Feb 17 13:14 p13390677_112040_Linux-x86-64_2of7.zip-rw-r-r- 1 oracle oinstall 1205251894 Feb 17 13:16 p13390677_112040_Linux-x86-64_3of7.ziprootrac1 oracle# chown grid:oinstall p13390677_112040_Linux-x86-64_3of7.ziprootrac1 oracle# mv p13390677_112040_Linux-x86-64_3of7.zip /home/gridrootrac1 oracle# su gridgridrac1 $ unzip *.zip在两个节点安装cvuqdisk-1.0.9-1.rpm：rootrac1 rpm# pwd/home/grid/grid/rpmrootrac1 rpm# export CVUQDISK_GRP=oinstallrootrac1 rpm# rpm -ivh cvuqdisk-1.0.9-1.rpm Preparing. # 100% 1:cvuqdisk # 100%八 启动grid安装1、用户等效性Rac1:ssh-keygen -t rsa ssh-keygen -t dsaRac2:ssh-keygen -t rsa ssh-keygen -t dsa #以上用默认配置,一路回车即可rac1: cat /.ssh/*.pub /.ssh/authorized_keys ssh gridrac2 cat /.ssh/*.pub /.ssh/authorized_keys rac2: cat /.ssh/*.pub /.ssh/authorized_keys ssh gridrac1 cat /.ssh/*.pub /.ssh/authorized_keys # rac1,rac2双节点执行ssh rac1-priv date ,ssh rac2-priv dategridrac1 .ssh$ ssh rac1-priv dateThe authenticity of host rac1-priv (28) cant be established.RSA key fingerprint is 2a:61:34:b9:39:4a:1a:1d:a9:11:fe:27:7d:17:17:08.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added rac1-priv,28 (RSA) to the list of known hosts.Mon Feb 17 14:51:03 CST 2014gridrac1 .ssh$ ssh rac1-priv dateMon Feb 17 14:51:05 CST 2014gridrac1 .ssh$ ssh rac2-priv dateThe authenticity of host rac2-priv (29) cant be established.RSA key fingerprint is 2a:61:34:b9:39:4a:1a:1d:a9:11:fe:27:7d:17:17:08.Are you sure you want to continue connecting (yes/no)? yesWarning: Permanently added rac2-priv,29 (RSA) to the list of known hosts.Mon Feb 17 14:51:21 CST 2014gridrac1 .ssh$ ssh rac2-priv dateMon Feb 17 14:51:23 CST 20142、环境检查gridrac1 grid$ ./runcluvfy.sh stage -post hwos -n rac1,rac2 -verbose3、启动安装./runI*注：add 增加rac2注:test ssh注：第一个包补装，第二个包忽略；Task resolv.conf检查失败是因为没配置DNS，忽略。注：PRVF-9802，无法验证共享磁盘，只要确定共享磁盘没问题，该问题可忽略注：先在rac1运行，成功后在rac2运行rootrac1 Packages# /u01/app/grid/root.shPerforming root user operation for Oracle 11g The following environment variables are set as: ORACLE_OWNER= grid ORACLE_HOME= /u01/app/gridEnter the full pathname of the local bin directory: /usr/local/bin: Copying dbhome to /usr/local/bin . Copying oraenv to /usr/local/bin . Copying coraenv to /usr/local/bin .Creating /etc/oratab file.Entries will be added to the /etc/oratab file as needed byDatabase Configuration Assistant when a database is createdFinished running generic part of root script.Now product-specific root actions will be performed.Using configuration parameter file: /u01/app/grid/crs/install/crsconfig_paramsCreating trace directoryUser ignored Prerequisites during installationInstalling Trace File AnalyzerOLR initialization - successful root wallet root wallet cert root cert export peer wallet profile reader wallet pa wallet peer wallet keys pa wallet keys peer cert request pa cert request peer cert pa cert peer root cert TP profile reader root cert TP pa root cert TP peer pa cert TP pa peer cert TP profile reader pa cert TP profile reader peer cert TP peer user cert pa user certAdding Clusterware entries to upstartCRS-2672: Attempting to start ora.mdnsd on rac1CRS-2676: Start of ora.mdnsd on rac1 succeededCRS-2672: Attempting to start ora.gpnpd on rac1CRS-2676: Start of ora.gpnpd on rac1 succeededCRS-2672: Attempting to start ora.cssdmonitor on rac1CRS-2672: Attempting to start ora.gipcd on rac1CRS-2676: Start of ora.cssdmonitor on rac1 succeededCRS-2676: Start of ora.gipcd on rac1 succeededCRS-2672: Attempting to start ora.cssd on rac1CRS-2672: Attempting to start ora.diskmon on rac1CRS-2676: Start of ora.dis