opatch被异常中断后的处理.doc

今天在打一个patch的时候，已经是用opatch执行到了最后一个patch，一时手欠，没看清楚telnet的窗口，不小心按下了ctrl+C，于是，opatch被中断了。汗了，晚节不保啊！再次执行opatch apply，报错：$ opatch apply 6163771Invoking OPatch .6Oracle Interim Patch Installer version .6Copyright (c) 2009, Oracle Corporation. All rights reserved.Oracle Home : /oracle/app/oracle/product/10.2.0/db_1Central Inventory : /oracle/app/oracle/oraInventory from : /var/opt/oracle/oraInst.locOPatch version : .6OUI version : .0OUI location : /oracle/app/oracle/product/10.2.0/db_1/ouiLog file location : /oracle/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/opatch2009-04-08_16-29-59PM.logPatch history file: /oracle/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/opatch_history.txtApplySession applying interim patch 6163771 to OH /oracle/app/oracle/product/10.2.0/db_1Running prerequisite checks.OPatch detected non-cluster Oracle Home from the inventory and will patch the local system only.Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.(Oracle Home = /oracle/app/oracle/product/10.2.0/db_1)Is the local system ready for patching? y|nYUser Responded with: YBacking up files and inventory (not for auto-rollback) for the Oracle HomeBacking up files affected by the patch 6163771 for restore. This might take a while.Backing up files affected by the patch 6163771 for rollback. This might take a while.Patching component oracle.rdbms, .0.Updating archive file /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a with lib/libserver10.a/kfc.oUpdating archive file /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a with lib/libserver10.a/kfcb.oUpdating archive file /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a with lib/libserver10.a/kfcl.oUpdating archive file /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a with lib/libserver10.a/kfr.oUpdating archive file /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a with lib/libserver10.a/kfrb.oThe following actions have failed:Archive not applied /arch/ora_patch/patch_ia/7409356/6163771/files/lib/libserver10.a/kfc.o to /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a. Archive not applied /arch/ora_patch/patch_ia/7409356/6163771/files/lib/libserver10.a/kfcb.o to /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a. Archive not applied /arch/ora_patch/patch_ia/7409356/6163771/files/lib/libserver10.a/kfcl.o to /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a. Archive not applied /arch/ora_patch/patch_ia/7409356/6163771/files/lib/libserver10.a/kfr.o to /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a. Archive not applied /arch/ora_patch/patch_ia/7409356/6163771/files/lib/libserver10.a/kfrb.o to /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a. Do you want to proceed? y|nY User Responded with: YRunning make for target ioracleMake failed to invoke /usr/ccs/bin/make -f ins_rdbms.mk ioracle ORACLE_HOME=/oracle/app/oracle/product/10.2.0/db_1.ld: I/O error, file /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a: Fatal error.Stop.The following make actions have failed :Re-link fails on target ioracle.Do you want to proceed? y|nYUser Responded with: YApplySession adding interim patch 6163771 to inventoryVerifying the update.Inventory check OK: Patch ID 6163771 is registered in Oracle Home inventory with proper meta-data.ApplySession failed: ApplySession failed in system modification phase. Verification of patch failed: Error verification failed: ar: kfc.o not foundOPatch will attempt to restore the system.Restoring the Oracle Home.Checking if OPatch needs to invoke make to restore some binaries.Make failed to invoke /usr/ccs/bin/make -f ins_rdbms.mk ioracle ORACLE_HOME=/oracle/app/oracle/product/10.2.0/db_1.ld: I/O error, file /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a: Fatal error.Stop.-Failed to run make commands. They are stored in file /oracle/app/oracle/product/10.2.0/db_1/.patch_storage/6163771_Jun_19_2008_17_41_45/make.txtInvoke these commands manually to restore the binaries in the Oracle Home.OPatch failed to restore OH /oracle/app/oracle/product/10.2.0/db_1. Consult OPatch document to restore the home manually before proceeding.-The following warnings have occurred during OPatch execution:1) OUI-67124:Archive not applied /arch/ora_patch/patch_ia/7409356/6163771/files/lib/libserver10.a/kfc.o to /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a. Archive not applied /arch/ora_patch/patch_ia/7409356/6163771/files/lib/libserver10.a/kfcb.o to /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a. Archive not applied /arch/ora_patch/patch_ia/7409356/6163771/files/lib/libserver10.a/kfcl.o to /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a. Archive not applied /arch/ora_patch/patch_ia/7409356/6163771/files/lib/libserver10.a/kfr.o to /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a. Archive not applied /arch/ora_patch/patch_ia/7409356/6163771/files/lib/libserver10.a/kfrb.o to /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a. 2) OUI-67200:Make failed to invoke /usr/ccs/bin/make -f ins_rdbms.mk ioracle ORACLE_HOME=/oracle/app/oracle/product/10.2.0/db_1.ld: I/O error, file /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a: Fatal error.Stop.3) OUI-67124:Re-link fails on target ioracle.4) OUI-67200:Make failed to invoke /usr/ccs/bin/make -f ins_rdbms.mk ioracle ORACLE_HOME=/oracle/app/oracle/product/10.2.0/db_1.ld: I/O error, file /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a: Fatal error.Stop.-OPatch failed with error code 115$用opatch lsinventory检查，发现这个patch还是已经安装上去了的：$ opatch lsinventoryInvoking OPatch .6Oracle Interim Patch Installer version .6Copyright (c) 2009, Oracle Corporation. All rights reserved.Oracle Home : /oracle/app/oracle/product/10.2.0/db_1Central Inventory : /oracle/app/oracle/oraInventory from : /var/opt/oracle/oraInst.locOPatch version : .6OUI version : .0OUI location : /oracle/app/oracle/product/10.2.0/db_1/ouiLog file location : /oracle/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/opatch2009-04-08_17-13-49PM.logPatch history file: /oracle/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/opatch_history.txtLsinventory Output file location : /oracle/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/lsinv/lsinventory2009-04-08_17-13-49PM.txt-Installed Top-level Products (2): Oracle Database 10g .0Oracle Database 10g Release 2 Patch Set 3 .0There are 2 products installed in this Oracle Home.Interim patches (18) :Patch 6163771 : applied on Wed Apr 08 17:12:16 SGT 2009 Created on 19 Jun 2008, 17:41:45 hrs US/Pacific Bugs fixed: 6163771Patch 6052226 : applied on Wed Apr 08 17:11:53 SGT 2009 Created on 15 Sep 2008, 03:10:44 hrs PST8PDT Bugs fixed: 6052226Patch 5868257 : applied on Wed Apr 08 16:24:42 SGT 2009 Created on 21 Oct 2008, 11:41:00 hrs PST8PDT Bugs fixed: 5868257-OPatch succeeded.$ok，既然你已经在完成安装的列表中了，那我卸载可以不？$ opatch rollback -id 6163771Invoking OPatch .6Oracle Interim Patch Installer version .6Copyright (c) 2009, Oracle Corporation. All rights reserved.Oracle Home : /oracle/app/oracle/product/10.2.0/db_1Central Inventory : /oracle/app/oracle/oraInventory from : /var/opt/oracle/oraInst.locOPatch version : .6OUI version : .0OUI location : /oracle/app/oracle/product/10.2.0/db_1/ouiLog file location : /oracle/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/opatch2009-04-08_18-47-48PM.logPatch history file: /oracle/app/oracle/product/10.2.0/db_1/cfgtoollogs/opatch/opatch_history.txtRollbackSession rolling back interim patch 6163771 from OH /oracle/app/oracle/product/10.2.0/db_1Running prerequisite checks.OPatch detected non-cluster Oracle Home from the inventory and will patch the local system only.Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.(Oracle Home = /oracle/app/oracle/product/10.2.0/db_1)Is the local system ready for patching? y|nYUser Responded with: YBacking up files affected by the patch 6163771 for restore. This might take a while.Patching component oracle.rdbms, .0.Updating archive file /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a with lib/libserver10.a/kfc.oUpdating archive file /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a with lib/libserver10.a/kfcb.oUpdating archive file /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a with lib/libserver10.a/kfcl.oUpdating archive file /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a with lib/libserver10.a/kfr.oUpdating archive file /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a with lib/libserver10.a/kfrb.oThe following actions have failed:Archive not rolled back /oracle/app/oracle/product/10.2.0/db_1/.patch_storage/6163771_Jun_19_2008_17_41_45/files/lib/libserver10.a/kfc.o to /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a. Archive not rolled back /oracle/app/oracle/product/10.2.0/db_1/.patch_storage/6163771_Jun_19_2008_17_41_45/files/lib/libserver10.a/kfcb.o to /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a. Archive not rolled back /oracle/app/oracle/product/10.2.0/db_1/.patch_storage/6163771_Jun_19_2008_17_41_45/files/lib/libserver10.a/kfcl.o to /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a. Archive not rolled back /oracle/app/oracle/product/10.2.0/db_1/.patch_storage/6163771_Jun_19_2008_17_41_45/files/lib/libserver10.a/kfr.o to /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a. Archive not rolled back /oracle/app/oracle/product/10.2.0/db_1/.patch_storage/6163771_Jun_19_2008_17_41_45/files/lib/libserver10.a/kfrb.o to /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a. Do you want to proceed? y|nY User Responded with: YRunning make for target ioracleMake failed to invoke /usr/ccs/bin/make -f ins_rdbms.mk ioracle ORACLE_HOME=/oracle/app/oracle/product/10.2.0/db_1.ld: I/O error, file /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a: Fatal error.Stop.The following make actions have failed :Re-link fails on target ioracle.Do you want to proceed? y|nN User Responded with: NRollbackSession failed: RollbackSession failed in system modification phase. Re-link fails on target ioracle.OPatch will attempt to restore the system.Restoring the Oracle Home.Checking if OPatch needs to invoke make to restore some binaries.Make failed to invoke /usr/ccs/bin/make -f ins_rdbms.mk ioracle ORACLE_HOME=/oracle/app/oracle/product/10.2.0/db_1.ld: I/O error, file /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a: Fatal error.Stop.-Failed to run make commands. They are stored in file /oracle/app/oracle/product/10.2.0/db_1/.patch_storage/6163771_Jun_19_2008_17_41_45/make.txtInvoke these commands manually to restore the binaries in the Oracle Home.OPatch failed to restore OH /oracle/app/oracle/product/10.2.0/db_1. Consult OPatch document to restore the home manually before proceeding.-The following warnings have occurred during OPatch execution:1) OUI-67124:Archive not rolled back /oracle/app/oracle/product/10.2.0/db_1/.patch_storage/6163771_Jun_19_2008_17_41_45/files/lib/libserver10.a/kfc.o to /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a. Archive not rolled back /oracle/app/oracle/product/10.2.0/db_1/.patch_storage/6163771_Jun_19_2008_17_41_45/files/lib/libserver10.a/kfcb.o to /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a. Archive not rolled back /oracle/app/oracle/product/10.2.0/db_1/.patch_storage/6163771_Jun_19_2008_17_41_45/files/lib/libserver10.a/kfcl.o to /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a. Archive not rolled back /oracle/app/oracle/product/10.2.0/db_1/.patch_storage/6163771_Jun_19_2008_17_41_45/files/lib/libserver10.a/kfr.o to /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a. Archive not rolled back /oracle/app/oracle/product/10.2.0/db_1/.patch_storage/6163771_Jun_19_2008_17_41_45/files/lib/libserver10.a/kfrb.o to /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a. 2) OUI-67200:Make failed to invoke /usr/ccs/bin/make -f ins_rdbms.mk ioracle ORACLE_HOME=/oracle/app/oracle/product/10.2.0/db_1.ld: I/O error, file /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a: Fatal error.Stop.3) OUI-67124:Re-link fails on target ioracle.4) OUI-67200:Make failed to invoke /usr/ccs/bin/make -f ins_rdbms.mk ioracle ORACLE_HOME=/oracle/app/oracle/product/10.2.0/db_1.ld: I/O error, file /oracle/app/oracle/product/10.2.0/db_1/lib/libserver10.a: Fatal error.Stop.-OPatch failed with error code 115$完蛋了，无论是重新apply还是rollback，都失败了。在这个情况下，我们可以在$ORACLE_HOME找到一个隐含的文件夹，里面的rollback.sh和make.txt文件，来清楚已经安装的信息，从而让我们重新安装。因为在进行opatch的过程在，在$ORACLE_HOME下有个隐含目录.patch_storage，在这个目录下的对应的目录，$ cat rollback.sh#!/bin/sh# Copyright (c) 2009, Oracle Corporation. All rights reservedecho This script is going to rollback the changes made to system files onecho this Oracle Home only. It does not perform any of the following:echo - Inventory updateecho - Oracle binary re-linkecho - Running init/pre/post scriptsecho - Customized steps performed manually by userecho Please use this script with supervision from Oracle Technical Support.echo To rollback a patch, please use opatch rollback.echo NOTE: JDK should be present in the Oracle Home to rollback Java Archives.# Get ORACLE_HOME from environment variable ORACLE_HOMEOH=$ORACLE_HOME# Error out if OH is not setif $OH = ; then echo Oracle Home is not set. echo Please set env. variable ORACLE_HOME and try again. echo Script failed to proceed. exit 1fiecho About to modify Oracle Home( $OH )echo Do you want to proceed? Y/Nif $1 = -silent ; then response=Yelse read response;fiif $response = y | $response = Y ; then echo User responded with : Yelse echo User responded with : $response exit 0fiecho Date Time : date $OH/cfgtoollogs/opatch/opatch_history.txtecho Oracle Home : $OH $OH/cfgtoollogs/opatch/opatch_history.txtecho Command : rollback.sh $OH/cfgtoollogs/opatch/opatch_history.txtcd $OH/lib/usr/ccs/bin/ar -rc $OH/lib/libserver10.a $OH/.patch_storage/6163771_Jun_19_2008_17_41_45/files/lib/libserver10.a/kfc.ocd $OH/lib/usr/ccs/bin/ar -rc $OH/lib/libserver10.a $OH/.patch_storage/6163771_Jun_19_2008_17_41_45/files/lib/libserver10.a/kfcb.ocd $OH/lib/usr/ccs/bin/ar -rc $OH/lib/libserver10.a $OH/.patch_storage/6163771_Jun_19_2008_17_41_45/files/lib/libserver10.a/kfcl.ocd $OH/lib/usr/ccs/bin/ar -rc $OH/lib/libserver10.a $OH/.patch_storage/6163771_Jun_19_2008_17_41_45/files/lib/libserver10.a/kfr.ocd $OH/lib/usr/ccs/bin/ar -rc $OH/lib/libserver10.a $OH/.patch_storage/6163771_Jun_19_2008_17_41_45/files/lib/libserver10.a/kfrb.oecho Rollback script completed.另一个rollback.sh脚本用于回滚失败的opatch执行过程，和一个make.txt脚本用于回滚编译ioracle的过程：cd $ORACLE_HOME/rdbms/lib;/usr/ccs/bin/make -f ins_rdbms.mk ioracle ORACLE_HOME=$ORACLE_HOME其实我们从回滚脚本可以具体的看到，第一个是将一些kfc.o，kfcb.o，kfcl.o的文件从libserver10.a中抽取出类备份且还原出来。也就是说，在执行opatch的过程是将kfc.o，kfcb.o，kfcl.o等文件写入到libserver10.a文件中的。第二步，重新编译ioracle。因此，我们可以利用这2个脚本，和利用原来在opatch的时候，做的备份（注意，这里一定要用原来的备份，一定要做原来的kfc.o，kfcb.o，kfcl.o的文件，在后续的操作步骤中，请注意这点。），我们来进行还原操作。操作步骤为：情况1，如果你的没有再次执行opatch脚本，即在.patch_storage目录下的rollbak脚本没有被覆盖，你可以：1.运行rollback脚本：% sh $ORACLE_HOME/.patch_storage/1234567/rollback_1234567 .sh注意，可以能会如下报错./rollback_5901872.sh89: System: not found. ./rollback_5901872.sh97: System: not found.这个可以不必忽略。 2.运行make脚本% sh $ORACLE_HOME/.patch_storage/1234567/1234567_make.txt注，make脚本仅在relink阶段出错的时候才需要使用。3.重命名脚本的原目录% mv $ORACLE_HOME/.patch_storage/1234567 $ORACLE_HOME/.patch_storage/1234567.orig情况2，如果你已经再次执行opatch脚本，即在.patch_storage目录下的rollbak脚本被覆盖了，你需要：1.从命名你的原路径：% mv $ORACLE_HOME/.patch_storage/1234567 $ORACLE_HOME2.重新生成rolback脚本，注意用no_inventory和on_relink参数% cd % opatch apply -no_inventory -no_relink3.将生产的rollback脚本和make脚本移动至原路径，以便利用原来的备份。% cp $ORACLE_HOME/.patch_storage/1234567/rollback_1234567.sh $ORACLE_HOME/1234567% cp $ORACLE_HOME/.patch_storage/1234567/1234567_make.txt $ORACLE_HOME/1234567 % mv $ORACLE_HOME/.patch_storage/1234567 $ORACLE_HOME/1234567.rereun% mv $ORACLE_HOME/1234567$ORACLE_HOME/.patc