Oracle databases were running from old home '/u01/app/oracle/product/19.0.0/dbhome_1', new home '/u01/app/oracle/product/19.0.0/dbhome_2' is created by excuting 'opatchauto -out-of-place' from old home.
After pathing, all databases are swithed to new home. After old home was deinstalled successfully, 'srvctl stop home' failed with following errors,
PRCH-1059 : Failed while checking status of Oracle home
PRCH-1059 : Failed while checking status of Oracle home
For example,
Enable trace for srvctl to check what is happening,
[oracle@node1]$ srvctl stop home -oraclehome $ORACLE_HOME -statefile stophome.txt -node node1 PRCH-1000 : Failed to stop resources running from Oracle home /u01/app/oracle/product/19.0.0/dbhome_2 /u01/app/oracle/product/19.0.0/dbhome_1/javavm/admin/classes.bin (No such file or directory) [oracle@node1]$ srvctl status home -oraclehome $ORACLE_HOME -statefile statushome.txt -node node1 PRCH-1059 : Failed while checking status of Oracle home /u01/app/oracle/product/19.0.0/dbhome_2 /u01/app/oracle/product/19.0.0/dbhome_1/javavm/admin/classes.bin (No such file or directory)
export SRVM_TRACE=true export SRVCTL_TRACEFILE=/tmp/srvctl.trcRun 'srvctl stop/status home' again, and find following message from trace file '/tmp/srvctl.trc',
[main] [ 2025-05-16 16:39:55.244 EDT ] [DatabaseImpl.getInstanceStateDetails:13428] searching for instances using filter : ((NAME == ora.db01.db) AND ((STATE == ONLINE) OR (STATE == INTERMEDIATE))) ... [main] [ 2025-05-16 16:39:55.246 EDT ] [CRSNative.searchEntities:1221] found 2 entities [main] [ 2025-05-16 16:39:55.246 EDT ] [DatabaseImpl.getInstanceStateDetails:13432] online instances are : {ora.db01.db 1 1={LAST_SERVER=node1, STATE_DETAILS=Open,HOME=/u01/app/oracle/product/19.0.0/dbhome_2}, ora.db01.db 2 1={LAST_SERVER=node2, STATE_DETAILS=Open,HOME=/u01/app/oracle/product/19.0.0/dbhome_2}} [main] [ 2025-05-16 16:39:55.246 EDT ] [DatabaseImpl.getInstanceStateDetails:13435] getting details of instance ora.db01.db 1 1 ... [main] [ 2025-05-16 16:39:55.246 EDT ] [DatabaseImpl.getInstanceStateDetails:13435] getting details of instance ora.db01.db 2 1 ... [main] [ 2025-05-16 16:39:55.246 EDT ] [CRSEntity.getAttributes:435] From local cache, try to get the value for attribute: PATCH_IN_PROGRESS [main] [ 2025-05-16 16:39:55.246 EDT ] [DatabaseImpl.processDB4OJVMPatching:12947] out-of-place patching is not being performed for this database [main] [ 2025-05-16 16:39:55.246 EDT ] [CRSEntity.getAttributes:435] From local cache, try to get the value for attribute: DATABASE_TYPE [main] [ 2025-05-16 16:39:55.246 EDT ] [CRSEntity.getAttributes:435] From local cache, try to get the value for attribute: ROLE [main] [ 2025-05-16 16:39:55.247 EDT ] [CRSEntity.getAttributes:435] From local cache, try to get the value for attribute: DATABASE_TYPE [main] [ 2025-05-16 16:39:55.247 EDT ] [CRSEntity.getAttributes:435] From local cache, try to get the value for attribute: PATCH_IN_PROGRESS [main] [ 2025-05-16 16:39:55.247 EDT ] [CRSEntity.getAttributes:435] From local cache, try to get the value for attribute: ORACLE_HOME_OLD [main] [ 2025-05-16 16:39:55.247 EDT ] [DatabaseImpl.getInstanceStateDetails:13428] searching for instances using filter : ((NAME == ora.db02.db) AND ((STATE == ONLINE) OR (STATE == INTERMEDIATE))) ... [main] [ 2025-05-16 16:39:55.248 EDT ] [CRSNative.searchEntities:1221] found 0 entities [main] [ 2025-05-16 16:39:55.248 EDT ] [DatabaseImpl.getInstanceStateDetails:13432] online instances are : {} [main] [ 2025-05-16 16:39:55.248 EDT ] [CRSEntity.getAttributes:435] From local cache, try to get the value for attribute: PATCH_IN_PROGRESS [main] [ 2025-05-16 16:39:55.248 EDT ] [CRSEntity.getAttributes:435] From local cache, try to get the value for attribute: ORACLE_HOME_OLD [main] [ 2025-05-16 16:39:55.248 EDT ] [DatabaseImpl.processDB4OJVMPatching:12956] old Home is /u01/app/oracle/product/19.0.0/dbhome_1 [main] [ 2025-05-16 16:39:55.249 EDT ] [DatabaseImpl.performCheckSum:12858] exception FileNotFoundException [main] [ 2025-05-16 16:39:55.254 EDT ] [InterruptHandler.unRegisterInterruptHandler:78] UNRegistering shutdown hook..... [main] [ 2025-05-16 16:39:55.254 EDT ] [InterruptHandler.unRegisterInterruptHandler:83] UnRegistered shutdown hook..... [main] [ 2025-05-16 16:39:55.254 EDT ] [OPSCTLDriver.main:291] OPSCTL execute() failed. Unregistered OPSCTL driver's interrupt handler [main] [ 2025-05-16 16:39:55.254 EDT ] [OPSCTLDriver.main:300] exiting abnormally due to FrameworkException [main] [ 2025-05-16 16:39:55.266 EDT ] [OPSCTLDriver.main:306] PRCH-1059 : Failed while checking status of Oracle home /u01/app/oracle/product/19.0.0/dbhome_2 /u01/app/oracle/product/19.0.0/dbhome_1/javavm/admin/classes.bin (No such file or directory) oracle.ops.opsctl.StatusAction.internalExecuteHome(StatusAction.java:5229) oracle.ops.opsctl.StatusAction.executeHome(StatusAction.java:5123) oracle.ops.opsctl.Action.execute(Action.java:586) oracle.ops.opsctl.OPSCTLDriver.execute(OPSCTLDriver.java:561) oracle.ops.opsctl.OPSCTLDriver.main(OPSCTLDriver.java:280)From the trace, we can see that,
* There are two databases db01 (resource name ora.db01.db) and db02 (resource name ora.db02.db).
* srvctl searched for instances of db01 using filter: ((NAME == ora.db01.db) AND ((STATE == ONLINE) OR (STATE == INTERMEDIATE))), and found 2 entities (means two running instances). One instance is running from new home on server node1, and the other one is running on server node2.
* srvctl searched for instances of db02 using filter: ((NAME == ora.db02.db) AND ((STATE == ONLINE) OR (STATE == INTERMEDIATE))), but found 0 entities (means no running instances).
It seems that srvctl thinks 'found 0 entities' is sign of unfinished patching process and tries to process/validate OJVM patch (guessing based on keywords 'DatabaseImpl.processDB4OJVMPatching') which needs to access old home. Unfortunately, the old home has been deinstalled. Therefore, the errors occurred.
[oracle@node1]$ srvctl status database -db db02 Instance db02_1 is not running on node node1 Instance db02_2 is not running on node node2 [oracle@node1]$ $GRID_HOME/bin/crsctl status resource -w "NAME == ora.db02.db" NAME=ora.db02.db TYPE=ora.database.type TARGET=OFFLINE, OFFLINE STATE=OFFLINE, OFFLINE [oracle@node1]$ srvctl status database -db db01 Instance db01_1 is running on node node1 Instance db01_2 is running on node node2 [oracle@node1]$ $GRID_HOME/bin/crsctl status resource -w "NAME == ora.db01.db" NAME=ora.db01.db TYPE=ora.database.type TARGET=ONLINE, ONLINE STATE=ONLINE on node1, ONLINE on node2Solution:
Option 1, If db02 is not needed, the issue can be fixed by removing databse db02 from OCR with command,
crsctl modify resource ora.db02.db -attr ORACLE_HOME_OLD='' -unsupported
srvctl remove database -db db02
Option 2, If db02 is still needed, manually start it with command
srvctl start database -db db02
It will update status of OCR entities. Thereafter, srvctl will find correct instances' information and will not check back to old Oracle home.
Option 3, If, for some reason, db02 cannot be started right now, as workaround, the issue can be fixed by removing old Oracle home information from OCR with following command
[oracle@node1]$ $GRID_HOME/bin/crsctl status resource -w "NAME == ora.db02.db" -p -attr NAME,GEN_USR_ORA_INST_NAME,ORACLE_HOME_OLD NAME=ora.db02.db GEN_USR_ORA_INST_NAME=db02_1 ORACLE_HOME_OLD=/u01/app/oracle/product/19.0.0/dbhome_3 GEN_USR_ORA_INST_NAME=db02_2 ORACLE_HOME_OLD=/u01/app/oracle/product/19.0.0/dbhome_3 [oracle@node1]$ $GRID_HOME/bin/crsctl modify resource ora.db02.db -attr ORACLE_HOME_OLD='' -unsupported [oracle@node1]$ [oracle@node1]$ $GRID_HOME/bin/crsctl status resource -w "NAME == ora.db02.db" -p -attr NAME,GEN_USR_ORA_INST_NAME,ORACLE_HOME_OLD NAME=ora.db02.db GEN_USR_ORA_INST_NAME=db02_1 ORACLE_HOME_OLD= GEN_USR_ORA_INST_NAME=db02_2 ORACLE_HOME_OLD=
No comments:
Post a Comment