Showing posts with label Oracle. Show all posts
Showing posts with label Oracle. Show all posts

Wednesday, September 3, 2025

19c datapatch failed with error "ORA-04088: error during execution of trigger 'WMSYS.NO_VM_DDL'"

When running datapatch against Oracle 19c database, it failes with error "ORA-04088: error during execution of trigger 'WMSYS.NO_VM_DDL'".

The datapatch output is something like following,
Patch 37960098 apply (pdb PDB01): WITH ERRORS
  logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/37960098/27641878/37960098_apply_CDB01_PDB01_2025Aug20_15_44_17.log (errors)
  -> Error at line 140412: script rdbms/admin/owmcvws.plb
      - ORA-04088: error during execution of trigger 'WMSYS.NO_VM_DDL'
      - ORA-00604: error occurred at recursive SQL level 2
      - ORA-04061: existing state of  has been invalidated
      - ORA-04061: existing state of package body "WMSYS.LT_CTX_PKG" has been
      - invalidated
      - ORA-04065: not executed, altered or dropped package body "WMSYS.LT_CTX_PKG"
      - ORA-06508: PL/SQL: could not find program unit being called: "WMSYS.LT_CTX_PKG"
      - ORA-06512: at "WMSYS.LTADM", line 9800
      - ORA-04061: existing state of package body "WMSYS.LT_CTX_PKG" has been
      - invalidated
      - ORA-04065: not executed, altered or dropped package body "WMSYS.LT_CTX_PKG"
      - ORA-06508: PL/SQL: could not find program unit being called: "WMSYS.LT_CTX_PKG"
      - ORA-06512: at "WMSYS.LTADM", line 9532
      - ORA-06512: at "WMSYS.OWM_DYNSQL_ACCESS", line 549
      - ORA-06512: at line 17
      - ORA-06512: at line 7
      - ORA-06512: at line 7
  -> Error at line 142841: script rdbms/admin/owmcvws.plb
      - ORA-04088: error during execution of trigger 'WMSYS.NO_VM_DDL'
      - ORA-00604: error occurred at recursive SQL level 1
      - ORA-04068: existing state of packages has been discarded
      - ORA-04061: existing state of package body "WMSYS.LTUTIL" has been invalidated
      - ORA-04065: not executed, altered or dropped package body "WMSYS.LTUTIL"
      - ORA-06508: PL/SQL: could not find program unit being called: "WMSYS.LTUTIL"
      - ORA-06512: at "WMSYS.LTADM", line 9437
      - ORA-06512: at "WMSYS.OWM_DYNSQL_ACCESS", line 544
      - ORA-06512: at line 15
Solution

Disable the trigger which caused the issues, then re-run datapatch.

Friday, May 16, 2025

PRCH-1000 : Failed to stop resources running from Oracle home After out-of-place patching

Oracle databases were running from old home '/u01/app/oracle/product/19.0.0/dbhome_1', new home '/u01/app/oracle/product/19.0.0/dbhome_2' is created by excuting 'opatchauto -out-of-place' from old home. 
After pathing, all databases are swithed to new home. After old home was deinstalled successfully, 'srvctl stop home' failed with following errors,

  PRCH-1059 : Failed while checking status of Oracle home

And 'srvctl status home' also failed with following errors,

  PRCH-1059 : Failed while checking status of Oracle home

Tuesday, December 17, 2024

APEX 404 Not Found after ORDS upgrade to 23.1 or later when SYS shows NO in column COMMON of DBA_USERS in non-cdb

When upgrading Oracle REST Data Services (ORDS) to version 23.1 or later, it completes with warnings as following
WARNING: Procedure APEX_ADMIN could not be added to allow list.
 The function or procedure is a common object.
 You must execute the following as a common user: 
 begin APEX_240100.wwv_flow_listener.sync_ords_gateway_allow_list; end;
WARNING: ORDS migrated 0 entry points from APEX to ORDS PL/SQL
 Procedure Gateway Allow List.
Now accessing APEX will return 404 error as following
                   Not Found
               HTTP Status Code: 404
       Request ID: EHFC-h-DNoA9A0Qo-_W
     Request Timestamp: 2024-12-17T13:20:38.684973Z
               Error Code: ORDS-2201
The procedure named apex could not be accessed, it may not be declared, or the user 
executing this request may not have been granted execute privilege on the procedure, or a
function specified by security.requestValidationFunction configuration property has prevented
access. Check the spelling of the procedure, check that the execute privilege has been
granted to the caller and check the configured security.requestValidationFunction function. If
using the PL/SQL Gateway Procedure Allow List, check that the procedure has been allowed
via ords_admin.add_plsql_gateway_procedure.

CAUSE

Since pluggable database was introduced in Oracle database from 12.1, Oracle supplied users (such as SYS, SYSTEM, DBSNMP, etc) are set to YES for new colume COMMON (added from 12.1) of view DBA_USERS no matter whether the database is CDB or non-CDB.

However, if the database was upgraded from 11g or earlier to 12.1, the upgrade may not set COMMON properly because of bug

  25117045 - 12C UPGRADE DOES NOT SET THE COMMON-USER FLAG FOR ORACLE-SUPPLIED USERS

For example, I have a database which upgrade history as following
SQL> select version,comments from dba_registry_history where action='UPGRADE' order by action_time;

VERSION       COMMENTS
_____________ _________________________________________
10.2.0.3.0    Upgraded from 9.2.0.6.0
10.2.0.4.0    Upgraded from 10.2.0.3.0
10.2.0.5.0    Upgraded from 10.2.0.4.0
11.2.0.3.0    Upgraded from 10.2.0.5.0
12.1.0.2.0    Upgraded from 11.2.0.3.0
12.2.0.1.0    Upgraded from 12.1.0.2.0
12.2.0.1.0    Upgraded from 12.1.0.2.0
19.0.0.0.0    Upgraded from 12.2.0.1.0 to 19.7.0.0.0
The COMMON of Oracle supplied users as following
SQL> select username,common from dba_users where oracle_maintained='Y';

USERNAME      COMMON
_____________ _________
SYSTEM                    NO
SYS                       NO
SYSDG                     YES
SYSKM                     YES
SYSBACKUP                 YES
SYSRAC                    YES
DBSNMP                    NO
    ... ...
FLOWS_FILES               YES
APEX_PUBLIC_ROUTER        YES
APEX_240100               YES
It shows that, SYS is not common user, but APEX_240100 is common user. The procedure APEX_ADMIN is owned by COMMON user APEX_240100, which cannot be processed by local (non-common) user SYS. Therefore, ORDS upgrade returned the warnings.

WORKAROUND

The APEX 404 error can be fixed as a workaround by changing the pool.xml of ORDS databse pool.

Remove following line from the file

<entry key="security.requestValidationFunction">ords_util.authorize_plsql_gateway</entry>

Add following ling to the file

<entry key="security.requestValidationFunction">wwv_flow_epg_include_modules.authorize</entry>

Restart ORDS application.

SOLUTION

Since the root cause is the COMMON of SYS user, we can update the COMMON to YES for Oracle supplied user (including SYS, SYSTEM). Then re-run the ORDS upgrade.

To update COMMON, following the steps,

1. Stop applications which are accessing database

If this is a RAC database, shutdown all instances except one. The remaining steps will be performed on the running instance. Ensure that no application  sessions are connected. Optionally, put the system into restricted session mode so that non-DBA connections cannot be made (alter system enable restricted session).

2. Backup the database

Make a backup of the database, or create a backup of table user$ with CREATE TABLE SELECT FROM 
connect / as sysdba
create table sys.backup_of_user$
   as select * from sys.user$;
create table sys.backup_of_user$_updated
   as select * from sys.user$ where bitand(spare1, 256) = 256;
3. Update the data dictionary using following commands
connect / as sysdba
update user$
 set spare1 = spare1 - bitand(spare1, 128) + 128
 where bitand(spare1, 256) = 256;
commit;
4. Ensure that stale user$ information is removed from the cache

If possible, shutdown and restart the instance. If that is not possible, try to flush the shared pool three times with commands
alter system flush shared_pool;
If RAC, start all other instances. If the system was previously put into restricted session mode, that can now be disabled (alter system disable restricted session).

5. Verify that the users (SYS, SYSTEM, etc) are now common users
SQL> select username,common from dba_users where oracle_maintained='Y';

USERNAME      COMMON
_____________ _________
SYSTEM                    YES
SYS                       YES
SYSDG                     YES
SYSKM                     YES
SYSBACKUP                 YES
SYSRAC                    YES
DBSNMP                    YES
    ... ...
FLOWS_FILES               YES
APEX_PUBLIC_ROUTER        YES
APEX_240100               YES

Monday, November 25, 2024

Script - List blocking session tree in Oracle single instance or RAC database

 File Name
  sess_blocking_tree.sql

Description
  Displays Oracle databsae session blocking tree.
  It support both single instance and RAC

Requirements
 User needs SELECT privilege on following views
    gv$instance
    gv$session
Usage
  Run with SQL*Plus or SQLcl
  @sess_blocking_tree

-- -----------------------------------------------------------------------------------
-- File Name    : sess_blocking_tree.sql
-- Author       : https://www.dbaplus.ca
-- Description  : Displays session blocking heirarchy
-- Requirements : Access to the following views
--                gv$instance
--                gv$session
-- Usage        : @sess_blocking_tree
-- Last Modified: 25-Nov-2024
-- -----------------------------------------------------------------------------------
set termout off
-- ----------------------------------------------------
-- Save current SET variable
-- ----------------------------------------------------
column v_set_save_file new_value V_SET_SAVE_FILE
select 'tmp_SQLPlus_ENV_&_USER' v_set_save_file from dual;
STORE SET &V_SET_SAVE_FILE replace
-- ----------------------------------------------------
-- Save current NLS_DATE_FORMAT
-- ----------------------------------------------------
column v_nls_date_format new_value V_NLS_DATE_FORMAT
select sys_context('USERENV','NLS_DATE_FORMAT') v_nls_date_format from dual;
-- ------------------------------------------------------------------------
-- Show INST_ID column when RAC database has more than one instance running
-- ------------------------------------------------------------------------
column v_inst_id new_value V_INST_ID
select decode(count(*),1,'','INST_ID,') v_inst_id  from gv$instance;
-- ------------------------------------------------------------------------
-- SET env variable
-- ------------------------------------------------------------------------
set linesize 512
set pagesize 999
set verify off
column username format a30
column osuser format a10
column machine format a25
alter session set nls_date_format='YYYY-MM-DD HH24:MI:SS';
set termout on
----------------------------------------------------------------------------- 
SELECT level,
       LPAD(' ', (level-1)*2, ' ') || NVL(s.username, '(oracle)') AS username,
       s.osuser, &V_INST_ID
       s.sid,
       s.serial#,
       s.lockwait,
       s.status,
       s.module,
       s.machine,
       s.program,
       s.logon_Time
from   gv$session s
where  level > 1
       or exists (select 1
               from   gv$session
               where  blocking_session = s.sid and blocking_instance=s.inst_id)
connect by prior s.sid = s.blocking_session
           and prior s.inst_id=s.blocking_instance
start with s.blocking_session is null;
---------------------------------------------
-- Restore SET variable and NLS_DATE_FORMAT
---------------------------------------------
set termout off
start &V_SET_SAVE_FILE
alter session set nls_date_format='&V_NLS_DATE_FORMAT';
column v_set_save_file clear
column v_nls_date_format clear
column v_inst_id clear
set termout on

Wednesday, April 10, 2024

Oracle Database uses Net Directory Naming with Microsoft Active Directory

 The demonstration is tested on following environment,
Windows Domain controller: DC01.lab.dbaplus.ca - Windows 2022
Workstation 1:  wkstn01 – Windows 11 + Oracle Client 19.3+ RU 19.22
Workstation 2:  wkstn02 – Oracle Linux 9.3 + Oracle Client 19.3

Sunday, April 7, 2024

Listener failed to start after Oracle Database out-of-place patching

 After applying patches on Oracle Database home with out-of-place patching. The listener cannot be started with error,

TNS-01201: Listener cannot find executable <ORACLE_HOME>/bin/oracle for SID <ORACLE_SID>

Friday, March 29, 2024

Install Oracle Database 19c Client on Oracle Linux 9 Subsystem on Windows 11

Beginning with 19.22 Jan2024 RU, the 19c database is certified on Oracle Linux 9.x and Red Hat Enterprise Linux (RHEL) 9.x. The minimum kernel version has to be 5.15.0 with Oracle Linux 9 and 5.14.0 with RHEL.

Therefore, I want to try 19c client on my favorite  Windows Subsystem for Linux which is running Oracle Linux 9. The installation includes following steps,

 1) Update WSL kernel version
 
    It is strongly recommended to update the kernel version to 5.15 or higher which is minimum version for Oracle Database 19c on Oracle Linux 9.

 2) Install Oracle Linux 9.1 Subsystem

    You also have other choices (e.g. Oracle Linux 9.2, Oracle Linux 9.3) if you are installing from Microsoft Store instead of command line.

 3) Configure Oracle Linux 9 for Oracle Database installation

 4) Install Oracle Database 19c (19.22) Client

Thursday, February 9, 2023

Script - Check LOBSEGMENT Storage Usage of Oracle table BLOB column stored as SECUREFILE or BASICFILE

NAME
  CkSpaceBLOB.sql

DESCRIPTION
  Check the space usage of BLOB column which is stored as LOBSEGMENT type segment  

USAGE
     CkSapceBLOB <table_owner> <table_name> <lob_column_name>

Monday, January 30, 2023

Oracle Database 12.2/18c/19c ORA-00600: [qosdExpStatRead: expcnt mismatch]

Oracle database 12.2, 18c or 19c database alert log file reports ORA-00600: [qosdExpStatRead: expcnt mismatch]. The error message is something like following
2023-01-29T15:55:03.286965-05:00
Errors in file /u01/app/oracle/diag/rdbms/cdb01/cdb01/trace/cdb01_mz00_513756.trc  (incident=97764) (PDBNAME=PDB1):
ORA-00600: internal error code, arguments: [qosdExpStatRead: expcnt mismatch], [65561], [25], [3125688], [1], [], [], [], [], [], [], []
PDB1(3):Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.
2023-01-29T15:55:03.291141-05:00
Errors in file /u01/app/oracle/diag/rdbms/cdb01/cdb01/trace/cdb01_mz00_513756.trc:
ORA-00600: internal error code, arguments: [qosdExpStatRead: expcnt mismatch], [65561], [25], [3125688], [1], [], [], [], [], [], [], []

Saturday, January 28, 2023

19c database createtion getting ORA-46385 On AUDSYS.AUD$UNIFIED

When creating Oracle 19c databases, got errors as following,

comment on table AUDSYS.AUD$UNIFIED is
                        *
ERROR at line 1:

ORA-46385: DML and DDL operations are not allowed on table

"AUDSYS"."AUD$UNIFIED".
It was reported by Oracle support on 19.5. It is still there when creating non-cdb on 19.16. 

When the Unified Auditing is enabled on Oracle database home, the database creation tries to create a comment on internal Unified Auditing table and gets this error.

It is completely harmless and can be safely ignored.

Thursday, June 16, 2022

Oracle Data Pump Export/Import SODA Collection Data

Simple Oracle Document Access (SODA ) collections can be exported and imported using Oracle Data Pump Utilities starting with version 19.6.

In Oracle database, a SODA collection is analogous to database table. A SODA collection is mapped to /backed by a table. A SODA document is analogous to, and is in fact backed by, a row of a database table. The row has one column for each document component: key, content, version, and so on.

Wednesday, May 25, 2022

ORA-02291: integrity constraint (WWV_FLOW_FND_GU_INT_G_FK) violated - parent key not found - When Importing Workspace In APEX configured with ORDS & WebLogic

When creating users during importing workspace in APEX, got ORA-02291 error like following
ORA-02291: integrity constraint (APEX_200100.WWV_FLOW_FND_GU_INT_G_FK) violated - parent key not found
ORA-06512: at "APEX_200100.WWV_FLOW_FND_USER_INT", line 2067
ORA-06512: at "APEX_200100.WWV_FLOW_FND_USER_API", line 339
ORA-06512: at line 2
02291. 00000 - "integrity constraint (%s.%s) violated - parent key not found"
*Cause:    A foreign key value has no matching primary key value.
*Action:   Delete the foreign key or add a matching primary key.
This error occurs because pre-created user groups (SQL Developer, etc) do not have the same IDs in two APEX instances. It was reported as bug of APEX 4.2.1 when ORDS is used and deployed with WebLogic. But it also reported on APEX 18.1 and I got it on my APEX  20.1.

Thursday, March 31, 2022

Script - Enable Customized Password Complexity in Oracle non-cdb or cdb environment

The script creates a password complexity verification function, and enable the function with database profile. It also sets password policy with profile.

Friday, March 11, 2022

Export/import APEX workspace and application with SQLcl

One of the reason why I prefer SQLcl to SQL*Plus is that SQLcl has built-in "apex" commands. APEX workspace and application can be easily export / import with SQLcl.

Monday, March 7, 2022

Database failed to start with errors ORA-01565 ORA-17503 ORA-27300 ORA-27301 ORA-27302

 When starting database, get errors ORA-01565 ORA-17503 ORA-27300 ORA-27301 ORA-27302. For example,
$ srvctl start database -db db01
PRCR-1079 : Failed to start resource ora.db01.db
CRS-5017: The resource action "ora.db01.db start" encountered the following error:
ORA-01078: failure in processing system parameters
ORA-01565: error in identifying file '+DATA/db01/parameterfile/spfiledb01.ora'
ORA-17503: ksfdopn:10 Failed to open file +DATA/db01/parameterfile/spfiledb01.ora
ORA-27300: OS system dependent operation:open failed with status: 13
ORA-27301: OS failure message: Permission denied
ORA-27302: failure occurred at: sskgmsmr_7
. For details refer to "(:CLSN00107:)" in "/u01/app/grid/diag/crs/host01/crs/trace/ohasd_oraagent_grid.trc".

CRS-2674: Start of 'ora.db01.db' on 'host01' failed

Thursday, March 3, 2022

Oracle SQLcl installation and configuration

Oracle SQL Developer Command Line (SQLcl) is a free command-line interface for Oracle Database that combines the power of SQL*Plus and SQL Developer. It allows you to interactively or batch execute SQL and PL/SQL. SQLcl provides in-line editing, statement completion, and command recall for a feature-rich experience, as well as supporting your previously written SQL*Plus scripts.

Monday, February 21, 2022

Oracle RAC 19c pdb open failed with ORA-12963: A read-only instance cannot be the first instance to open a database

When opening pluggable database (pdb) on one node of two-node 19c RAC, got following errors
sys@rac01> alter pluggable database pdb1 open;
alter pluggable database pdb1 open
*
ERROR at line 1:
ORA-12963: A read-only instance cannot be the first instance to open a database.

Wednesday, February 16, 2022

Wednesday, February 9, 2022

Configure Reverse Path Filter parameter "rp_filter" on Oracle Linux and Red Hat Enterprise Linux

When installing Oracle Grid Infrastructure (GI) 12c or higher version (18c, 19c, 21c), multiple network interfaces (NIC) can be used as private interconnection to implement inter-communication load balance and failover. However, if this configuration is adopted on Oracle Linux (OL) 6 or Red Hat Enterprise Linux (RHEL) 6 or higher, the GI installation prerequists checking shows error 

Wednesday, February 2, 2022

Oracle 19c GI installation failed with PRVG-13605 : Query of chrony daemon for its time sources could not be performed

The issue happens on Oracle Linux / Red Hat Enterprise Linux 7/8 on which chronyd is adopted as the network time protocol daemon instead of ntpd. When installing Oracle Grid Infrastructure 19c on these platform, the installation fails during prerequisit check, the message shows that

Verifying Network Time Protocol (NTP) ...FAILED
  Verifying chrony daemon is synchronized with at least one external time
  source ...FAILED
  rac01: PRVG-13605 : Query of chrony daemon for its time sources could not
         be performed on node "rac01".
         PRVG-2043 : Command "/usr/bin/chronyc sources " failed on node
         "rac01" and produced the following output:
         506 Cannot talk to daemon

  rac02: PRVG-13605 : Query of chrony daemon for its time sources could not
            be performed on node "rac02".
            PRVG-2043 : Command "/usr/bin/chronyc sources " failed on node
            "rac02" and produced the following output:
            506 Cannot talk to daemon