RAC DATABASE

RAC SETUP :

==============

192.168.1.101 - Public IP of NODE1 - (server connectivity as public)

192.168.0.101 Private IP NODE1 (node to node connectivity)

192.168.1.111 Virtual IP of NODE1(same subnet mask as Public IP,1Ping method and runs on public of each node)

192.168.1.102 - Public IP of NODE2 (server connectivity as public)

192.168.0.102 - private IP of NODE2 - (node to node connectivity)

192.168.1.112 Virtual IP of NODE2 (same subnet mask as Public IP,1Ping method and runs on public of each node)

192.168.1.121 db-scan.db.com db-scan (1static scan list)

192.168.1.122 db-scan.db.com db-scan

192.168.1.123 db-scan.db.com db-scan

We are configuring only 1static listner.becoz no DNS sever

(For 2node cluster we need total 9IPS- 2pub,2priv,2vip,3scan(dns) we are using statis scan only 1 ip)

======================================

Configure IPs

192.168.0.101 - Private IP - Make sure to turn it on automatically

192.168.1.101 - Public IP - Make sure to turn it on automatically

## as root

export PATH=$PATH:/bin:/usr/bin

vi /etc/hosts

127.0.0.1 localhost.db.com localhost

# Public

192.168.1.101 db1.db.com db1

192.168.1.102 db2.db.com db2

# Private

192.168.0.101 db1-priv.db.com db1-priv

192.168.0.102 db2-priv.db.com db2-priv

# Virtual

192.168.1.111 db1-vip.db.com db1-vip

192.168.1.112 db2-vip.db.com db2-vip

# SCAN

192.168.1.121 db-scan.db.com db-scan

192.168.1.122 db-scan.db.com db-scan

192.168.1.123 db-scan.db.com db-scan

echo "grid" | passwd --stdin grid

echo "%oinstall ALL=(ALL:ALL) NOPASSWD: ALL" >> /etc/sudoers

hostnamectl set-hostname db2.db.com

chmod 777 /u01

chmod 777 /u01/oracle/

chown oracle:oinstall /u01/app/oracle/product/19/db_2

chown oracle:oinstall /u01/app/oracle1

chown oracle:oinstall /u01/app/grid_base

chown oracle:oinstall /u01/app/19c/grid

chown oracle:oinstall /u01/app/grid_base

chown oracle:oinstall /u01/app/19c/grid

chmod 777 /u01/app/grid_base

chmod 777 /u01/app/19c/grid

chown oracle:oinstall /u01/app/oracle/product/19/db_2

chmod 777 /u01/app/oracle/product/19/db_2

----------

chown oracle:oinstall /u01/app/oracle/product/19.3/db_home1

chown oracle:oinstall /u01/app/oracle1

chown grid:oinstall /u01/app/grid_base

chown grid:oinstall /u01/app/19c/grid

/u01/app/oracle/product/19/db_2;

cd /u01/oracle; ls -l

chown oracle:asmadmin /dev/oracleasm/disks/ASMDATA1

chmod 660 /dev/oracleasm/disks/ASMDATA1

ALTER DISKGROUP ASMDATA MOUNT FORCE;

ALTER DISKGROUP ASMDATA SET ATTRIBUTE 'compatible.asm' = '21.0';

NODE 1-

## ORACLE - /home/oracle/.bashrc

PATH=$PATH:$HOME/bin ; export PATH

TMP=/tmp ; export TMP

TMPDIR=$TMP ; export TMPDIR

ORACLE_SID=ora19c1 ; export ORACLE_SID

ORACLE_UNQNAME=ora19c ; export ORACLE_UNQNAME

ORACLE_GLOBAL_NAME=ora19c1.db.com ; export ORACLE_GLOBAL_NAME

ORACLE_BASE=/u01/app/oracle ; export ORACLE_BASE

ORACLE_HOME=/u01/app/oracle/product/19/db_2; export ORACLE_HOME

ORACLE_GRID=/u01/app/19c/grid_home ; export ORACLE_GRID

ORACLE_HOSTNAME=db1.db.com; export ORACLE_HOSTNAME

PATH=/usr/sbin:$PATH ; export PATH

PATH=$ORACLE_HOME/bin:$PATH ; export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH

CLASSPATH=$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib ; export CLASSPATH

CLASSPATH=$ORACLE_HOME/JRE:$CLASSPATH ; export CLASSPATH

#export DISPLAY=192.168.1.6:7.0

---- vi grid_env

export ORACLE_HOME=/u01/app/19c/grid_home

export PATH=$ORACLE_HOME/bin:$PATH

export ORACLE_SID=+ASM1

LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH

CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPAT H

----

cat db_env

ORACLE_SID=ora19c1; export ORACLE_SID

ORACLE_HOME=/u01/app/oracle/product/19/db_2; export ORACLE_HOME

PATH=$ORACLE_HOME/bin:$BASE_PATH; export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib; export LD_LIBRARY_PATH

CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib; export CLASSPAT H

You have new mail in /var/spool/mail/oracle

---

NODE 2

## ORACLE - /home/oracle/.bashrc

PATH=$PATH:$HOME/bin ; export PATH

TMP=/tmp ; export TMP

TMPDIR=$TMP ; export TMPDIR

ORACLE_SID=ora19c2 ; export ORACLE_SID

ORACLE_UNQNAME=ora19c ; export ORACLE_UNQNAME

ORACLE_GLOBAL_NAME=ora19c2.db.com ; export ORACLE_GLOBAL_NAME

ORACLE_BASE=/u01/app/oracle1 ; export ORACLE_BASE

ORACLE_HOME=/u01/app/oracle/product/19.3/db_home1; export ORACLE_HOME

ORACLE_GRID=/u01/app/19c/grid ; export ORACLE_GRID

ORACLE_HOSTNAME=db2.db.com ; export ORACLE_HOSTNAME

PATH=/usr/sbin:$PATH ; export PATH

PATH=$ORACLE_HOME/bin:$PATH ; export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib ; export LD_LIBRARY_PATH

CLASSPATH=$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib ; export CLASSPATH

CLASSPATH=$ORACLE_HOME/JRE:$CLASSPATH ; export CLASSPATH

#export DISPLAY=192.168.1.6:7.0

------

## GRID - /home/grid/.bashrc

PATH=$PATH:$HOME/bin ; export PATH

TMP=/tmp ; export TMP

TMPDIR=$TMP ; export TMPDIR

ORACLE_SID=+ASM1 ; export ORACLE_SID

ORACLE_UNQNAME=+ASM1 ; export ORACLE_UNQNAME

ORACLE_GLOBAL_NAME=ora19c1.db.com ; export ORACLE_GLOBAL_NAME

ORACLE_BASE=/u01/app/oracle1; export ORACLE_BASE

ORACLE_HOME=/u01/app/oracle/product/19/db_2; export ORACLE_HOME

ORACLE_GRID=/u01/app/19c/grid_home ; export ORACLE_GRID

ORACLE_HOSTNAME=db1.db.com ; export ORACLE_HOSTNAME

PATH=/usr/sbin:$PATH ; export PATH

PATH=$ORACLE_HOME/bin:$PATH ; export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib ; export LD_LIBRARY_PATH

CLASSPATH=$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib ; export CLASSPATH

CLASSPATH=$ORACLE_HOME/JRE:$CLASSPATH ; export CLASSPATH

#export DISPLAY=192.168.1.6:7.0

NODE 2

## GRID - /home/grid/.bashrc

PATH=$PATH:$HOME/bin ; export PATH

TMP=/tmp ; export TMP

TMPDIR=$TMP ; export TMPDIR

ORACLE_SID=+ASM2 ; export ORACLE_SID

ORACLE_UNQNAME=+ASM2 ; export ORACLE_UNQNAME

ORACLE_GLOBAL_NAME=ora19c2.db.com ; export ORACLE_GLOBAL_NAME

ORACLE_BASE=/u01/app/oracle1; export ORACLE_BASE

ORACLE_HOME=/u01/app/oracle/product/19/db_2; export ORACLE_HOME

ORACLE_GRID=u01/app/19c/grid_home ; export ORACLE_GRID

ORACLE_HOSTNAME=db2.db.com ; export ORACLE_HOSTNAME

PATH=/usr/sbin:$PATH ; export PATH

PATH=$ORACLE_HOME/bin:$PATH ; export PATH

LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib ; export LD_LIBRARY_PATH

CLASSPATH=$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib ; export CLASSPATH

CLASSPATH=$ORACLE_HOME/JRE:$CLASSPATH ; export CLASSPATH

#export DISPLAY=192.168.1.6:7.0

-----------

Add entry in /etc/oratab

+ASM1: /u01/app/19c/grid:Y

sudo chown -R oracle:oinstall /u01/app/oraInventory

sudo chown -R oracle:oinstall /u01/app/19c/grid

sudo chmod -R 775 /u01/app/oraInventory

sudo chmod -R 775 /u01/app/19c/grid

ssh db1 'sudo chown -R oracle:oinstall /u01/app/oraInventory'

ssh db1 'sudo chown -R oracle:oinstall /u01/app/19c/grid'

sudo chmod -R 775 /u01/app/oraInventory

sudo chmod -R 775 /u01/app/19c/grid

export ORACLE_HOME=/u01/app/19c/grid

export PATH=$ORACLE_HOME/bin:$PATH

export ORACLE_SID=+ASM

Check if /dev/sdb1, /dev/sdc1, /dev/sdd1 had ownership as oracle user. If not then issue below commands as root user on both nodes

# chown oracle:oinstall /dev/nvme0n2p1

# chown oracle:oinstall /dev/nvme0n2p2

# chown oracle:oinstall /dev/nvme0n2p3

2. Also give full permissions on above disks

# chmod 777 /dev/nvme0n2p1

# chmod 777 /dev/nvme0n2p2

# chmod 777 /dev/nvme0n2p3

chown oracle:oinstall /dev/oracleasm/disks/DATA

chmod 660 /dev/oracleasm/disks/DATA

oracleasm scandisks

oracleasm listdisks

=======================================

Here's an easier-to-understand version of the Oracle RAC (Real Application Clusters) working process:

Oracle RAC Working Process:

===========================

Step 1: Starting Clusterware Services

OHAS starts the Oracle Clusterware by initiating background processes like crsd.bin for Cluster Ready Services and cssd.bin for the Cluster Synchronization Service.

CSSD checks the status of each node using heartbeats stored in the Voting Disk. It verifies connectivity and cluster membership before proceeding with node startup.

CRS is initiated, which brings up essential resources and manages cluster components.

Step 2: ASM and Storage Management Initialization :

=================================================

ASM initializes, providing a volume manager and file system for database storage. It is managed by the Oracle Clusterware and ensures consistent data access across nodes.

ASM starts its own set of background processes (ASM instance), managing disk groups and rebalancing workloads across disks to prevent bottlenecks.

Step 3: Database Instance Startup :

=================================

Each node starts its own instance of the Oracle database, connecting to the shared ASM storage. Each instance has dedicated processes for memory and process management (e.g., DBWR, LGWR).

Cache Fusion, facilitated by the interconnect, allows instances to share data in real-time, synchronizing blocks among nodes to ensure data consistency.

Step 4: SCAN and VIP Addressing :

===============================

The SCAN service provides a single point of entry for clients. It allows connections to be routed across nodes, depending on load and node availability.

VIPs facilitate quick redirection of connections in case a node goes down. If a node fails, its VIP will move to another node, minimizing disruption for the clients.

Failover and Recovery in RAC:

=============================

In the event of a node failure, Oracle Clusterware identifies the failed node and performs eviction using the Voting Disk to prevent split-brain.

CRS automatically restarts the failed instance on another node, and VIP migrates, ensuring high availability.

ASM rebalance may occur if disk groups are affected, ensuring continuous access to data without impact on database operations.

Oracle RAC's combination of Clusterware, ASM, Grid Infrastructure, and network components ensures a high-performing, resilient, and scalable database environment. By balancing workloads and providing failover capabilities, Oracle RAC supports businesses with demanding, always-available applications.

======================================

CRSCTL Commands :- Cluster control

Cluster Related Commands

crs_stat t Shows HA resource status (hard to read)

crsstat Output of crs_stat t formatted nicely

crsctl check crs CSS,CRS,EVM appears healthy

crsctl stop crs Stop crs and all other services

crsctl disable crs Prevents CRS from starting on reboot

crsctl enable crs Enables CRS start on reboot

crs_stop all Stops all registered resources

crs_start all Starts all registered resources

crsctl stop cluster -all Stops the cluster in all nodes

crsctl stop cluster -n db2 stop specific node cluster

crsctl start cluster -all Starts the cluster in all nodes

olsnodes -n check all nodes in cluster

olsnodes -l Check current node

crsctl check crs check the status of Oracle Clusterware components:

crsctl check cluster -all Check all nodes crs status

crsctl check cluster -n db2 Check specific node crs status

crsctl status resource -t -init All cluster resource status

crsctl status server -f Nodes server status

crsctl status res ora.db1.vip -p VIP status checking

-->OCR COMMANDS

./ocrcheck OCR status

./ocrdump /tmp/ocrdump.bkp Every resource of cluster stored ocrdump

./ocrconfig -showbackup auto OCR autobackup check (4backup every hr max 3bkp kept) (every 1day and every week 1 max 2)

./ocrconfig -manualbackup Manual backup of OCR

./ocrconfig -backuploc <newloc> To change the backup location (Default loc- +ASMOCR:/db-cluster/OCRBACKUP/day.ocr.259.1178436777)

./ocrconfig -restore (filename) Restore the ocr backup

cat /etc/oracle/olr.loc Default loc of OLR file

./ocrcheck -config -local To check the OLR loc

./ocrdump -local /tmp/olr.bkp To see the contents of OLR

./ocrdump -local -xml /tmp/olr.bkp To contents of OLR in xml

./ocrconfig -local -manualbackup To take manualbackup of OLR file

./crsctl query css votedisk To check votedisk status (why css bz votedisk is adminstrated by css process)

/u01/app/19c/grid/log/db1 crs alert log loc

systemctl status ohasd

systemctl enable ohasd

Wrapper scrpits to start ohasd

/etc - initab

/etc/init.d - ohasd

/etc/rc.d/init.d - ohasd

SRVCTL Commands :- server control

Database Related Commands

srvctl start instance -d <db_name> -i <inst_name> Starts an instance

srvctl start database -d <ora19c> -o nomount|mount|open Starts all database choose

srvctl stop database -d <db_name> Stops all instances, closes database

srvctl stop instance -d <db_name> -i <inst_name> Stops an instance

srvctl start service -d <db_name> -s <service_name> Starts a service

srvctl stop service -d <db_name> -s <service_name> Stops a service

srvctl status service -d <db_name> Checks status of a service

srvctl status instance -d <db_name> -i <inst_name> Checks an individual instance

srvctl status database -d <db_name> Checks status of all instances

srvctl start nodeapps -n <node_name> Starts gsd, vip, listener, and ons

srvctl stop nodeapps -n <node_name> Stops gsd, vip and listener

srvctl status scan Status of scan listener

srvctl config scan Configuration of scan listener

srvctl status asm Status of ASM instance

srvctl start asm start ASM

srvctl stop asm -f Stop ASM

ASMCMD- start and stop with asmcmd

asmcmd-startup

asmcmd-shutdown --abort

srvctl config scan_listener All scan listener status show (configured by clusterware)

srvctl config scan scan ip that we assigned in hosts

srvctl config database -d ora19c All config information will show

srvctl status asm -n db1 -a ASM status node wise

srvctl config network Network config

srvctl config vip -n db2 VIP checking on nodes

srvctl config listener Listener configure

Network IP and VIP addition in cluster :

srvctl add network -k 2 -S 192.168.1.20/255.255.255.0

srvctl add vip -n db1 -k 2 -A db1-vip2/255.255.255.0

srvctl add vip -n db2 -k 2 -A db2-vip2/255.255.255.0

crsctl stat res db1-vip2 -p

appvipcfg create

-network=1

-ip 192.168.1.66

-vipname=MyVIP

-user=root

======================================

ASM QUERY -

To check diskgroups from asm instance :

select group_number,name,state,type from v$asm_diskgroup;

To check asm instance is connected and running:

select group_number,db_name,status from v$asm_client;

To check the asm disks and their status :

select group_number,mount_status,state ,header_status from v$asm_disk;

Instances status

select INSTANCE_NUMBER,INSTANCE_NAME,HOST_NAME from gv$instance;

CREATING DISKGROUP -

EXTERNAL REDUNDANCY -

CREATE DISKGROUP DATA1 EXTERNAL REDUNDANCY DISK

'/dev/oracleasm/disks/DATA1';

NORMAL REDUNDANCY - 1main and 1copy of data

CREATE DISKGROUP DATA2 NORMAL REDUNDANCY

FAILGROUP FG1 DISK

'/dev/oracleasm/disks/D1' NAME DISK2

FAILGROUP FG2 DISK

'/dev/oracleasm/disks/D2' NAME DISK2;

HIGH REDUNDANCY : 1 Main and 2copy of data

CREATE DISKGROUP DATA3 HIGH REDUNDANCY DISK

'/dev/oracleasm/disks/D3','/dev/oracleasm/disks/D4','/dev/oracleasm/disks/D5';

Add disks into diskgroup :

ALTER DISKGROUP DG1 ADD DISK

'/dev/oracleasm/disks/D1',

'/dev/oracleasm/disks/D2';

DROP DISK - First we need to dismount the disk on all nodes and mount on 1 node then drop disk.

ALTER DISKGROUP DATA1 DISMOUNT;

DROP DISKGROUP DATA1 including contents;

ALTER DISKGROUP DATA1 DROP DISK D1;

ALTER DISKGROUP DATA1 DISK A*;

ALTER DISKGROUP DATA1 UNDROP DISKS; (We can undrop the disks using this cmd)

CHANGE REDUNDANCY NOT POSSIBLE AND CHANGE ALLOCATION UNIT NOT POSSIBLE (If we want to do then create new diskgroup move datafile to that dg and drop old DG)

Alter diskgrup -

ALTER DISKGROUP DATA1 MOUNT;

MOVE DATAFILE TO ANOTHER DISKGROUPS -

ALTER DATABASE MOVE DATAFILE '+DATA/DEVDB/DATAFILE/system.dbf' TO '+DGROUB';

compatibility check in ASM disks :

COL name FOR A10;

COL compatibility FOR A15;

SELECT name, compatibility FROM v$asm_diskgroup;

ALTER DISKGROUP ASMDATA1 SET ATTRIBUTE 'compatible.asm' = '19.0';

ASM spfile location

show parameter spfile;

/u01/app/19c/grid/dbs/spfile+ASM2.ora

Password file ASM

srvctl config asm

Password file: +OCR1/orapwASM

ASMCMD - DATAFILE location in asm

+ASMDATA1/ORA19C/DATAFILE

lsdsk -k (To check the asmdisks)

ASM DYNAMIC PERFORMANCE VIEWS-

V$ASM_ALIAS V$ASM_ATTRIBUTE V$ASM_CLIENT

V$ASM_DISK V$ASM_DISK_IOSTAT V$ASM_DISK_STAT

V$ASM_DISKGROUP V$ASM_DISKGROUP_STAT V$ASM_FILE

V$ASM_OPERATION V$ASM_TEMPLATE

=======================================

mkdir -p /u01/app/19c/grid

mkdir -p /d01/app/oracle/product/19c/db_1

chown -R oracle:oinstall /grid

chown -R oracle:oinstall /u01/software

chmod -R 775 /grid/

chmod -R 775 /u01/app/oracle1

ROOT SCRIPTS:-

/u01/app/oraInventory/orainstRoot.sh

/u01/software/root.sh

Download GRID and DB SW

unzip on the server

export CV_ASSUME_DISTID=OEL7.6

cd /u01/app/19c/grid/

./gridSetup.sh

# yum clean all

sudo chown -R oracle:oinstall /u01

sudo chmod -R 775 /u01

sudo chown -R oracle:oinstall /u01/app/19c/grid

=======================

PARTITION CRATION

[root@test ~]# fdisk /dev/nvme0n2

Welcome to fdisk (util-linux 2.32.1).

Changes will remain in memory only, until you decide to write them.

Be careful before using the write command.

Device does not contain a recognized partition table.

Created a new DOS disklabel with disk identifier 0x7955b83e.

Command (m for help): n

Partition type

p primary (0 primary, 0 extended, 4 free)

e extended (container for logical partitions)

Select (default p): p

Partition number (1-4, default 1):

First sector (2048-104857599, default 2048):

Last sector, +sectors or +size{K,M,G,T,P} (2048-104857599, default 104857599): +10G

Created a new partition 1 of type 'Linux' and of size 10 GiB.

Command (m for help): n

Partition type

p primary (1 primary, 0 extended, 3 free)

e extended (container for logical partitions)

Select (default p): p

Partition number (2-4, default 2):

First sector (20973568-104857599, default 20973568):

Last sector, +sectors or +size{K,M,G,T,P} (20973568-104857599, default 104857599): +20G

Created a new partition 2 of type 'Linux' and of size 20 GiB.

Command (m for help): n

Partition type

p primary (2 primary, 0 extended, 2 free)

e extended (container for logical partitions)

Select (default p): p

Partition number (3,4, default 3):

First sector (62916608-104857599, default 62916608):

Last sector, +sectors or +size{K,M,G,T,P} (62916608-104857599, default 104857599):

Created a new partition 3 of type 'Linux' and of size 20 GiB.

Command (m for help): p

Disk /dev/nvme0n2: 50 GiB, 53687091200 bytes, 104857600 sectors

Units: sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disklabel type: dos

Disk identifier: 0x7955b83e

Device Boot Start End Sectors Size Id Type

/dev/nvme0n2p1 2048 20973567 20971520 10G 83 Linux

/dev/nvme0n2p2 20973568 62916607 41943040 20G 83 Linux

/dev/nvme0n2p3 62916608 104857599 41940992 20G 83 Linux

Command (m for help): w

The partition table has been altered.

Calling ioctl() to re-read partition table.

Syncing disks.

[root@test ~]# fdisk -l

Disk /dev/nvme0n1: 100 GiB, 107374182400 bytes, 209715200 sectors

Units: sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disklabel type: dos

Disk identifier: 0xd3d3fe05

Device Boot Start End Sectors Size Id Type

/dev/nvme0n1p1 * 2048 616447 614400 300M 83 Linux

/dev/nvme0n1p2 616448 8892415 8275968 4G 82 Linux swap / Solaris

/dev/nvme0n1p3 8892416 209715199 200822784 95.8G 83 Linux

Disk /dev/nvme0n2: 50 GiB, 53687091200 bytes, 104857600 sectors

Units: sectors of 1 * 512 = 512 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disklabel type: dos

Disk identifier: 0x7955b83e

Device Boot Start End Sectors Size Id Type

/dev/nvme0n2p1 2048 20973567 20971520 10G 83 Linux

/dev/nvme0n2p2 20973568 62916607 41943040 20G 83 Linux

/dev/nvme0n2p3 62916608 104857599 41940992 20G 83 Linux

/dev/oracleasm/disks

sudo rpm -ivh oracleasm-support-2.1.11-2.el7.x86_64.rpm

ls /usr/sbin/oracleasm

sudo /usr/sbin/oracleasm configure -i

sudo /usr/sbin/oracleasm init

[root@RAC1 ~]# oracleasm createdisk CRS1 /dev/nvme0n2p1

Writing disk header: done

Instantiating disk: done

[root@RAC1 ~]# oracleasm createdisk DATA1 /dev/nvme0n2p2

Writing disk header: done

Instantiating disk: done

[root@RAC1 ~]# oracleasm createdisk FRA1 /dev/nvme0n2p3

Writing disk header: done

Instantiating disk: done

[root@RAC1 ~]#

oracleasm scandisks

oracleasm listdisks

sudo yum install chrony

sudo systemctl enable chronyd

sudo systemctl start chronyd

======================

ssh oracle@RAC1

source /u01/app/19c/grid/crs/install/crsenv

./cluvfyrac.sh stage -pre crsinst -n oraracn1,oraracn2 -fixup -verbose

$GRID_HOME/bin/cluvfyrac.sh stage -pre crsinst -n oraracn1,oraracn2 -fixup -verbose

Display Config and Start Grid Installation

su –

export DISPLAY=:0.0

xhost +

su – oracle

[oracle@dbwr1 ~]$ DISPLAY=dbwr1.localdomain:0.0; export DISPLAY

Install and Configure Oracle 19c Grid Infrastructure for a Cluster

[oracle@dbwr1 ~]$ cd /u01/app/19c/grid/

[oracle@dbwr1 grid]$ ./gridSetup.sh

Launching Oracle Grid Infrastructure Setup Wizard…

================================================

ssh connectivity using grid:

ssh-keygen

then copy cd .ssh/id_rsa.pub public key to home/oracle/.ssh authorized_keys all nodes

/u01/app/19c/grid/oui/prov/resources/scripts/sshUserSetup.sh -user oracle -hosts "db1.db.com db2.db.com" -advanced -noPromptPassphrase

================================================

[root@oraracn2 ~]# systemctl enable oracleasm.service

[root@oraracn2 ~]# systemctl start oracleasm.service

[root@oraracn2 ~]# systemctl enable chronyd

[root@oraracn2 ~]# systemctl restart chronyd

[root@oraracn2 ~]# chronyc -a 'burst 4/4'

200 OK

[root@oraracn2 ~]# chronyc -a makestep

200 OK

[root@oraracn2 ~]#

=========================

Cluster deconfig :-

[oracle@oraracn1 ~]$ /u01/app/19c/grid/crs/install/rootcrs.sh -force -deconfig -paramfile "/tmp/deinstall2024-08-19_12-10-14AM/response/deinstall_OraGI21Home1.rsp" -lastnode

for deconfig -

-> grid_home./crs/install/rootcrs.sh -deconfig -force

<GI_HOME>/crs/install/crsgenconfig_params

sudo /u01/app/19c/grid/crs/install/rootcrs.sh -force -deconfig -paramfile "/correct/path/to/deinstall_OraGI21Home1.rsp" -lastnode

NOde removal -

sudo /u01/app/19c/grid/crs/install/rootcrs.sh -deconfig -force -lastnode

sudo /u01/app/19c/grid/crs/install/rootcrs.sh -deconfig -force

sudo /u01/app/19c/grid/crs/install/rootcrs.sh

delete ocr.loc from /etc/oracle for node deletion

===============================

node addition steps in rac

cluvfy stage -pre nodeadd -n db1

.oraenv

ORACLE_SID =+ASM2

olsnodes -n -s

cd $ORACLE_HOME/addnode

pwd ; ls -lastnode

./addnode.sh -silent "CLUSTER_NEW_NODES=db1" "CLUSTER_NEW_VIRTUAL_HOSTNAMES=db1-vip" -ignorePrereq

As a root on new node

/u01/app/oraInventory/orainsRoot.sh

/u01/app/19c/grid/root.sh

Update Inventory on node 3

/u01/app/oracle/product/19/db_2/oui/bin/runInstaller -updateNodeList -noCluster ORACLE_HOME=/u01/app/oracle/product/19/db_2 CLUSTER_NODES=db1,db2,db3 CRS=false "INVENTORY_LOCATION=/u01/app/oraInventory" LOCAL_NODE=3

OR We can use ./gridSetup.sh

node addition then select node add it

==================================

Adding Oracle_home to new instance

cd $ORACLE_HOME/addnode

./addnode.sh -silent "CLUSTER_NEW_NODES=db1" "CLUSTER_NEW_VIRTUAL_HOSTNAMES=db1-vip" -ignorePrereq

/u01/app/oracle/product/19/db_2/root.sh

OR GUI

./dbca >>> select add instance

srvctl add instance -d DEVDB -i DEVDB3 -n db3

Deleting node from cluster

===========================

1)./dbca >> Instance management -- delete instance

srvctl status database -d DEVDB

srvctl config database -d DEVDB

srvctl remove instance -d DEVDB -i DEVDB3 -n db3

2)Disable and stop listner

srvctl disable listener -l LISTENER_DB3

srvctl stop listener -l LISTENER_DB3

3)Update Inventory on delete node

$grid_home/bin/runInstaller -updateNodeList ORACLE_HOME=/u01/app/oracle/product/19/db_2 "CLUSTER_NODES={db3}" -local

4)Deinstall oracle_home from node3

$ORACLE_HOME/deinstall -local

$ORACLE_HOME/bin/runInstaller -updateNodeList ORACLE_HOME=/u01/app/oracle/product/19/db_2 "CLUSTER_NODES={db1,db2}"

5)olsnodes -s -t

6)Disable the clusterware daemon process from node 3

root user - /u01/app/19c/grid/crs/install/root.sh -deconfig -force -verbose

7)delete clusterware configuration from other nodes

root user - /u01/app/19c/grid/bin/crsctl delete node -n db3

8)check crs status:

crsctl check crs -all

crsctl stat res -t

9)Update Inventory

cd $grid_home/bin/runInstaller -updateNodeList ORACLE_HOME=$ORACLE_HOME "CLUSTER_NODES={db3} CRS=TRUE -local

10)Deinstall GRID_HOME

cd /u01/app/19c/grid/deinstall -./deinstall -local

cd $grid_home/bin/runInstaller -updateNodeList ORACLE_HOME=/u01/app/19c/grid "CLUSTER_NODES={db1,db2}" CRS=TRUE

11)verify the integrity of the cluster after nodes removed -

cluvfy stage -post db1 -n db3

==============================

SQL> select instance_name,status from gv$instance;

INSTANCE_NAME STATUS

---------------- ------------

ora19c1 OPEN

ora19c2 OPEN

select instance_name,status from v$instance;

INSTANCE_NAME STATUS

---------------- ------------

ora19c2 OPEN

Alter system switch logfile; -->Affects only the current instance (Everything will be run in background)

Alter system archive log current; --> To force a global log switch (Stops untill the archiving has been done then it will return to sql prompt

wait for complete work then return the sql prompt )

-rwxrwxr-x 1 oracle oinstall 11904 Sep 10 01:51 cvuqdisk-1.0.10-1.rpm

[root@db1 rpm]# rpm -i cvuqdisk-1.0.10-1.rpm

package cvuqdisk-1.0.10-1.x86_64 is already installed

[root@db1 rpm]#

===========================

Linux performance scrpits -

#!/bin/bash

white="\033[1;23m"

cyan="\033[1;36m"

reset="\033[0m"

echo " "

echo -e "${white}------------------------------------------------------------------System Information--------------------------------------------------------------${reset}"

echo -e "${cyan}Hostname:${reset}\t\t"hostname -f

echo -e "${cyan}IP Address:${reset}\t\t"hostname -I

echo -e "${cyan}Uptime:${reset}\t\t"uptime | awk '{print $3,$4}' | sed 's/,//'

echo -e "${cyan}Machine Type:${reset}\t\t"vserver=$(lscpu | grep Hypervisor | wc -l); if [ $vserver -gt 0 ]; then echo "VM"; else echo "Physical"; fi

echo -e "${cyan}Product Name:${reset}\t\t"cat /sys/class/dmi/id/product_name

echo -e "${cyan}Operating System:${reset}\t\t"cat /etc/redhat-release

echo -e "${cyan}Kernel:${reset}\t\t"uname -r

echo -e "${cyan}Architecture:${reset}\t\t"arch

echo -e "${cyan}Processor Name:${reset}\t\t"awk -F':' '/^model name/ {print $2}' /proc/cpuinfo | uniq | sed -e 's/^[ \t]*//'

echo -e "${cyan}CPU Cores:${reset}\t\t"grep -c ^processor /proc/cpuinfo

echo -e "${cyan}Total RAM:${reset}\t\t"free -h | awk '/^Mem:/{print $2}'

echo -e "${cyan}Disk Usage:${reset}\t\t"

df -h | grep '^/dev/'

echo -e "${cyan}Load Average:${reset}\t\t"uptime | awk -F'load average:' '{ print $2 }'

echo " "

echo -e "${white}------------------------------------------------------------------Resource Usage--------------------------------------------------------------${reset}"

echo -e "${cyan}CPU Usage:${reset}\t\t"awk '{u=$2+$4; t=$2+$4+$5; if (NR==1){u1=u; t1=t;} else print ($2+$4-u1) * 100 / (t-t1) "%";}' <(grep 'cpu ' /proc/stat) < (sleep 1;grep 'cpu ' /proc/stat)

echo -e "${cyan}Memory Usage:${reset}\t\t"free | awk '/Mem/{printf("%.2f%"), $3/$2*100}'

echo -e "${cyan}Swap Usage:${reset}\t\t"free | awk '/Swap/{printf("%.2f%"), $3/$2*100}'

echo -e "${cyan}Top 5 Processes by CPU Usage:${reset}"

ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%cpu | head -6

echo -e "${cyan}Top 5 Processes by Memory Usage:${reset}"

ps -eo pid,ppid,cmd,%mem,%cpu --sort=-%mem | head -6

echo -e "${cyan}Disk I/O:${reset}\t\t"

iostat -x | head -10

echo -e "${cyan}Network Usage:${reset}\t\t"

netstat -i | grep -vE '^Kernel|Iface|lo'

echo -e "${cyan}Active Network Connections:${reset}"

netstat -ant | grep 'ESTABLISHED'

echo -e "${cyan}Swap Activity:${reset}"

vmstat 1 5

echo -e "${cyan}Zombie Processes:${reset}\t\t"

ps aux | awk '{ if ($8 == "Z") print $0; }'

echo -e "${cyan}System Temperature:${reset}\t\t"

sensors | grep 'Core'

echo " "

===================

CREATE PFILE = '/u01/app/init.ora' FROM SPFILE;

ALTER SYSTEM SET PGA_AGGREGATE_TARGET = 500M SCOPE=SPFILE;

ALTER SYSTEM SET SGA_TARGET = 1563M SCOPE=SPFILE;

ALTER SYSTEM SET SGA_MAX_SIZE = 2.5G SCOPE=SPFILE;

--------------------

CREATE CONTROLFILE REUSE DATABASE "ORCL" RESETLOGS ARCHIVELOG

MAXLOGFILES 16

MAXLOGMEMBERS 3

MAXDATAFILES 100

MAXINSTANCES 1

MAXLOGHISTORY 292

DATAFILE

'+asmdata/ora19c/datafile/SYSTEM.257.1180244259',

'+asmdata/ora19c/datafile/SYSAUX.258.1180244387',

'+asmdata/ora19c/datafile/UNDOTBS1.259.1180244453',

'+asmdata/ora19c/datafile/UNDOTBS2.269.1180250085',

'+asmdata/ora19c/datafile/USERS.260.1180244461'

LOGFILE

GROUP 1 ('+asmdata/ora19c/onlinelog/group_1.262.1180244835') SIZE 50M,

GROUP 2 ('+asmdata/ora19c/onlinelog/group_2.263.1180244835') SIZE 50M,

GROUP 3 ('+asmdata/ora19c/onlinelog/group_3.270.1180250547') SIZE 50M,

GROUP 4 ('+asmdata/ora19c/onlinelog/group_4.271.1180250549') SIZE 50M

CHARACTER SET AL32UTF8;

==============================

PATCHING IN RAC :

=======================

[oracle@db1 OPatch]$ export PATH=$ORACLE_HOME/OPatch:$PATH

[oracle@db1 OPatch]$ echo $ORACLE_HOME

/u01/app/19c/grid

[oracle@db1 OPatch]$ opatch lspatches

There are no Interim patches installed in this Oracle Home "/u01/app/19c/grid".

OPatch succeeded.

Critical patch update (CPU) program APR 2024 patch availability documents (db only)

Opatch latest version downloads :

[oracle@db1 OPatch]$ ./opatch version

OPatch Version: 12.2.0.1.26

OPatch succeeded.

3.3 Tools

This section contains the following:

Section 3.3.1 "Oracle OPatch"

3.3.1 Oracle OPatch

Minimum Product Requirements for Oracle OPatch

The CPU security vulnerabilities are fixed in the listed release and later releases. The Oracle OPatch downloads can be found at Patch 6880880.

Oracle database :

3.1.7.4 Oracle Database 19

Oracle GoldenGate :

3.1.9 Oracle GoldenGate

-----------------------------------

GI :

Conflict precheck :

/u01/app/19c/grid/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/patches/36352207/36115667 -oh /u01/app/19c/grid

/u01/app/19c/grid/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/patches/36352207/36352352 -oh /u01/app/19c/grid

/u01/app/19c/grid/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/patches/36352207/36360754 -oh /u01/app/19c/grid

/u01/app/19c/grid/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/patches/36352207/36360764 -oh /u01/app/19c/grid

/u01/app/19c/grid/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/patches/36352207/36360767 -oh /u01/app/19c/grid

/u01/app/19c/grid/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/patches/36352207/36460255 -oh /u01/app/19c/grid

System space precheck :

/u01/app/19c/grid/OPatch/opatch prereq CheckSystemSpace -phBaseDir /u01/patches/36352207/36115667 -oh /u01/app/19c/grid

/u01/app/19c/grid/OPatch/opatch prereq CheckSystemSpace -phBaseDir /u01/patches/36352207/36352352 -oh /u01/app/19c/grid

/u01/app/19c/grid/OPatch/opatch prereq CheckSystemSpace -phBaseDir /u01/patches/36352207/36360754 -oh /u01/app/19c/grid

/u01/app/19c/grid/OPatch/opatch prereq CheckSystemSpace -phBaseDir /u01/patches/36352207/36360764 -oh /u01/app/19c/grid

/u01/app/19c/grid/OPatch/opatch prereq CheckSystemSpace -phBaseDir /u01/patches/36352207/36360767 -oh /u01/app/19c/grid

/u01/app/19c/grid/OPatch/opatch prereq CheckSystemSpace -phBaseDir /u01/patches/36352207/36460255 -oh /u01/app/19c/grid

Apply on GI :

/u01/app/19c/grid/OPatch/opatch apply -oh /u01/app/19c/grid -local -silent /u01/patches/36352207/36115667

/u01/app/19c/grid/OPatch/opatch apply -oh /u01/app/19c/grid -local -silent /u01/patches/36352207/36352352

/u01/app/19c/grid/OPatch/opatch apply -oh /u01/app/19c/grid -local -silent /u01/patches/36352207/36360754

/u01/app/19c/grid/OPatch/opatch apply -oh /u01/app/19c/grid -local -silent /u01/patches/36352207/36360764

/u01/app/19c/grid/OPatch/opatch apply -oh /u01/app/19c/grid -local -silent /u01/patches/36352207/36360767

/u01/app/19c/grid/OPatch/opatch apply -oh /u01/app/19c/grid -local -silent /u01/patches/36352207/36460255

DB :

Conflict Precheck :

/u01/app/oracle/product/19/db_2/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/patches/36352207/36352352 -oh /u01/app/oracle/product/19/db_2

/u01/app/oracle/product/19/db_2/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/patches/36352207/36360754 -oh /u01/app/oracle/product/19/db_2

System space precheck :

/u01/app/oracle/product/19/db_2/OPatch/opatch prereq CheckSystemSpace -phBaseDir /u01/patches/36352207/36352352 -oh /u01/app/oracle/product/19/db_2

/u01/app/oracle/product/19/db_2/OPatch/opatch prereq CheckSystemSpace -phBaseDir /u01/patches/36352207/36360754 -oh /u01/app/oracle/product/19/db_2

Apply on DB :

/u01/app/oracle/product/19/db_2/OPatch/opatch apply -oh /u01/app/oracle/product/19/db_2 -local -silent /u01/patches/36352207/36352352

/u01/app/oracle/product/19/db_2/OPatch/opatch apply -oh /u01/app/oracle/product/19/db_2 -local -silent /u01/patches/36352207/36360754

ROLLBACK PATCH:

/u01/app/19c/grid/OPatch/opatch rollback -id 36115667

/u01/app/oracle/product/19/db_2/OPatch/opatch rollback -id 36352352

Manual VS OPatchauto

Verfication patch:

#/u01/app/19c/grid/OPatch/opatchauto apply /u01/patches/36352207/36115667 -analyze

(No conflict and systemspace opatchauto will take care all )

Apply patch :

#u01/app/19c/grid/OPatch/opatchauto apply /u01/patches/36352207/36115667

GI HOME PATCHING:

==================

1)stop GI home as root user

/u01/app/19c/grid/bin/crsctl stop crs

2)Unlock the GI home as root user (To change all ownership of grid_home root to oracle - oinstall)

/u01/app/19c/grid/crs/install/rootcrs.sh -prepatch

3)Grant permissions patch directory as root

chown -R oracle:oinstall /u01/patches

4)As grid user prechecks

su - oracle

cd /u01/app/19c/grid

mv OPatch OPatch_bkp

unzip p6880880_230000_Linux-x86-64.zip -d /u01/app/19c/grid/ (opatch version upgradation)

/u01/app/19c/grid/Opatch/opatch version

/u01/app/19c/grid/Opatch/opatch lsinv | grep applied

/u01/app/19c/grid/Opatch/opatch lsinv | grep desc

/u01/app/19c/grid/Opatch/opatch lspatches -oh /u01/app/19c/grid

conflict checks :

/u01/app/19c/grid/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/patches/36352207/36115667 -oh /u01/app/19c/grid

System space checks :

/u01/app/19c/grid/OPatch/opatch prereq CheckSystemSpace -phBaseDir /u01/patches/36352207/36115667 -oh /u01/app/19c/grid

5) Apply patches :

/u01/app/19c/grid/OPatch/opatch apply -oh /u01/app/19c/grid -local -silent /u01/patches/36352207/36115667

6)Post verification :

/u01/app/19c/grid/OPatch/opatch version

/u01/app/19c/grid/OPatch/opatch lsinv | grep applied

/u01/app/19c/grid/OPatch/opatch lsinv | grep desc

/u01/app/19c/grid/Oatch/opatch lspatches -oh /u01/app/19c/grid

/u01/app/19c/grid/.patch_storage/36115667_Dec_16_2023_08_49_53

7) lock and start GI as root

/u01/app/19c/grid/crs/install/rootcrs.sh -postpatch

8)Verify crs status :

crsctl check crs

============================

DB HOME PATCHING :

==========================

1)Stop DB home as root user

ORACLE_HOME=/u01/app/oracle/product/19/db_2 /u01/app/oracle/product/19/db_2 /bin/srvctl stop home -o /u01/app/oracle/product/19/db_2 -s /u01/pacthes/db.state -n db1

2)Grant permissions on patch dir as root

chown -R oracle:oinstall /u01/pacthes

3)As a oracle prechecks

su - oracle

/u01/app/oracle/product/19/db_2/Opatch/opatch version

cd /u01/app/oracle/product/19/db_2

mv OPatch OPatch_bkp

unzip p6880880_230000_Linux-x86-64.zip -d /u01/app/oracle/product/19/db_2

/u01/app/19c/grid/Opatch/opatch version

/u01/app/19c/grid/Opatch/opatch lsinv | grep applied

/u01/app/19c/grid/Opatch/opatch lsinv | grep desc

/u01/app/19c/grid/Opatch/opatch lspatches -oh /u01/app/oracle/product/19/db_2

conflict check :

/u01/app/oracle/product/19/db_2/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /u01/patches/36352207/36352352 -oh /u01/app/oracle/product/19/db_2

system space checks :

/u01/app/oracle/product/19/db_2/OPatch/opatch prereq CheckSystemSpace -phBaseDir /u01/patches/36352207/36352352 -oh /u01/app/oracle/product/19/db_2

4)Apply patch

/u01/app/oracle/product/19/db_2/OPatch/opatch apply -oh /u01/app/oracle/product/19/db_2 -local -silent /u01/patches/36352207/36352352

5)Post verification

/u01/app/19c/grid/OPatch/opatch version

/u01/app/19c/grid/OPatch/opatch lsinv | grep applied

/u01/app/19c/grid/OPatch/opatch lsinv | grep desc

/u01/app/19c/grid/OPatch/opatch lspatches -oh /u01/app/oracle/product/19/db_2

6)start ORACLE_HOME as root

ORACLE_HOME=/u01/app/oracle/product/19/db_2 /u01/app/oracle/product/19/db_2 /bin/srvctl start home -o /u01/app/oracle/product/19/db_2 -s /u01/pacthes/db.state -n db1

==============

DB PATCHING :

==============

1)Validate components and objects :

column comp_name format a40

column version format a12

column status format a15

select comp_name,version,status from dba_registry;

column owner format a15

column object_name format a40

column object_type format a20

select owner,object_name,object_type where status =" INVALID" and owner

like '%SYS%' order by object_type,owner,object_name;

fixup :

@?/rdbms/admin/utlprp.sql 16

2) Run the database patch

cd $ORACLE_HOME/opatch

./datapatch -verbose

3)Verfication postpatch steps :

set pagesize 1000;

set linesize 1000;

col status for a10;

col action_time for a10;

col description for a55;

select patch_id,status,action_time,description from dba_registry_sqlpatch;

======================================

Database Upgradation Steps For RAC 12C TO 19C

======================================

Step 1: Create a new 19c oracle database home and provide permission for both nodes

mkdir -p /u01/app/oracle/product/19C/dbhome

chown -R oracle:oinstall /u01

chmod -R 775 /u01

mkdir -p /u01/app/oracle/product/19C/dbhome

chown -R oracle:oinstall /u01

chmod -R 775 /u01

Step 2: We download the oracle database 19c software and uploaded zip file into the new 19c home location (one node of RAC)

cd /u01/app/oracle/product/19C/dbhome

Copy Oracle 19C Software

Unzip it

cd /u01/app/oracle/product/19C/dbhome

Copy Oracle 19C Software

Unzip it

Step 3: Install oracle 19c software only in one node of RAC environment for upgradation

Cd $ORACLE_HOME

./runInstaller ------------------------- Install software only 1st node only

Setup only software

Cd $ORACLE_HOME

./runInstaller ------------------------- Install software only 1st node only

Setup only software

Step 4: Check the Invalid Objects if there have any invalid objects compile them before the upgrade.

$ sqlplus / as sysdba

SQL*Plus: Release 12.2.0.1.0 Production on Thu Jan 5 16:12:57 2023

Connected to:

Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

$ SQL> select count(*) from dba_objects where status='INVALID';

COUNT(*)

----------

$ sqlplus / as sysdba

SQL*Plus: Release 12.2.0.1.0 Production on Thu Jan 5 16:12:57 2023

Connected to:

Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

$ SQL> select count(*) from dba_objects where status='INVALID';

COUNT(*)

----------

Step 5: Create the required directory where store the upgradation script

mkdir -p /home/oracle/preupgrade -- 1st Node only

chown -R oracle:oinstall /home/oracle/preupgrade

chmod -R 775 /u01/spool/preupgrade

mkdir -p /home/oracle/preupgrade -- 1st Node only

chown -R oracle:oinstall /home/oracle/preupgrade

chmod -R 775 /u01/spool/preupgrade

Step 6: Execute the Pre-upgrade script in 1st node only

/u01/app/oracle/product/12C/dbhome/jdk/bin/java -jar /u01/app/oracle/product/19C/dbhome/rdbms/admin/preupgrade.jar FILE DIR /home/oracle/preupgrade/

Note: Carefully view the pre-upgrade log and take care of the issues.

Step 7: GATHER DICTIONARY STATS

Before the upgrade process, gather stats. One of the recommendations is to export the stats as well.

sqlplus / as sysdba

SQL> EXECUTE DBMS_STATS.GATHER_DICTIONARY_STATS;

PL/SQL procedure successfully completed.

sqlplus / as sysdba

SQL> EXECUTE DBMS_STATS.GATHER_DICTIONARY_STATS;

PL/SQL procedure successfully completed.

Step 8: Purge Recycle Bin

Before the upgrade process, empty the recycle bin.

SQL> purge dba_recyclebin;

DBA Recyclebin purged.

Step 9: RUN PREUPGRADE FIXUP SQL

SQL> @/home/oracle/preupgrade/preupgrade_fixups.sql

Step 10: Verify Archive Destination Size

Ensure you have enough free space in db_recovery_file_dest and make changes to the parameter db_recovery_file_dest_size if needed increase size.

SQL> archive log list;

SQL> show parameter db_recovery_file_dest

Step 11: Stop Database Listener and Check the Flashback Status

# srvctl stop LISTENER – run from grid user

SQL> select flashback_on from v$database;

FLASHBACK_ON

------------------

Enable flashback database using the following procedure

SQL> ALTER SYSTEM SET DB_RECOVERY_FILE_DEST_SIZE=20G;

SQL> ALTER SYSTEM SET DB_RECOVERY_FILE_DEST='+FRA’;

SQL> alter database flashback on;

Step 12: Create a restore point before the upgrade issue.

SQL> create restore point PRE_UPGRADE guarantee flashback database;

Step 13: Oracle database upgrade for RAC environment.

Shutdown database of both nodes

copy spfile/create pfile and password file to new 19c oracle home.

COPY [SPFILE/PASSWORD] FILES FROM 12C TO 19C ORACLE HOME.

create pfile and store it in 19c home

sqlplus / as sysdba

create pfile=’/u01/app/oracle/product/19c/dbhome/dbs/initTESTDB1.ora’ from spfile;

NOTE: In pfile edit one parameter to cluster_database=False

Shutdown database & listener

srvctl stop LISTENER -- from grid user

srvctl stop database -d TESTDB -o immediate -- from oracle user

Step 14: Start database in upgrade from 19c home

export ORACLE_HOME=/u01/app/oracle/product/19C/dbhome

$ which sqlplus

/u01/app/oracle/product/19c/db_1/bin/sqlplus

sqlplus / as sysdba

SQL>startup upgrade

SQL> select status from v$instance;

Status

---------------------------------------

OPEN MIGRATE

Now run dbupgrade from 19c home

cd $ORACLE_HOME/bin

$ ./dbupgrade -- 1st node only & 2nd node will be down state

export ORACLE_HOME=/u01/app/oracle/product/19C/dbhome

Step 15: Post-Upgrade Activities

After the upgrade start the database from 19c home sqlplus / as sysdba

Step 16: Post upgrade run utlrp.sql to compile an invalid object

cd $ORACLE_HOME/rdbms/admin

SQL> @utlrp.sql

Step 17: Run the post-upgrade script

cd /home/oracle/preupgrade

SQL> @postupgrade_fixups.sql

Step 18: Upgrade timezone

SQL> select version from v$timezone_file;

VERSION

---------------

cd $ORACLE_HOME/rdbms/admin

SQL> @utltz_upg_check.sql

When it executed without any error run the below sql script

SQL> @utltz_upg_apply.sql

Now Run the script utlusts.sql

Now run the script catuppst.sql

SQL> @catuppst.sql

Rerun post upgrade fixup sql to check where exit any error or not

SQL> @postupgrade_fixups.sql

Now check the invalid object

SQL> select count(*) from dba_objects where status='INVALID';

COUNT(*)

----------------

Upgradation is completed successfully so delete the restore point that created before upgrade

Set compatible parameter

SQL> show parameter compatible

NAME TYPE VALUE

------------------------------------ ----------- ------------------------------

compatible string 12.2.0

SQL> alter system set compatible='19.0.0' scope=spfilesid='*';

SQL>col COMP_ID for a10

col COMP_NAME for a40

col VERSION for a15

set lines 200 pages 100

select COMP_ID,COMP_NAME,VERSION,STATUS from dba_registry;

Now set a new database home for the second node of RAC and startup database for the second node

After completing the migration upgrade oracle home for RAC

Oracle database new home upgradation in oracle RAC infrastructure using the following command

srvctlconfig database -d TESTDB -- Show the database configuration

su – oracle

cd $ORACLE_HOME/bin

./srvctl upgrade database -d databasename -oraclehome 19c_new_home

./srvctl upgrade database -d TESTDB -oraclehome/u01/app/oracle/product/19C/dbhome

./srvctl status database -d TESTDB -- Check status of database

./srvctl stop database -d TESTDB -o immediate -- stop database of the cluster

./srvctl start database -d TESTDB -- Start database of cluster

The database has been successfully upgraded to 19c.

SQL> select name, open_mode, version from v$database, v$instance;

NAME OPEN_MODE VERSION

--------- -------------------- -----------------

TESTDB READ WRITE 19.0.0.0.0

=============================

AutoUpgrade – Step-by-step

=============================

1)The new AutoUpgrade Utility – Download, documentation and supported versions

2)Create and adjust the config file for AutoUpgrade

3)Config file for AutoUpgrade – Advanced options

4)Config file for AutoUpgrade – Tweaking init parameters

5)AutoUpgrade: ANALYZE, FIXUPS, UPGRADE and DEPLOY modes

6)AutoUpgrade: Where do you find all the logfiles?

7)UPG: The AutoUpgrade Command Line Interface

8)Upgrading Multitenant databases with AutoUpgrade

9)Moving to a new server with AutoUpgrade

10)How to tweak the hidden settings in AutoUpgrade

11)AutoUpgrade and Data Guard, RAC, Restart and non-CDB to PDB

12)AutoUpgrade and Wallets

============================

You get it when you install Oracle Database 19c (19.3) or newer. Or – and this is the recommended source, you download the most recent version from MOS Note: 2485457.1 – AutoUpgrade Tool:

root@oraclelab1 patches]# ll

total 11868216

-rw-r--r--. 1 root root 4822622 May 23 10:36 autoupgrade.jar

[root@oraclelab1 patches]# cp autoupgrade.jar /u01/app/oracle/product/19.0.0.0/dbhome_1/rdbms/admin/

cp: overwrite ‘/u01/app/oracle/product/19.0.0.0/dbhome_1/rdbms/admin/autoupgrade.jar’? y

[root@oraclelab1 patches]# cd /u01/app/oracle/product/19.0.0.0/dbhome_1/rdbms/admin/

[root@oraclelab1 admin]# ll autoupgrade.jar

-rw-r--r--. 1 oracle oinstall 4822622 May 23 10:39 autoupgrade.jar

[root@oraclelab1 admin]# . oraenv

ORACLE_SID = [root] ? DEVDB

The Oracle base has been set to /u01/app/oracle

[root@oraclelab1 admin]# $ORACLE_HOME/jdk/bin/java -jar $ORACLE_HOME/rdbms/admin/autoupgrade.jar -version

build.version 22.3.220503

build.date 2022/05/03 11:55:20 -0400

build.hash 9e84e228

build.hash_date 2022/05/03 11:37:00 -0400

build.supported_target_versions 12.2,18,19,21

build.type production

[root@oraclelab1 admin]# $ORACLE_HOME/jdk/bin/java -version

java version "1.8.0_201"

[root@oraclelab1 admin]# cd /u01/patches/

[root@oraclelab1 patches]# mkdir 19c-autoupg

[root@oraclelab1 patches]# cd 19c-autoupg/

[root@oraclelab1 19c-autoupg]# ll

total 0

[root@oraclelab1 19c-autoupg]# export ORACLE_HOME=/u01/app/oracle/product/19.0.0.0/dbhome_1

[root@oraclelab1 19c-autoupg]# export PATH=$PATH:$ORACLE_HOME/jdk/bin

[root@oraclelab1 patches]# chown oracle: autoupgrade.jar

[root@oraclelab1 19c-autoupg]# $ORACLE_HOME/jdk/bin/java -jar $ORACLE_HOME/rdbms/admin/autoupgrade.jar -create_sample_file config

Created sample configuration file /u01/patches/19c-autoupg/sample_config.cfg

[root@oraclelab1 19c-autoupg]# ls -ltrh

total 8.0K

-rw-r--r--. 1 root root 5.8K May 23 10:41 sample_config.cfg

[root@oraclelab1 19c-autoupg]# cp sample_config.cfg DB2_config.cfg

[root@oraclelab1 19c-autoupg]# vi DB2_config.cfg

vi sample_config

Sample 1:

# Database number 1 - Full DB/CDB upgrade

upg1.log_dir=/u01/patches/19c-autoupg/DB2 # Path of the log directory for the upgrade job

upg1.sid=DB2 # ORACLE_SID of the source DB/CDB

upg1.source_home=/u01/app/oracle/product/12.2.0.1/dbhome_1 # Path of the source ORACLE_HOME

upg1.target_home=/u01/app/oracle/product/19.0.0.0/dbhome_1 # Path of the target ORACLE_HOME

upg1.start_time=NOW # Optional. [NOW | +XhYm (X hours, Y minutes after launch) | dd/mm/yyyy hh:mm:ss]

upg1.upgrade_node=oraclelab1.localdomain.com # Optional. To find out the name of your node, run the hostname utility. Default is ''localhost''

#upg1.run_utlrp=[yes|no] # Optional. Whether or not to run utlrp after upgrade

#upg1.timezone_upg=[yes|no] # Optional. Whether or not to run the timezone upgrade

#upg1.target_version=[12.2|18|19|21] # Oracle version of the target ORACLE_HOME. Only required when the target Oracle database version is 12.2

[root@oraclelab1 19c-autoupg]# ls -otrh

total 16K

-rw-r--r--. 1 oracle 5.8K May 23 10:41 sample_config.cfg

-rw-r--r--. 1 oracle 5.8K May 23 10:46 DB2_config.cfg

[root@oraclelab1 19c-autoupg]# su - oracle

Last login: Mon May 23 09:12:24 IST 2022 on pts/3

[oracle@oraclelab1 ~]$ . oraenv

ORACLE_SID = [oracle] ? DB2

The Oracle base has been set to /u01/app/oracle

[oracle@oraclelab1 ~]$ env |grep ORA

ORACLE_SID=DB2

ORACLE_BASE=/u01/app/oracle

ORACLE_HOME=/u01/app/oracle/product/12.2.0.1/dbhome_1

[oracle@oraclelab1 ~]$ export ORACLE_HOME=/u01/app/oracle/product/19.0.0.0/dbhome_1

[oracle@oraclelab1 ~]$

[oracle@oraclelab1 ~]$ env |grep ORA

ORACLE_SID=DB2

ORACLE_BASE=/u01/app/oracle

ORACLE_HOME=/u01/app/oracle/product/19.0.0.0/dbhome_1

[oracle@oraclelab1 ~]$ export PATH=$PATH:$ORACLE_HOME/jdk/bin

[oracle@oraclelab1 ~]$ cd /u01/patches/19c-autoupg/

[oracle@oraclelab1 19c-autoupg]$ ll

total 16

-rw-r--r--. 1 oracle oinstall 5886 May 23 10:46 DB2_config.cfg

-rw-r--r--. 1 oracle oinstall 5905 May 23 10:41 sample_config.cfg

[oracle@oraclelab1 19c-autoupg]$ $ORACLE_HOME/jdk/bin/java -jar $ORACLE_HOME/rdbms/admin/autoupgrade.jar -config DB2_config.cfg -mode ANALYZE

[oracle@oraclelab1 19c-autoupg]$ $ORACLE_HOME/jdk/bin/java -jar $ORACLE_HOME/rdbms/admin/autoupgrade.jar -config DB2_config.cfg -mode DEPLOY

AutoUpgrade 22.3.220503 launched with default internal options

Processing config file ...

+--------------------------------+

| Starting AutoUpgrade execution |

+--------------------------------+

1 Non-CDB(s) will be analyzed

Type 'help' to list console commands

upg> lsj

+----+-------+---------+---------+-------+----------+-------+----------------------------+

+----+-------+---------+---------+-------+----------+-------+----------------------------+

+----+-------+---------+---------+-------+----------+-------+----------------------------+

Total jobs 1

upg> status

Config

User configuration file [/u01/patches/19c-autoupg/DB2_config.cfg]

General logs location [/u01/patches/19c-autoupg/autoupgrade/cfgtoollogs/upgrade/auto]

Mode [ANALYZE]

Jobs Summary

Total databases in configuration file [1]

Total Non-CDB being processed [1]

Total Containers being processed [0]

Jobs finished successfully [0]

Jobs finished/aborted [0]

Jobs in progress [1]

upg> tasks

Please check the summary report at:

/u01/patches/19c-autoupg/autoupgrade/cfgtoollogs/upgrade/auto/status/status.html

/u01/patches/19c-autoupg/autoupgrade/cfgtoollogs/upgrade/auto/status/status.log

upg> Job 101 completed

------------------- Final Summary --------------------

Numb

Post check -

er of databases [ 1 ]

Jobs finished [1]

Jobs failed [0]

Jobs restored [0]

Jobs pending [0]

---- Drop GRP at your convenience once you consider it is no longer needed ----

Drop GRP from DB2: drop restore point AUTOUPGRADE_9212_DB2122010

Post check -

SQL>

SQL> column comp_name format a40

column version format a12

column status format a15

select comp_name,version,status from dba_registry;SQL> SQL> SQL>

COMP_NAME VERSION STATUS

---------------------------------------- ------------ ---------------

Oracle Database Catalog Views 19.0.0.0.0 VALID

Oracle Database Packages and Types 19.0.0.0.0 VALID

JServer JAVA Virtual Machine 19.0.0.0.0 VALID

Oracle XDK 19.0.0.0.0 VALID

Oracle Database Java Packages 19.0.0.0.0 VALID

OLAP Analytic Workspace 19.0.0.0.0 VALID

Oracle Real Application Clusters 19.0.0.0.0 OPTION OFF

Oracle XML Database 19.0.0.0.0 VALID

Oracle Workspace Manager 19.0.0.0.0 VALID

Oracle Text 19.0.0.0.0 VALID

Oracle Multimedia 19.0.0.0.0 VALID

COMP_NAME VERSION STATUS

---------------------------------------- ------------ ---------------

Spatial 19.0.0.0.0 VALID

Oracle OLAP API 19.0.0.0.0 VALID

Oracle Label Security 19.0.0.0.0 VALID

Oracle Database Vault 19.0.0.0.0 VALID

15 rows selected.

SQL> column owner format a15

column object_name format a40

column object_type format a20

select owner, object_name, object_type from dba_objects where status='INVALID' and owner like '%SYS%' order by object_type,owner,object_name;SQL> SQL> SQL>

no rows selected

SQL> select * from v$restore_point

2 ;

SCN DATABASE_INCARNATION# GUA STORAGE_SIZE

---------- --------------------- --- ------------

TIME

---------------------------------------------------------------------------

RESTORE_POINT_TIME PRE

--------------------------------------------------------------------------- ---

NAME

--------------------------------------------------------------------------------

PDB CLE PDB_INCARNATION# REP CON_ID

--- --- ---------------- --- ----------

1429198 2 YES 1048576000

23-MAY-22 10.50.18.000000000 AM

YES

SCN DATABASE_INCARNATION# GUA STORAGE_SIZE

---------- --------------------- --- ------------

TIME

---------------------------------------------------------------------------

RESTORE_POINT_TIME PRE

--------------------------------------------------------------------------- ---

NAME

--------------------------------------------------------------------------------

PDB CLE PDB_INCARNATION# REP CON_ID

--- --- ---------------- --- ----------

AUTOUPGRADE_9212_DB2122010

NO NO 0 NO 0

SQL> drop restore point AUTOUPGRADE_9212_DB2122010;

Restore point dropped.

SQL> select * from v$restore_point;

no rows selected

If Fails Restore it

[oracle@oraclelab1 ~]$ rman target /

Recovery Manager: Release 19.0.0.0.0 - Production on Mon May 23 11:38:22 2022

Version 19.3.0.0.0

connected to target database: DB2 (DBID=2268890156)

RMAN> flashback database to AUTOUPGRADE_9212_DB2122010;

=======================================

Oracle 19c Database Upgrade From 12.2.0.1 to 19.2.0.0 Using DBUA

===================================

[oracle@standby dbs]$ export ORACLE_SID=upgr

[oracle@standby dbs]$ sqlplus / as sysdba

SQL*Plus: Release 12.2.0.1.0 Production on Thu Mar 21 20:02:00 2019

Connected to an idle instance.

SQL> startup

ORACLE instance started.

Ensure backup is complete before upgrade.

Empty Recycle bin.

Run Gather statistics to finish upgrade soon.

SQL> SELECT * FROM v$backup WHERE status != 'NOT ACTIVE';

no rows selected

SQL> PURGE DBA_RECYCLEBIN ;

DBA_Recyclebin purged.

SQL> EXEC DBMS_STATS.GATHER_DICTIONARY_STATS;

PL/SQL procedure successfully completed.

SQL> !

Stop the running 12c listener

lsnrctl stop LISTENER

Run preupgrade tool and fixups

Oracle strongly recommends that you run the Pre-Upgrade Information Tool before starting the upgrade with DBUA. Although DBUA runs the Pre-Upgrade Information Tool as part of the pre-requisite checks, it is good practice to run the tool ahead of time in order to analyze the database and take actions that can decrease downtime for upgrading.

Connect to 12.2.0.1 database environment as SYS,

Pre-upgrade checks :

Run the preupgrade tool .

[oracle@standby dbs]$ java -jar /u01/app/oracle/product/19.0.0/dbhome_1/rdbms/admin/preupgrade.jar TEXT TERMINAL

Report generated by Oracle Database Pre-Upgrade Information Tool Version

19.0.0.0.0 Build: 1 on 2019-03-21T20:07:45

Upgrade-To version: 19.0.0.0.0

Run the preupgrade_fixup.sql

Before upgrade:

Log into the database and execute the preupgrade fixups

@/u01/app/oracle/cfgtoollogs/upgr/preupgrade/preupgrade_fixups.sql

After the upgrade:

Log into the database and execute the postupgrade fixups

@/u01/app/oracle/cfgtoollogs/upgr/preupgrade/postupgrade_fixups.sql

[oracle@standby dbs]$ !sq

sqlplus / as sysdba

SQL*Plus: Release 12.2.0.1.0 Production on Thu Mar 21 20:13:03 2019

Connected to:

Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SQL> @/u01/app/oracle/cfgtoollogs/upgr/preupgrade/preupgrade_fixups.sql

Executing Oracle PRE-Upgrade Fixup Script

Preup Preupgrade

Action Issue Is

Number Preupgrade Check Name Remedied Further DBA Action

------ ------------------------ ---------- -------------------------

1. pre_fixed_objects YES None.

2. tablespaces_info NO Informational only.

Further action is optional.

3. rman_recovery_version NO Informational only.

Further action is optional.

Now run the dbua utility from 12c oracle_home location.

[oracle@standby ~]$ export ORACLE_HOME=/u01/app/oracle/product/19.0.0/dbhome_1

[oracle@standby ~]$ export ORACLE_SID=upgr

[oracle@standby ~]$ export PATH=$ORACLE_HOME/bin:$PATH

[oracle@standby ~]$ dbua

DBUA performs some of the checks before actually starting the database upgrade. Some of the checks can be done manually to reduce downtime for the upgrade.

DBUA provides below options:

– Upgrade timezone.

– Gather dictionary statistics before upgrade.

– Make user tablespaces read only.

– Take RMAN backup before upgrade.

– Create Restore Point for Database Flashback

– Restore database backup to rollback upgrade

– Option to execute Custom scripts before and after upgrade

– show the location of DBUA logs and Alert log files.

– Option to upgrade existing listener to 18.x home or create a new listener in 18.x target home.

Steps -

1)select the Upgrade Oracle Database option, then click the “Next” button.

2)If the prerequisite checks highlight any issues, take the appropriate action to fix the issues. If all the prerequisite checks are passed, click the “Next” button.

3)Amend the upgrade options if necessary, then click the “Next” button.

4)Select the recovery options for use in the event of an upgrade failure, then click the “Next” button.

5)If the database is using the 12c listener, accept the defaults, so the listener will be upgraded. If you prefer to manually define a new listener, you can do that now too and click the “Next” button

6)Select the desired management options, then click the “Next” button

Post-upgrade checks:

Run “postupgrade_fixups.sql” to check post upgrade status.

SQL> @/u01/app/oracle/cfgtoollogs/upgr/preupgrade/postupgrade_fixups.sql

Session altered.

Now the database upgrade is complete and the database is ready for normal use.

SQL> select VERSION,name,open_mode from v$instance,v$database;

VERSION NAME OPEN_MODE

----------------- --------- --------------------

19.0.0.0.0 UPGR READ WRITE

SQL> SELECT * FROM v$timezone_file;

FILENAME VERSION CON_ID

-------------------- ---------- ----------

timezlrg_32.dat 32 0

SQL> select TZ_VERSION from registry$database;

TZ_VERSION

----------

===============================

Restore:

========

OLR OCR and Votedisk

OLR - node specific (Manual bkp)

OCR - cluster specific (auto bkp + includes votedisk )

Votedisk - cluster specific (No bkp)

ocrconfig -local -restore /u01/app/oracle/crsdata/oranode1/olr/backup_20230212_101854.olr

+RECO/oranode-clu/OCRBACKUP/backup00.ocr.379.1128582589

ocrconfig -copy +RECO:/oranode-clu/OCRBACKUP/backup00.ocr.379.1128582589

ocrconfig -restore +DATA

ocrconfig -restore +RECO:/oranode-clu/OCRBACKUP/backup00.ocr.379.1128582589

ocrconfig -restore +RECO/oranode-clu/OCRBACKUP/backup00.ocr.379.1128582589

amdu method

/u01/app/19.0.0.0/grid/bin/amdu -diskstring '/dev/oracleasm/disks/DATA1' -extract DATA.255

/u01/app/19.0.0.0/grid/bin/amdu -diskstring '/dev/oracleasm/disks/DATA2' -extract DATA.255

/u01/app/19.0.0.0/grid/bin/amdu -diskstring '/dev/oracleasm/disks/DATA3' -extract DATA.255

ocrconfig -restore /tmp/amdu_2023_02_12_11_05_05/DATA_255.f

dd if=/dev/zero of=/dev/oracleasm/disks/ACFSDG bs=4096 count=10

crsctl start crs -excl -nocrs

crsctl query css votedisk

crsctl replace votedisk +VOTE1

==========================

ACFS

crsctl stat res ora.ACFS_DG.dg -t

asmcmd volcreate -G ACFS_DG -s 10G acfs_test

asmcmd volinfo -G ACFS_DG acfs_test

crsctl stat res -t | grep -i "advm"

crsctl stat res ora.ACFS_DG.ACFS_TEST.advm -t

mkfs -t acfs /dev/asm/acfs_test-278

(mkfs -t ext3/ext4/xfs /dev/sdb)

mkdir -p /acfs_test

chown oracle:oinstall /acfs_test

/sbin/acfsutil registry -a /dev/asm/acfs_test-278 /acfs_test -u oracle

crsctl stat res -t | grep -i "acfsdg"

crsctl stat res ora.acfs_dg.acfs_test.acfs -t

srvctl status filesystem -d /dev/asm/acfs_test-278

srvctl start filesystem -d /dev/asm/acfs_test-278

srvctl stop filesystem -d /dev/asm/acfs_test-278

asmcmd volinfo -G ACFS_DG acfs_test

srvctl config filesystem

srvctl stop filesystem -d /dev/asm/acfs_test-278

/sbin/acfsutil rmfs /dev/asm/acfs_test-278

asmcmd voldisable -G ACFS_DG acfs_test

asmcmd voldelete -G ACFS_DG acfs_test

==============================

direct database upgrade 19c

===========================

In a rac cluster if one node gets down then how will you rectify the root couse ?

1)Immediate Response:

Ensure that other nodes in the cluster are still operational.

Verify if the failed node is completely unreachable or if it's just experiencing issues with certain services or components.

2)Check Cluster Logs and Alerts:

Review Oracle Clusterware logs, Cluster Health Monitor (CHM) logs, Oracle ASM alert logs, and operating system logs on the failed node to

identify any error messages or warnings leading up to the failure.

Use tools like Cluster Verification Utility (CVU) to assess the overall health of the cluster and identify potential issues.

3)Node Diagnostics:

Connect to the failed node remotely if possible and perform diagnostics to identify hardware or software issues.

Check system logs, hardware component status (using tools like Integrated Lights Out Manager - ILOM), and Oracle diagnostics

(e.g., Automatic Diagnostic Repository - ADR) for insights into the problem.

4)Attempt Node Restart:

If the node failure appears to be a transient issue, attempt to restart the node using Oracle Clusterware utilities like srvctl or crsctl.

Monitor the node's startup process for any errors or warnings.

5)Failover and Service Relocation:

If the failed node cannot be restarted immediately or if the failure persists, initiate a failover or service relocation to other

available nodes in the cluster.

Use Oracle Clusterware commands (srvctl) to relocate services or resources to healthy nodes to minimize downtime.

6)Root Cause Analysis:

Once the node is back online or services have been relocated, conduct a thorough root cause analysis to determine the underlying reason

for the node failure.

Investigate hardware failures, software issues, network problems, or configuration errors that may have contributed to the failure.

7)Resolution and Remediation:

Address the identified root cause by resolving hardware or software issues, applying patches or updates, adjusting configuration settings,

or implementing best practices for cluster management and maintenance.

Document the actions taken and any recommendations for preventing similar issues in the future.

8)Monitoring and Prevention:

Implement proactive monitoring and alerting mechanisms to detect and respond to potential node failures or performance issues in the future.

Configure Oracle Clusterware alarms, Grid Infrastructure Management Repository (GIMR), or third-party monitoring tools to provide real-time

visibility into cluster health and performance.

12) Mention what is split brain syndrome in RAC?

In Oracle RAC, all the instances/servers communicate with each other using a private network.

When the instance members in a RAC fail to ping/connect to each other via this private network and continue to

process data block independently. Then this process is referred as Split Brain Syndrome.

13) What happens if you keep split brain syndrome in RAC unresolved? How it can be resolved?

If you keep split brain syndrome unresolved, then there would be data integrity issue. The blocks changed in one instance

will not be locked and could be over-written by another instance. It is resolved by using the voting disk, it will decide which

node(s) will survive and which node(s) will be evicted

Voting disk -The Voting Disk. Contains information to determine which nodes are active members of the cluster at any given moment.

What is deadlock and how to resolve it

A deadlock occurs when two or more sessions are waiting for data locked by each other, resulting in all the sessions being blocked.

Oracle automatically detects and resolves deadlocks by rolling back the statement associated with the transaction that detects the deadlock.

Typically, deadlocks are caused by poorly implemented locking in application code.

Solution -

Oracle is smart and it is able to find deadlock situation within 3 seconds. Below are the options to avoid deadlocks inside the database