Using Dbvisit Standby with Oracle Clusterware
Introduction
The examples provided are of Dbvisit Standby Version 9.0.x, there should not be any changes on how this is being implemented in Dbvisit Standby Version 10
Also, this section is valid only for customer running on 19c versions below 19.7 (19.0, 19.3) - in those versions Standard Edition High Availability (SEHA) was not yet introduced and it was needed to make manual workaround for running the Oracle Database as active / passive. This is not needed in 19.7 and onwards, because SEHA takes care of everything.
Oracle Clusterware
There is a number of Linux operating systems that support the Oracle Database, and they are all excellent in their own right. I have always favored using Oracle Linux and over the years it has become easier and easier to install and configure complex database configurations on Oracle Linux with minimum effort. With Oracle Linux, you can get different levels of support and with these levels you get a number of extra options available to you at no additional cost (for these options), meaning if you, for example, pay for Oracle Linux basic or premier support you get Oracle Clusterware! This is amazing software for you to use just because you have Oracle Linux paid support agreement (Basic or Premier), nice!
Now taking this into account, let's start thinking a little outside the box and getting a little more out of your investment. There is a lot you can do here. Many companies nowadays are looking at more highly available solutions, and with Oracle Linux and Oracle Clusterware, you have something extra now in your arsenal that you can use to help you achieve this. Using these two components to create a cluster configuration where shared storage is used to help you create an Active / Passive cluster for your application to run on… but why not use this for your Oracle Standard Edition 2 (SE2) database as well? This is what I would like to explore more in this blog series. How you can configure an Oracle Linux 7 cluster consisting out of two nodes using shared storage and Oracle Clusterware, then install the SE2 database software and create an Active / Passive cluster configuration where the database instance will only run on one node at a time.
It is, however, important to note that I am not talking about Oracle RAC or Oracle RAC One Node here - These are amazing technologies that will require Oracle Enterprise Edition in 19c onwards as Oracle RAC is no longer available as an option for the SE2 software set. In this blog series, I am specifically going to show you how you can enable a two-node cluster running for your SE2 database in an Active / Passive configuration.
Without getting into the details of the licensing, when you run Oracle SE2 you are allowed a maximum of two CPU sockets per server, and in this scenario we will use two servers each with two CPU sockets, but remember the database instance will only run on one of the servers in the cluster at any given point in time (in an “active/passive” configuration) - and based on licensing of data recovery environments as discussed in this document - http://www.oracle.com/us/corporate/pricing/data-recovery-licensing-070587.pdf, you are able to run the database instance on the second node in the cluster for a maximum of 10 days per year. As always, please check your licensing with your Oracle account manager.
Now to add a little extra to the configuration - as I think everyone should always make sure they protect their databases against disaster; I will also in the last part of the series show you how you can configure Dbvisit Standby in this configuration - running as cluster resource, to help protect your primary database in case of disaster strikes.
In the examples I will make use of Oracle Clusterware 19c and Oracle 19c SE2 Database and configure it in an Active / Passive cluster where the single instance SE2 database is running on primary node 1, but in case of failure on that node the database will be failed over “as a cluster resource” to the second node in the cluster - this can be summarised in the following high-level diagram:
The blog series will cover the following topics:
- Configuration the Oracle Linux nodes using Oracle Linux 7
- Installing and configuration of the Oracle Clusterware - version 19c
- Configuration of Oracle ACFS (cluster filesystem)
- Installing the Oracle Database software - 19c SE2
- Creating a single instance database and adding it as a cluster resource
- Creating a Virtual IP that will be running on the “Active” node
- Installing Dbvisit Standby and adding it as cluster resources
- Creating a Standby Database on a 3rd server running Oracle Linux 7 with Oracle SE2 Database software installed
No before I get into the details - there are a few key requirements for this type of configuration I do need to mention:
- Two Oracle Linux 7 (7.6) servers with latest updates applied
- I will be using two Virtual Machines running VirtualBox
- Shared Storage that is presented to both nodes
- In VirtualBox you can add a disk to node 1, then make it “sharable” and add the same disk to node 2. This works really well for small testing configurations and is not recommended for production.
- Two network interfaces in each node:
- Interface 1 - used for public
- Interface 2 - used for private interconnect traffic as well as ASM
- A DNS server
- You ideally want to use a DNS server to add all the various IP addresses that are used by the cluster.
- In my case, I just created a small BIND DNS server
- In my test configuration, I will use the following IP ranges and these values are in my DNS server, but I also added them to my local /etc/hosts file.
10.0.2.21 oel7clus1.oraclekiwi.co.nz oel7clus1 #Primary Node 1 10.0.2.22 oel7clus2.oraclekiwi.co.nz oel7clus2 #Primary Node 2 10.0.2.23 oel7clus1-vip.oraclekiwi.co.nz oel7clus1-vip 10.0.2.24 oel7clus2-vip.oraclekiwi.co.nz oel7clus2-vip # 10.0.2.25 oel7clus-scan.oraclekiwi.co.nz oel7clus-scan ## The SCAN details in DNS # 10.0.2.26 oel7clus-scan.oraclekiwi.co.nz oel7clus-scan # 10.0.2.27 oel7clus-scan.oraclekiwi.co.nz oel7clus-scan 10.0.2.28 oel7clus-dbvip.oraclekiwi.co.nz oel7clus-dbvip # Cluster IP for Dbvisit 10.0.2.29 oel7fs1.oraclekiwi.co.nz oel7fs1 # Standby Server
Step 1: Oracle Linux preparation
Now in my configuration, I used Oracle Linux 7 update 6. I will not get into the details of installing Linux, but it really is easy and in most cases, you can just accept the defaults.
I do want to highlight that I used a 60GB disk and allocated 40G of the disk for /u01 filesystem, mainly as I use the /u01 filesystem a lot for various things and this is where I will configure Oracle Clusterware to run from.
I also make sure I assign enough memory and CPU to Virtual Machines. In VirtualBox management settings I added 8GB of memory.
You can play with the CPU settings, but this varies depending on your CPU options, example if you have 1 CPU with 4 cored and hyper threading enabled you will get more available (8 virtual CPUs).
I would recommend for testing you set the value to at least 2 on each Virtual Machine. For reference my disk layout after installation looked like the following:
root@oel7clus1 ~ : df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 3.8G 0 3.8G 0% /dev tmpfs 3.8G 0 3.8G 0% /dev/shm tmpfs 3.8G 89M 3.7G 3% /run tmpfs 3.8G 0 3.8G 0% /sys/fs/cgroup /dev/sda3 6.3G 3.6G 2.4G 61% / tmpfs 4.0G 4.0K 4.0G 1% /tmp /dev/sda1 488M 279M 174M 62% /boot /dev/mapper/oravg-u01 40G 49M 38G 1% /u01 tmpfs 771M 0 771M 0% /run/user/0 tmpfs 771M 0 771M 0% /run/user/1000
I always prefer to use role separation when installing Oracle Clusterware and Database software, so I will directly after installing my Virtual Machines, add the “oracle” and “grid” users. This can easily be done by the following commands:
groupadd -g 501 oinstall groupadd -g 502 dba groupadd -g 503 oper groupadd -g 504 asmadmin groupadd -g 505 asmdba groupadd -g 506 asmoper groupadd -g 507 bkpdba groupadd -g 508 dgdba groupadd -g 509 kmdba groupadd -g 510 racdba useradd -m -g oinstall -G oinstall,dba,oper,asmadmin,asmdba,bkpdba,dgdba,kmdba -u 501 oracle useradd -m -g oinstall -G oinstall,dba,asmadmin,asmdba,asmoper -u 502 grid
There are a few more configuration options required, but first, let's update all required software on the cluster nodes.
You should follow the recommendation of the Oracle Clusterware and Oracle Database installation documentation to make sure you have all the required operating system packages installed.
In this case, I took the easier way and based on past experience and installed the following via the yum package installer.
Please note that in this list I am actually installing a little bit extra, mainly as I tend to use some of these extra tools and why not just do it all at the same time and keep it simple.
So here is my list of options:
yum -y --enablerepo=ol7_latest,ol7_optional_latest,ol7_developer_EPEL install dkms ocfs2-tools-devel ocfs2-tools reflink btrfs-progs btrfs-progs-devel oracleasm-support kmod-oracleasm git rpmbuild patch bc binutils bzip2 bzip2-devel dos2unix gcc gcc-c++ glibc-static gzip libaio libaio-devel libgcc libstdc++ libstdc++-devel lsof make man man-pages net-tools nss-devel perl strace subversion sudo sysstat tar tree unix2dos unzip wget which zip zlib zlib-devel zlib-static gettext tcl tcl-devel openssh openssh-server xclock xhost psmisc xorg-x11-fonts* xorg-x11-server-Xvfb.x86_64 xterm libvncserver xorg-x11-server-devel avahi-libs tk libXtst gtk3 mesa-dri-drivers xorg-x11-utils screen bind-libs bind-utils compat-libcap1 compat-libstdc++-33 e2fsprogs-libs ethtool gssproxy keyutils ksh libaio-devel libbasicobjects libcollection libevent libini_config libnfsidmap libpath_utils libref_array libtirpc libverto-libevent mailx nfs-utils quota quota-nls rpcbind smartmontools tcp_wrappers gdb patch libnfs-utils nfs-utils dejavu*
Now that I have the above installed on each node, I make sure all is up to date by running “yum update” and then perform a reboot on each system.
The next steps I perform are the following and to make it easier the list is done below so that you can just copy and paste it - but please NOTE, test this first and do not just run this on production without making sure you understand what it does.
As I am just creating a TEST configuration, I will assign a basic password to the oracle and grid user as well as add “sudo” permission without requiring a password.
Also please note some of these values will need review for production environments and they are discussed in more detail in the online Oracle Database and Grid Infrastructure installation guides for Linux, so please make sure you review these prior to any production configurations.
cat >> /etc/pam.d/login <<EOF session required pam_limits.so EOF echo 'oracle:Kiwi123' | chpasswd echo 'grid:Kiwi123' | chpasswd echo "oracle ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers.d/oracle echo "grid ALL=(ALL) NOPASSWD: ALL" >> /etc/sudoers.d/grid sed -i "s/^.*requiretty/#Defaults requiretty/" /etc/sudoers cat >> /etc/sysctl.conf <<EOF fs.aio-max-nr = 3145728 fs.file-max = 6815744 kernel.shmall = 1073741824 kernel.shmmax = 4398046511104 kernel.shmmni = 4096 kernel.sem = 250 32000 100 128 kernel.msgmax = 8192 kernel.msgmnb = 65536 net.ipv4.ip_local_port_range = 9000 65500 net.core.rmem_default=262144 net.core.rmem_max=4194304 net.core.wmem_default=262144 net.core.wmem_max=1048586 EOF cat >> /etc/security/limits.conf <<EOF grid soft nproc 2047 grid hard nproc 16384 grid soft nofile 4096 grid hard nofile 65536 oracle soft nofile 131072 oracle hard nofile 131072 oracle soft nproc 131072 oracle hard nproc 131072 oracle soft core unlimited oracle hard core unlimited oracle soft memlock 50000000 oracle hard memlock 50000000 EOF cat >> /etc/profile <<EOF if [ \$USER = "oracle" ] || [ \$USER = "grid" ]; then if [ $SHELL = "/bin/ksh" ]; then ulimit -p 16384 ulimit -n 65536 ulimit -s 10240 else ulimit -u 16384 -n 65536 -s 10240 fi umask 022 fi EDITOR=vi; export EDITOR PS1="\u@\h\w$"; PS2="\u@ \h --> "; export PS1 export PS2 NLS_DATE_FORMAT="dd/mm/yyyy:hh24:mi:ss" export NLS_DATE_FORMAT EOF cat >> /etc/sysconfig/network <<EOF NOZEROCONF=yes
The last two things I do on the two primary cluster nodes are to update their /etc/hosts with the following (now remember, I do have the DNS configured as well and my /etc/resolv.conf does point to my DNS server):
10.0.2.21 oel7clus1.oraclekiwi.co.nz oel7clus1 #Primary Node 1 10.0.2.22 oel7clus2.oraclekiwi.co.nz oel7clus2 #Primary Node 2 10.0.2.23 oel7clus1-vip.oraclekiwi.co.nz oel7clus1-vip 10.0.2.24 oel7clus2-vip.oraclekiwi.co.nz oel7clus2-vip # 10.0.2.25 oel7clus-scan.oraclekiwi.co.nz oel7clus-scan ## The SCAN details in DNS # 10.0.2.26 oel7clus-scan.oraclekiwi.co.nz oel7clus-scan # 10.0.2.27 oel7clus-scan.oraclekiwi.co.nz oel7clus-scan 10.0.2.28 oel7clus-dbvip.oraclekiwi.co.nz oel7clus-dbvip # Cluster IP for Dbvisit 10.0.2.29 oel7fs1.oraclekiwi.co.nz oel7fs1 # Standby Server
My primary hosts are called oel7clus1 and oel7clus2 and I am just using my test domain oraclekiwi.co.nz
Then a crucial requirement for running Clusterware is that you must make sure you have SSH user equivalence configured.
In short, make sure that the oracle and grid user on both nodes can ssh between the nodes without any password or passphrase being requested. This is a standard requirement and really easy to do.
You can run the following commands on the two nodes first as the grid user then the oracle user to get this done - you will be prompted for confirmation and the user password while doing this - remember do not add a passphrase when prompted, just press enter.
ssh-keygen -t rsa ssh-copy-id -i ~/.ssh/id_rsa.pub oel7clus1 ssh-copy-id -i ~/.ssh/id_rsa.pub oel7clus2
Once you have done the above, you should be able to SSH between the two systems without any passwords, example:
grid@oel7clus1 ~ : ssh oel7clus1 Last login: Sun Sep 1 14:28:22 2019 grid@oel7clus1 ~ : ssh oel7clus2 Last login: Thu Aug 22 13:58:37 2019 grid@oel7clus2 ~ : exit logout Connection to oel7clus2 closed. grid@oel7clus1 ~ : exit logout Connection to oel7clus1 closed. grid@oel7clus1 ~ :
Step 2: Configure Disks
Before we want to get into starting the Oracle Clusterware installation, we need to make sure that we have the required shared disk to be used in the cluster.
In this test setup, I will make use of ASM and the ACFS filesystem. But before we get to that I need to add the disks. Now in this configuration, I added two disks.
1 disk at 3GB in size - this will be the disk hosting the OCR and will also be the Voting Disk in the cluster.
1 disk at 30GB in size - which will be configured as an ACFS filesystem to host my Database Software and Database Files.
As mentioned before I use VirtualBox in this example and I add the above-mentioned disks to node 1 (Make sure you use fixed-size not dynamic) then I set them to be sharable and then add them to the second node (when adding the disk state you want to use an existing disk and then add the two disks allocated to node 1 - but remember they must be set to sharable in the VirtualBox virtual media manager).
Now the two disks are seen as “/dev/sdb” and “/dev/sdc” in my configuration. You can easily get the details by running “fdisk -l”
Then from node 1 in the cluster, I run the following commands to partition the disk for use:
fdisk /dev/sdb o n p 1 < press enter < press enter w
Example:
root@oel7clus1 ~ : fdisk /dev/sdb
Welcome to fdisk (util-linux 2.23.2).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x44bf62a8.
Command (m for help): o
Building a new DOS disklabel with disk identifier 0x1bc47716.
Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-6291455, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-6291455, default 6291455):
Using default value 6291455
Partition 1 of type Linux and of size 3 GiB is set
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
root@oel7clus1 ~ :
I repeat the same for the second disk - /dev/sdc
I then end up with the partitions /dev/sdb1 and /dev/sdc1 which I will now use to create ASM disks from.
There are a few different options available to do this and everyone has their preference.
I like to use ASMLIB as I find it really easy to work with, but there are other options available to you which you can review.
But in this example, I will use ASMLIB. Let's first run “oracleasm configure” as the root user to configure it to run as the “grid” user with the correct group and to start on system boot:
Example:
root@oel7clus1 ~ : oracleasm configure -i
Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface [grid]: grid
Default group to own the driver interface [asmadmin]: asmadmin
Start Oracle ASM library driver on boot (y/n) [y]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
root@oel7clus1 ~ : systemctl restart oracleasm
Do make sure to do this on both the cluster nodes.
Next on node 1, let's mark the two disks we added as Oracle ASM disks.
As the “root” user run the following commands:
root@oel7clus1 ~ : oracleasm createdisk ASMDISK1 /dev/sdb1 Writing disk header: done Instantiating disk: done root@oel7clus1 ~ : oracleasm createdisk ASMDISK2 /dev/sdc1 Writing disk header: done Instantiating disk: done root@oel7clus1 ~ : oracleasm listdisks ASMDISK1 ASMDISK2
Now you have the two disks marked for use by ASM. Next, I would recommend you install the following ASMLIB package which you can download from the Oracle web site - https://www.oracle.com/linux/downloads/linux-asmlib-v7-downloads.html : oracleasmlib-2.0.12-1.el7.x86_64.rpm
Once you have this downloaded, you can use the oracleasm-discover command to see your disks:
root@oel7clus1 ~ : oracleasm-discover Using ASMLib from /opt/oracle/extapi/64/asm/orcl/1/libasm.so [ASM Library - Generic Linux, version 2.0.12 (KABI_V2)] Discovered disk: ORCL:ASMDISK1 [6289408 blocks (3220176896 bytes), maxio 1024, integrity none] Discovered disk: ORCL:ASMDISK2 [62912512 blocks (32211206144 bytes), maxio 1024, integrity none]
The above steps might seem daunting and a lot to do, but it is not that difficult and once you have done this once or twice it becomes really easy and things start making more sense.
We are ready to start the installation of Oracle Clusterware.
Step 3: Installing Oracle Clusterware
Now we are getting to the fun stuff. Make sure you have completed the pre-requisite steps and that you have your networking and DNS configured as well as that the grid Unix account can ssh between nodes without a password or passphrase.
On both nodes, let's pre-create a few directories, this is where we will install Oracle Clusterware.
As the root user:
cd /u01 mkdir -p app cd app mkdir grid 19.3.0 oracle cd /u01 chown -R grid:oinstall app chmod -R 775 app cd app chown -R oracle:oinstall oracle
In summary under the /u01 folder (mount point) on both systems you should have the following directory structure:
grid@oel7clus1 /u01 : tree app app ├── 19.3.0 ├── grid └── oracle 3 directories, 0 files
Now, as the “grid” Unix user, on only node 1 (oel7clus1) extract the Grid Infrastructure software into the /u01/app/19.3.0 folder using the “unzip -q” command.
In my example, I have the software available as zip files in /install
grid@oel7clus1 /u01/app/19.3.0 : unzip -q /install/LINUX.X64_193000_grid_home.zip grid@oel7clus1 /u01/app/19.3.0 : grid@oel7clus1 /u01/app/19.3.0 : ls addnode cv env.ora instantclient lib ord precomp root.sh sqlpatch utl assistants dbjava evm inventory md ords QOpatch root.sh.old sqlplus welcome.html bin dbs gpnp javavm network oss qos root.sh.old.1 srvm wlm cha deinstall gridSetup.sh jdbc nls oui racg rootupgrade.sh suptools wwg clone demo has jdk OPatch owm rdbms runcluvfy.sh tomcat xag crs diagnostics hs jlib opmn perl relnotes sdk ucp xdk css dmu install ldap oracore plsql rhp slax usm grid@oel7clus1 /u01/app/19.3.0 :
Tip:
Make sure you install the cvpudisk package located in the cv/rpm subfolder on both cluster nodes. As sudo is allowed for the grid user, you can just use the following commands to do this
grid@oel7clus1 /u01/app/19.3.0 : cd cv/rpm/ grid@oel7clus1 /u01/app/19.3.0/cv/rpm : ls cvuqdisk-1.0.10-1.rpm grid@oel7clus1 /u01/app/19.3.0/cv/rpm : sudo rpm -ivh cvuqdisk-1.0.10-1.rpm Preparing... ################################# [100%] package cvuqdisk-1.0.10-1.x86_64 is already installed
Now in my configuration, I already installed the package on both nodes, but you might have to install this package on both your nodes. You can “scp” the file from the primary node where it is extracted into the second node /tmp and install from there. Do not extract the full Grid install (zip) file on the second node, the software will be copied and configured there during the installer process which we will run next.
Starting the Installer:
To start the installer, as the “grid” user, navigate to the ORACLE_HOME location for Clusterware which is /u01/app/19.3.0 in this example. Then run “./gridSetup.sh”
The steps and screenshots below will take you through the configuration of Oracle Clusterware on a standalone cluster.
grid@oel7clus1 /u01/app/19.3.0 : ./gridSetup.sh Launching Oracle Grid Infrastructure Setup Wizard...
- As this is a new installation we pick Configure for a new cluster and click on next
- The next screen as seen below we select a Standalone Cluster and click next to continue
- Now are getting to the next step where you need to supply the cluster name as well as the SCAN name and port, remember you have to have these in the DNS server.
- For more detail on SCAN and SCAN listeners please see the Oracle Documentation as this is a big topic in itself
- Next, we get the screen where we need to supply the second node in the cluster’s details, this would be the public hostname as well as the Virtual IP/Hostname.
- Once you have supplied the details - and once again, these should be in your DNS server and resolvable.
- Once you have added the details you will see the summary page, now click next to continue
- Next, we get the option to specify which subnets to use as Public or ASM or Private. In my example I will use the one network interface (also used as the interconnect between the two servers) for both ASM but also as the private link.
- Next we are going to put the OCR and Voting disk on ASM, so we select Oracle Flex ASM for Storage
- Next, we get the option to not use the Grid Infrastructure Management repository. Now as this is a test configuration, I do not need this and will leave it out as this will save a little on resources, but for production configurations, you most likely will enable this as with it there are a number of benefits as listed on the screen.
- Now we get to the Disk Groups, we first give the 1st Disk Group the name OCR_VOT (which is just a name I gave it to note this Disk Group is used for the OCR and Voting disks. It is a small disk and I select redundancy to external - you might want to review the Oracle Documentation on the redundancy levels. Note that depending on your redundancy level you may require twice the disk space.
- It might be that you do not see any disks to select (marked 5 below) - if that is the case, select “Change Discovery Path” marked with 3 below and set the Discovery Path to ORCL:* or whatever the path might be on your system, but as I am using ASMLIB in this example, I can use this path and it should then pick up the provisioned disks.
- I then select the first smaller disk only and click on next to continue.
- Next you need to specify the ASM instance passwords. Now you should make sure you give it a complex password and depending on your configuration you may want to use various different passwords for the SYS and ASMSNMP accounts.
- In this example, I will use the same password for both
- I am not using IPMI so click next to continue
- In my configuration, I am not using Enterprise Manager, so I leave this step out and click next.
- Next specify the Groups as seen below and click next.
- My default groups that I have created and assigned to the grid UNIX user was automatically picked up.
- Next confirm the Oracle Base location, which I leave as the default /u01/app/grid
- Next I do want to execute the configuration scripts automatically - these are run as the root user, so you will need to specify the root password, or if you have “sudo” configured you can use that as well.
- Next verification will run to make sure you can continue
- In my configuration I did get a few checks that failed, after review, I found that my ASM devices were correct even though it was reported, the resolv.conf file where my DNS configuration is pointed was also correct. The Memory and swap space will specify recommended values, I am fine with the values on my test system and selected “Ignore All” and continued. In production configurations, you do want to make sure you meet the requirements.
- You will get to a summary screen from where you can then start the installation configuration. Once you start this the process can take time.
- After a while, you will be prompted to confirm you still want to run the configuration scripts as the root user.
- I do find this strange, as I did agree to this a few screens before, but all you have to do is click yes and the installer continues. Again note that this process can take time.
- Once everything is configured without any issues you will be directed to the last “Finish” screen, you can now click close.
Excellent, installation complete and I can quickly verify this by seeing ASM instances running on both my cluster nodes.
I can use the “crsctl status resource” command to get full details on my cluster resources, for example:
grid@oel7clus1 /u01/app/19.3.0 : crsctl status resource -t -------------------------------------------------------------------------------- Name Target State Server State details -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.LISTENER.lsnr ONLINE ONLINE oel7clus1 STABLE ONLINE ONLINE oel7clus2 STABLE ora.chad ONLINE ONLINE oel7clus1 STABLE ONLINE ONLINE oel7clus2 STABLE ora.net1.network ONLINE ONLINE oel7clus1 STABLE ONLINE ONLINE oel7clus2 STABLE ora.ons ONLINE ONLINE oel7clus1 STABLE ONLINE ONLINE oel7clus2 STABLE -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup) 1 ONLINE ONLINE oel7clus1 STABLE 2 ONLINE ONLINE oel7clus2 STABLE 3 OFFLINE OFFLINE STABLE ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE oel7clus2 STABLE ora.LISTENER_SCAN2.lsnr 1 ONLINE ONLINE oel7clus1 STABLE ora.LISTENER_SCAN3.lsnr 1 ONLINE ONLINE oel7clus1 STABLE ora.OCR_VOT.dg(ora.asmgroup) 1 ONLINE ONLINE oel7clus1 STABLE 2 ONLINE ONLINE oel7clus2 STABLE 3 OFFLINE OFFLINE STABLE ora.asm(ora.asmgroup) 1 ONLINE ONLINE oel7clus1 Started,STABLE 2 ONLINE ONLINE oel7clus2 Started,STABLE 3 OFFLINE OFFLINE STABLE ora.asmnet1.asmnetwork(ora.asmgroup) 1 ONLINE ONLINE oel7clus1 STABLE 2 ONLINE ONLINE oel7clus2 STABLE 3 OFFLINE OFFLINE STABLE ora.cvu 1 ONLINE ONLINE oel7clus1 STABLE ora.oel7clus1.vip 1 ONLINE ONLINE oel7clus1 STABLE ora.oel7clus2.vip 1 ONLINE ONLINE oel7clus2 STABLE ora.qosmserver 1 ONLINE ONLINE oel7clus1 STABLE ora.scan1.vip 1 ONLINE ONLINE oel7clus2 STABLE ora.scan2.vip 1 ONLINE ONLINE oel7clus1 STABLE ora.scan3.vip 1 ONLINE ONLINE oel7clus1 STABLE -------------------------------------------------------------------------------- grid@oel7clus1 /u01/app/19.3.0 :
Now that we have Oracle Clusterware installed, we will need one more thing before we can use ACFS - we need to apply the latest patch update to get the Clusterware software up to date and to make sure that the Linux Kernel is supported for running ACFS filesystem. For more detail on ACFS and required patches to match kernel versions please see Oracle Support. Now as I am using one of the latest Oracle Linux 7 updates, I do need to apply patches to allow Oracle Clusterware to use ACFS. Do not be scared, this is not a difficult process and in the next step, I will quickly take you through this step.
But before I do this, a quick way to see if you can create ACFS filesystems is to run the “asmca” utility - and as you can see in my configuration I cannot see ACFS or ADMV options at all, so lets first install the patch.
Step 4: Apply Patch to Oracle Clusterware (if required)
As mentioned in my configuration as I am using the latest Oracle Linux 7 with all its updates I will need to apply the latest Oracle Grid Infrastructure patch to make sure I can use ACFS - which I would like to show you in this example.
Now the patch I will be applying is patch number: 29708769
Please see Oracle support for more detail on this patch.
I have downloaded the patch and it is shared via NFS folder on both my nodes. After reviewing and running the required Patch tests I applied the patch first to node 1 then once completed to node 2 with the following commands:
root@oel7clus1 ~ : . oraenv ORACLE_SID = [root] ? +ASM1 The Oracle base has been set to /u01/app/grid root@oel7clus1 ~ : ls /u01/app 19.3.0 grid oracle oraInventory root@oel7clus1 ~ : export PATH=/u01/app/19.3.0/OPatch:$PATH root@oel7clus1 ~ : which opatch /u01/app/19.3.0/OPatch/opatch root@oel7clus1 ~ : cd root@oel7clus1 ~ : opatchauto apply /patch/patches/ 29708769/ PatchSearch.xml root@oel7clus1 ~ : opatchauto apply /patch/patches/29708769 -oh /u01/app/19.3.0 OPatchauto session is initiated at Sun Sep 1 16:14:04 2019 System initialization log file is /u01/app/19.3.0/cfgtoollogs/opatchautodb/systemconfig2019-09-01_04-14-07PM.log. Session log file is /u01/app/19.3.0/cfgtoollogs/opatchauto/opatchauto2019-09-01_04-14-16PM.log The id for this session is QTRD Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.3.0 Patch applicability verified successfully on home /u01/app/19.3.0 Bringing down CRS service on home /u01/app/19.3.0 CRS service brought down successfully on home /u01/app/19.3.0 Start applying binary patch on home /u01/app/19.3.0 Binary patch applied successfully on home /u01/app/19.3.0 Starting CRS service on home /u01/app/19.3.0 CRS service started successfully on home /u01/app/19.3.0 OPatchAuto successful. --------------------------------Summary-------------------------------- Patching is completed successfully. Please find the summary as follows: Host:oel7clus1 CRS Home:/u01/app/19.3.0 Version:19.0.0.0.0 Summary: ==Following patches were SKIPPED: Patch: /patch/patches/29708769/29401763 Reason: This patch is already been applied, so not going to apply again. ==Following patches were SUCCESSFULLY applied: Patch: /patch/patches/29708769/29834717 Log: /u01/app/19.3.0/cfgtoollogs/opatchauto/core/opatch/opatch2019-09-01_16-17-15PM_1.log Patch: /patch/patches/29708769/29850993 Log: /u01/app/19.3.0/cfgtoollogs/opatchauto/core/opatch/opatch2019-09-01_16-17-15PM_1.log Patch: /patch/patches/29708769/29851014 Log: /u01/app/19.3.0/cfgtoollogs/opatchauto/core/opatch/opatch2019-09-01_16-17-15PM_1.log OPatchauto session completed at Sun Sep 1 16:27:04 2019 Time taken to complete the session 13 minutes, 1 second root@oel7clus1 ~ :
Then follow the same steps on the second node:
root@oel7clus2 /patch : . oraenv ORACLE_SID = [root] ? +ASM2 The Oracle base has been set to /u01/app/grid root@oel7clus2 /patch : export PATH=/u01/app/19.3.0/OPatch:$PATH root@oel7clus2 /patch : which opatch /u01/app/19.3.0/OPatch/opatch root@oel7clus2 /patch : opatchauto apply /patch/patches/29708769 -oh /u01/app/19.3.0 OPatchauto session is initiated at Sun Sep 1 17:05:18 2019 System initialization log file is /u01/app/19.3.0/cfgtoollogs/opatchautodb/systemconfig2019-09-01_05-05-21PM.log. Session log file is /u01/app/19.3.0/cfgtoollogs/opatchauto/opatchauto2019-09-01_05-05-31PM.log The id for this session is 6418 Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.3.0 Patch applicability verified successfully on home /u01/app/19.3.0 Bringing down CRS service on home /u01/app/19.3.0 CRS service brought down successfully on home /u01/app/19.3.0 Start applying binary patch on home /u01/app/19.3.0 Binary patch applied successfully on home /u01/app/19.3.0 Starting CRS service on home /u01/app/19.3.0 CRS service started successfully on home /u01/app/19.3.0 OPatchAuto successful. --------------------------------Summary-------------------------------- Patching is completed successfully. Please find the summary as follows: Host:oel7clus2 CRS Home:/u01/app/19.3.0 Version:19.0.0.0.0 Summary: ==Following patches were SKIPPED: Patch: /patch/patches/29708769/29401763 Reason: This patch is already been applied, so not going to apply again. ==Following patches were SUCCESSFULLY applied: Patch: /patch/patches/29708769/29834717 Log: /u01/app/19.3.0/cfgtoollogs/opatchauto/core/opatch/opatch2019-09-01_17-08-19PM_1.log Patch: /patch/patches/29708769/29850993 Log: /u01/app/19.3.0/cfgtoollogs/opatchauto/core/opatch/opatch2019-09-01_17-08-19PM_1.log Patch: /patch/patches/29708769/29851014 Log: /u01/app/19.3.0/cfgtoollogs/opatchauto/core/opatch/opatch2019-09-01_17-08-19PM_1.log OPatchauto session completed at Sun Sep 1 17:19:08 2019 Time taken to complete the session 13 minutes, 50 seconds root@oel7clus2 /patch :
Step 5: Adding the DATA Diskgroup and ACFS Filesystem
Now that patching is complete, we can review the status of the “asmca” utility to see if we get the options for ADMV and ACFS.
On the first node, I will start “asmca” as the grid user, the following screen will show:
As you can see you now get options to configure Volumes and ACFS filesystems.
But before we get into that, we need to do one thing, we need to create a new ASM diskgroup using our 30GB disk that we shared between the two nodes.
To do this we navigate to Disk Groups and click on “Create"
- Next we specify the Disk group name as “DATA”, again select redundancy as external (none) and then specify the larger disk (ASMDISK2 in this example) then click on OK to start the Disk Group creation which can take a minute.
- When done you will see both your Disk Groups listed and “Mounted” on both systems - meaning it is available on both systems now.
- Next we want to create a new Volume - I navigate to “Volumes” under Disk Groups and add the details as seen below and click on OK to create it.
- Next I can now create the cluster file system - ACFS. Navigate to ACFS File Systems and click on create.
- Next fill in the details as seen below:
- 1) is the mount point I will use between the two systems in my case - /u02
- 2) I specify to auto mount this
- 3) I want to make sure the Unix “oracle” user owns this folder, so I add the “oracle” user as the owner and the primary group as “oinstall”
- 4) I just add a description, as I will install the Oracle 19c SE2 database software into /u02 I just specify Oracle Software
- 6) I select the volume I want to create the filesystem on (created above)
- 7) Automatically run the required configuration commands are selected and then click on OK (8) to continue
- This process can take a few minutes, but once done you will see something like below and the mounted is 2 of 2 - meaning that the /u02 filesystem is now mounted on both my cluster nodes. Excellent!
Now when looking at the two hosts I can see the shared storage /u02 folder mounted
grid@oel7clus1 ~ : df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 3.8G 0 3.8G 0% /dev tmpfs 3.8G 845M 3.0G 22% /dev/shm tmpfs 3.8G 257M 3.6G 7% /run tmpfs 3.8G 0 3.8G 0% /sys/fs/cgroup /dev/sda3 6.3G 3.7G 2.3G 63% / tmpfs 4.0G 31M 4.0G 1% /tmp /dev/sda1 488M 279M 174M 62% /boot /dev/mapper/oravg-u01 40G 10G 28G 27% /u01 tmpfs 771M 0 771M 0% /run/user/0 /dev/sdd1 20G 7.3G 12G 40% /install tmpfs 771M 0 771M 0% /run/user/502 /dev/asm/acfsvol1-145 25G 600M 25G 3% /u02
Step 6: Installing the Oracle Database SE2 Software
Now we get to the part where we will install the Oracle Database SE2 software into the shared storage /u02 location.
Now it is possible to install the database software on the local disk on both the cluster nodes, example, have the exact same install in /u01/app/oracle/product/19.3.0/dbhome_1 - or you can make use of a shared Oracle Home and have the Oracle Database software installed on the Shared Disk - meaning on /u02.
In this example I will make use of the second option, and install the Oracle Database software in /u02 filesystem.
As the “oracle” Unix account, first create the path for the Oracle Database 19c SE2 software home folder, then extract the downloaded install file into this location, example:
oracle@oel7clus1 ~ : cd /u02 oracle@oel7clus1 /u02 : ls lost+found oracle@oel7clus1 /u02 : mkdir -p app/oracle/product/19.3.0/dbhome_1 oracle@oel7clus1 /u02 : cd app/oracle/product/19.3.0/dbhome_1/ oracle@oel7clus1 /u02/app/oracle/product/19.3.0/dbhome_1 : ls oracle@oel7clus1 /u02/app/oracle/product/19.3.0/dbhome_1 : unzip -q /install/LINUX.X64_193000_db_home.zip oracle@oel7clus1 /u02/app/oracle/product/19.3.0/dbhome_1 : ls addnode bin css data deinstall dmu env.ora install javavm jlib md nls OPatch ord oui plsql R relnotes root.sh.old.1 sdk sqlj srvm usm xdk apex clone ctx dbjava demo drdaas has instantclient jdbc ldap mgw odbc opmn ords owm precomp racg root.sh runInstaller slax sqlpatch suptools utl assistants crs cv dbs diagnostics dv hs inventory jdk lib network olap oracore oss perl QOpatch rdbms root.sh.old schagent.conf sqldeveloper sqlplus ucp wwg oracle@oel7clus1 /u02/app/oracle/product/19.3.0/dbhome_1 :
The next step is to start the Oracle 19c Database Installer.
oracle@oel7clus1 /u02/app/oracle/product/19.3.0/dbhome_1 : ./runInstaller Launching Oracle Database Setup Wizard...
- The installer start and I select to install the software only. I will create the database afterwards
- Next I want to make sure I do not select Real. Application Cluster (RAC) - but Single Instance, as this is going to be an Oracle SE2 installation where Oracle RAC is not available.
- Next we get the Database Edition - we select Standard Edition 2 and continue
- Next we make sure we specify the Oracle Base location as the shared folder /u02 and in this case the path is /u02/app/oracle
- Next the “oracle” Unix user groups are detected and auto-populated.
- Again I want to automatically run the configuration as the root user and supply the details
- Next the prerequisites complain about my swap space, I am ok with that and click on “ignore all” and continue
- We get the summary screen and continue with the installation
- This step can take time, you will be asked again to run the configuration - click yes to continue.
- Once the installation is done you will get the Finish screen. You have now successfully installed Oracle Database 19c SE2!
Step 7: Creating the Database using DBCA
The next step is to create the Single Instance database on the shared storage location - /u02
Now you can use the Graphical Interface of the “Database Configuration Assistant” - DBCA to do this, but in this example I just use the silent option from the command line which is actually really easy to use - since 12c Oracle made some really good changes to the command line option and using these are now super easy to create new databases.
In the command below I will create a database, make sure the datafiles are using Oracle Managed Files (OMF), enable archive log mode (highly recommended!) and I specify the recovery area to also be on the shared storage /u02 location. I am doing this so that in this example my whole database installation and datafiles are all on the shared storage. In this example I am also choosing not to use Multitenant.
dbca -createDatabase \ -silent \ -automaticMemoryManagement false \ -characterSet AL32UTF8 \ -createAsContainerDatabase false \ -datafileDestination /u02/oracle/oradata \ -enableArchive true \ -gdbName DEV \ -nationalCharacterSet AL16UTF16 \ -recoveryAreaDestination /u02/oracle/fast_recovery_area \ -redoLogFileSize 100 \ -storageType fs \ -sysPassword Kiwi123 \ -systemPassword Kiwi123 \ -templateName General_Purpose.dbc \ -totalMemory 850 \ -useOMF true
When I execute the above, I get the following output:
oracle@oel7clus1 /u02/app/oracle/product/19.3.0/dbhome_1 : . oraenv ORACLE_SID = [oracle] ? a ORACLE_HOME = [/home/oracle] ? /u02/app/oracle/product/19.3.0/dbhome_1 The Oracle base has been changed from /u01/app/oracle to /u02/app/oracle oracle@oel7clus1 /u02/app/oracle/product/19.3.0/dbhome_1 : dbca -createDatabase \ oracle@oel7clus1 --> -silent \ oracle@oel7clus1 --> -automaticMemoryManagement false \ oracle@oel7clus1 --> -characterSet AL32UTF8 \ oracle@oel7clus1 --> -createAsContainerDatabase false \ oracle@oel7clus1 --> -datafileDestination /u02/oracle/oradata \ oracle@oel7clus1 --> -enableArchive true \ oracle@oel7clus1 --> -gdbName DEV \ oracle@oel7clus1 --> -nationalCharacterSet AL16UTF16 \ oracle@oel7clus1 --> -recoveryAreaDestination /u02/oracle/fast_recovery_area \ oracle@oel7clus1 --> -redoLogFileSize 100 \ oracle@oel7clus1 --> -storageType fs \ oracle@oel7clus1 --> -sysPassword Kiwi123 \ oracle@oel7clus1 --> -systemPassword Kiwi123 \ oracle@oel7clus1 --> -templateName General_Purpose.dbc \ oracle@oel7clus1 --> -totalMemory 850 \ oracle@oel7clus1 --> -useOMF true [WARNING] [DBT-06208] The 'SYS' password entered does not conform to the Oracle recommended standards. CAUSE: a. Oracle recommends that the password entered should be at least 8 characters in length, contain at least 1 uppercase character, 1 lower case character and 1 digit [0-9]. b.The password entered is a keyword that Oracle does not recommend to be used as password ACTION: Specify a strong password. If required refer Oracle documentation for guidelines. [WARNING] [DBT-06208] The 'SYSTEM' password entered does not conform to the Oracle recommended standards. CAUSE: a. Oracle recommends that the password entered should be at least 8 characters in length, contain at least 1 uppercase character, 1 lower case character and 1 digit [0-9]. b.The password entered is a keyword that Oracle does not recommend to be used as password ACTION: Specify a strong password. If required refer Oracle documentation for guidelines. Prepare for db operation 10% complete Registering database with Oracle Restart 14% complete Copying database files 43% complete Creating and starting Oracle instance 45% complete 49% complete 53% complete 56% complete 62% complete Completing Database Creation 68% complete 70% complete 71% complete Executing Post Configuration Actions 100% complete Database creation complete. For details check the logfiles at: /u02/app/oracle/cfgtoollogs/dbca/DEV. Database Information: Global Database Name:DEV System Identifier(SID):DEV Look at the log file "/u02/app/oracle/cfgtoollogs/dbca/DEV/DEV.log" for further details. oracle@oel7clus1 /u02/app/oracle/product/19.3.0/dbhome_1 :
The above process can take a few minutes depending on your hardware, but once done, you have an Oracle Database 19c SE2 running from the Shared Storage on node one in the cluster! You can check this with the srvctl status option as shown below.
oracle@oel7clus1 /u02/app/oracle/product/19.3.0/dbhome_1 : srvctl status database -d DEV -v Instance DEV is running on node oel7clus1. Instance status: Open.
Now this does not stop there, there is a lot more to do, but you are almost there.
Next we need to make a few changes to allow for the Active/Passive switch of our services
Step 8: Adding a Virtual IP
Adding a Virtual IP that will be used by Dbvisit Standby and we can also build a dependancy on this IP for the Database and Dbvisit Standby resources to be added later. Meaning - if you relocate or switch the VIP to the other node, the resources will be stopped and started on the other node.
You could also use this IP address and create a new dedicated listener for the database. Now the IP address we will use in this example is 10.0.2.28 - it is added to the DNS and is known as "oel7clus-dbvip"
The following commands are used to create the Virtual IP:
root@oel7clus1 ~ : which appvipcfg
/u01/app/19.3.0/bin/appvipcfg
root@oel7clus1 ~ :
root@oel7clus1 ~ : appvipcfg create -network=1 -ip=10.0.2.28 -vipname=oel7clus-dbvip -user=root
Using configuration parameter file: /u01/app/19.3.0/crs/install/crsconfig_params
The log of current session can be found at:
/u01/app/grid/crsdata/oel7clus1/scripts/appvipcfg.log
root@oel7clus1 ~ : crsctl setperm resource oel7clus-dbvip -u user:oracle:r-x
root@oel7clus1 ~ :
root@oel7clus1 ~ : crsctl status resource oel7clus-dbvip -v
NAME=oel7clus-dbvip
TYPE=app.appviptypex2.type
LAST_SERVER=
STATE=OFFLINE
TARGET=OFFLINE
CARDINALITY_ID=1
OXR_SECTION=0
RESTART_COUNT=0
FAILURE_COUNT=0
FAILURE_HISTORY=
ID=oel7clus-dbvip 1 1
INCARNATION=0
LAST_RESTART=NEVER
LAST_STATE_CHANGE=NEVER
STATE_DETAILS=
INTERNAL_STATE=STABLE
TARGET_SERVER=
RESOURCE_GROUP=
INSTANCE_COUNT=1
root@oel7clus1 ~ : crsctl start resource oel7clus-dbvip -n oel7clus1
CRS-2672: Attempting to start 'oel7clus-dbvip' on 'oel7clus1'
CRS-2676: Start of 'oel7clus-dbvip' on 'oel7clus1' succeeded
root@oel7clus1 ~ :
root@oel7clus1 ~ : crsctl status resource oel7clus-dbvip -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
oel7clus-dbvip
1 ONLINE ONLINE oel7clus1 STABLE
--------------------------------------------------------------------------------
root@oel7clus1 ~ :
Now that we have the Virtual IP running on node 1 (oel7clus1) we can move onto the next step which is to modify the database resource.
Step 9: The Database Resource Modifications
One of the next challenges you will run into with the above configuration is when you try and fail-over or relocate the database instance from the first node in the cluster to the second node.
As the DBCA register the database as a resource, you can view the database details using the srvctl utility. Example:
oracle@oel7clus1 ~ : srvctl config database -d DEV Database unique name: DEV Database name: DEV Oracle home: /u02/app/oracle/product/19.3.0/dbhome_1 Oracle user: oracle Spfile: /u02/app/oracle/product/19.3.0/dbhome_1/dbs/spfileDEV.ora Password file: Domain: Start options: open Stop options: immediate Database role: PRIMARY Management policy: AUTOMATIC Server pools: Disk Groups: Mount point paths: /u02 Services: Type: SINGLE OSDBA group: dba OSOPER group: oper Database instance: DEV Configured nodes: oel7clus1 CSS critical: no CPU count: 0 Memory target: 0 Maximum memory: 0 Default network number for database services: Database is administrator managed oracle@oel7clus1 ~ :
Now if you just attempt to perform a relocate you will get the following error:
oracle@oel7clus1 ~ : srvctl relocate database -d DEV -n oel7clus2 PRCD-1146 : Database DEV is not a RAC One Node database
Ok, so we are not using Oracle RAC and we are not using Oracle RAC One Node, we just have a stock standard single instance database that was created on shared storage.
So what can we do now? You might think that you are now stuck, but there is actually another option here. You can remove the database resource, and re-add it as standard resource.
Lets first look at the resource as it is now:
oracle@oel7clus1 ~ : . oraenv ORACLE_SID = [DEV] ? +ASM1 The Oracle base has been changed from /u02/app/oracle to /u01/app/grid oracle@oel7clus1 ~ : crsctl status resource ora.dev.db -v NAME=ora.dev.db TYPE=ora.database.type LAST_SERVER=oel7clus1 STATE=ONLINE on oel7clus1 TARGET=ONLINE CARDINALITY_ID=1 OXR_SECTION=0 RESTART_COUNT=0 FAILURE_COUNT=0 FAILURE_HISTORY= ID=ora.dev.db 1 1 INCARNATION=2 LAST_RESTART=09/10/2019 11:04:31 LAST_STATE_CHANGE=09/10/2019 11:04:31 STATE_DETAILS=Open,HOME=/u02/app/oracle/product/19.3.0/dbhome_1 INTERNAL_STATE=STABLE TARGET_SERVER=oel7clus1 RESOURCE_GROUP= INSTANCE_COUNT=1
As you can see the type is "ora.database.type"
What we can do now is remove this resource, and then create our own custom resource to allow for the database instance to be moved between the nodes.
This will require a few steps to get this done.
- Remove the Database Resource (Outage Required)
- First we stop the database, then we remove it as a resource. This can be done with the srvctl utility.
Below are the commands executed to perform these tasks:
oracle@oel7clus1 ~ : srvctl stop database -d DEV oracle@oel7clus1 ~ : srvctl status database -d DEV Instance DEV is not running on node oel7clus1 oracle@oel7clus1 ~ : srvctl remove database -d DEV Remove the database DEV? (y/[n]) y oracle@oel7clus1 ~ : oracle@oel7clus1 ~ : srvctl status database -d DEV PRCD-1120 : The resource for database DEV could not be found. PRCR-1001 : Resource ora.dev.db does not exist
- Next we need to create an "Action Script"
- This is a script that will be executed by Clusterware to start and stop the cluster resource. Now you can make this complex, or you can keep it simple, which is usually the best way to start.
Here is a sample action script used:
#!/bin/bash # # SAMPLE Action Script to start an Oracle Database ############################################## # Function to change database environments # Description: # Use oraenv to set the environment if needed # This is optional to set the environment ############################################## ## set following to ensure oraenv is picked up from /usr/local/bin export PATH=/usr/local/bin:$PATH export NLS_DATE_FORMAT='DD:MM:YYYY:HH24:MI:SS' ## Setting environment for the Oracle Instance set_env () { export ORAENV_ASK=NO export ORACLE_SID=$1 . oraenv >> /dev/null export ORAENV_ASK=YES } ################# ## Main Section ################# # This is logged to CRSD agent log file echo "`date` Action script '$_CRS_ACTION_SCRIPT' for resource [$_CRS_NAME] called for action $1" # set environment set_env DEV case "$1" in 'start') sqlplus -s /nolog <<!EOF connect / as sysdba startup; exit; !EOF RET=0 echo "Running start DEV database resource with return code $RET" ;; 'stop') sqlplus -s /nolog <<!EOF connect / as sysdba shutdown immediate; exit; !EOF RET=0 echo "Running stop DEV database resource with return code $RET" ;; 'check') NUM=`ps -ef | grep ora_pmon_DEV | grep -v grep | wc -l` if [ $NUM = 0 ]; then ## return code 1 for check means OFFLINE RET=1 else ## return code 0 for check means ONLINE RET=0 fi echo "Running check DEV database resource with return code $RET" ;; 'clean') sqlplus -s /nolog <<!EOF connect / as sysdba shutdown abort; exit; !EOF RET=0 echo "Running clean DEV database resource with return code $RET" ;; *) RET=1 echo "Usage: db-crs.sh start|stop|check|clean" ;; esac if [ $RET -eq 0 ]; then exit 0 else exit 1 fi- Now we need to place this script on the shared storage as it needs to be available for execution from either one of the cluster nodes. In this example I used "/u02/app/oracle/admin/DEV" - now remember /u02 is my shared ACFS filesystem that is mounted on both nodes
- Here are the steps:
oracle@oel7clus1 /u02/app/oracle/admin/DEV : mkdir scripts oracle@oel7clus1 /u02/app/oracle/admin/DEV : cd scripts oracle@oel7clus1 /u02/app/oracle/admin/DEV/scripts : vi db-crs.sh oracle@oel7clus1 /u02/app/oracle/admin/DEV/scripts : chmod u+x db-crs.sh oracle@oel7clus1 /u02/app/oracle/admin/DEV/scripts : ./db-crs.sh Tue Sep 10 11:44:33 NZST 2019 Action script '' for resource [] called for action Usage: db-crs.sh start|stop|check|clean
- Now remember my database DEV is down at the moment, but I can always use this script to start and stop it quickly as a test.
Now after the below I am happy that my script is executing correct, and yes I could add a lot more checks and output to the script, but as a basic starting point this will do.
oracle@oel7clus1 /u02/app/oracle/admin/DEV/scripts : ./db-crs.sh start Tue Sep 10 11:46:41 NZST 2019 Action script '' for resource [] called for action start ORACLE instance started. Total System Global Area 671088184 bytes Fixed Size 8900152 bytes Variable Size 197132288 bytes Database Buffers 457179136 bytes Redo Buffers 7876608 bytes Database mounted. Database opened. Running start DEV database resource with return code 0 oracle@oel7clus1 /u02/app/oracle/admin/DEV/scripts : ./db-crs.sh check Tue Sep 10 11:47:07 NZST 2019 Action script '' for resource [] called for action check Running check DEV database resource with return code 0 oracle@oel7clus1 /u02/app/oracle/admin/DEV/scripts : ./db-crs.sh stop Tue Sep 10 11:47:12 NZST 2019 Action script '' for resource [] called for action stop Database closed. Database dismounted. ORACLE instance shut down. Running stop DEV database resource with return code 0 oracle@oel7clus1 /u02/app/oracle/admin/DEV/scripts :
- Next, create a cluster resource file with the details of the new resource we want to add. This just simplify the options to be passed in via the command line. You could do it all as one big command, but why not use a file that you can just reference and then easily update to your needs where required.
- The following example was used. You will see that it states a number of key parameters such as pointing at the action script to be used (created above).
Now you can go and review these parameters in the Oracle Documentation to find the right values for your configuration, but I am starting simple here to get you going. Remember always test on a test environment first before you do things in production, that way you can test various different options without affecting production, then once you find the parameters that work for you, and they are tested, you can then execute it in production.
- You might want to review the START_DEPENDENCIES and STOP_DEPENDENCIES as you might want to specify different values here to meet your requirements.
NAME = DEV.db TYPE = cluster_resource DESCRIPTION = Single Instance Oracle Database resource ACL = owner:oracle:rwx,pgrp:oinstall:rwx,other::r-- ACTION_SCRIPT = /u02/app/oracle/admin/DEV/scripts/db-crs.sh PLACEMENT = restricted ACTIVE_PLACEMENT = 1 AUTO_START = restore CARDINALITY = 1 CHECK_INTERVAL = 10 DEGREE = 1 ENABLED = 1 HOSTING_MEMBERS = oel7clus1 oel7clus2 LOGGING_LEVEL = 1 RESTART_ATTEMPTS = 1 START_DEPENDENCIES = hard (ora.data.acfsvol1.acfs,oel7clus-dbvip) weak (type:ora.listener.type,global:type:ora.scan_listener.type,uniform:ora.ons) pullup (ora.data.acfsvol1.acfs) START_TIMEOUT = 600 STOP_DEPENDENCIES = hard (shutdown:ora.data.acfsvol1.acfs, oel7clus-dbvip) STOP_TIMEOUT = 600 UPTIME_THRESHOLD = 1h
- Adding and starting the resource
- Next we want to add the new resource and once done start it:
oracle@oel7clus1 /u02/app/oracle/admin/DEV/scripts : crsctl add resource DEV.db -type cluster_resource -file /u02/app/oracle/admin/DEV/scripts/add-DEV-resource.sc oracle@oel7clus1 /u02/app/oracle/admin/DEV/scripts : crsctl status resource DEV.db NAME=DEV.db TYPE=cluster_resource TARGET=OFFLINE STATE=OFFLINE oracle@oel7clus1 /u02/app/oracle/admin/DEV/scripts : crsctl status resource DEV.db -v NAME=DEV.db TYPE=cluster_resource LAST_SERVER= STATE=OFFLINE TARGET=OFFLINE CARDINALITY_ID=1 OXR_SECTION=0 RESTART_COUNT=0 FAILURE_COUNT=0 FAILURE_HISTORY= ID=DEV.db 1 1 INCARNATION=0 LAST_RESTART=NEVER LAST_STATE_CHANGE=NEVER STATE_DETAILS= INTERNAL_STATE=STABLE TARGET_SERVER= RESOURCE_GROUP= INSTANCE_COUNT=1 oracle@oel7clus1 /u02/app/oracle/admin/DEV/scripts : crsctl start resource DEV.db CRS-2672: Attempting to start 'DEV.db' on 'oel7clus1' CRS-2676: Start of 'DEV.db' on 'oel7clus1' succeeded oracle@oel7clus1 /u02/app/oracle/admin/DEV/scripts : crsctl status resource DEV.db -v NAME=DEV.db TYPE=cluster_resource LAST_SERVER=oel7clus1 STATE=ONLINE on oel7clus1 TARGET=ONLINE CARDINALITY_ID=1 OXR_SECTION=0 RESTART_COUNT=0 FAILURE_COUNT=0 FAILURE_HISTORY= ID=DEV.db 1 1 INCARNATION=1 LAST_RESTART=09/10/2019 12:03:17 LAST_STATE_CHANGE=09/10/2019 12:03:17 STATE_DETAILS= INTERNAL_STATE=STABLE TARGET_SERVER=oel7clus1 RESOURCE_GROUP= INSTANCE_COUNT=1
- Now if we review the resources after the above we can see the following:
oracle@oel7clus1 /u02/app/oracle/admin/DEV/scripts : crsctl status resource -t -------------------------------------------------------------------------------- Name Target State Server State details -------------------------------------------------------------------------------- Local Resources -------------------------------------------------------------------------------- ora.DATA.ACFSVOL1.advm ONLINE ONLINE oel7clus1 STABLE ONLINE ONLINE oel7clus2 STABLE ora.LISTENER.lsnr ONLINE ONLINE oel7clus1 STABLE ONLINE ONLINE oel7clus2 STABLE ora.chad ONLINE ONLINE oel7clus1 STABLE ONLINE ONLINE oel7clus2 STABLE ora.data.acfsvol1.acfs ONLINE ONLINE oel7clus1 mounted on /u02,STAB LE ONLINE ONLINE oel7clus2 mounted on /u02,STAB LE ora.net1.network ONLINE ONLINE oel7clus1 STABLE ONLINE ONLINE oel7clus2 STABLE ora.ons ONLINE ONLINE oel7clus1 STABLE ONLINE ONLINE oel7clus2 STABLE ora.proxy_advm ONLINE ONLINE oel7clus1 STABLE ONLINE ONLINE oel7clus2 STABLE -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- DEV.db 1 ONLINE ONLINE oel7clus1 STABLE oel7clus-dbvip 1 ONLINE ONLINE oel7clus1 STABLE ora.ASMNET1LSNR_ASM.lsnr(ora.asmgroup) 1 ONLINE ONLINE oel7clus1 STABLE 2 ONLINE ONLINE oel7clus2 STABLE 3 ONLINE OFFLINE STABLE ora.DATA.dg(ora.asmgroup) 1 ONLINE ONLINE oel7clus1 STABLE 2 ONLINE ONLINE oel7clus2 STABLE 3 ONLINE OFFLINE STABLE ora.LISTENER_SCAN1.lsnr 1 ONLINE ONLINE oel7clus2 STABLE ora.LISTENER_SCAN2.lsnr 1 ONLINE ONLINE oel7clus1 STABLE ora.LISTENER_SCAN3.lsnr 1 ONLINE ONLINE oel7clus1 STABLE ora.OCR_VOT.dg(ora.asmgroup) 1 ONLINE ONLINE oel7clus1 STABLE 2 ONLINE ONLINE oel7clus2 STABLE 3 OFFLINE OFFLINE STABLE ora.asm(ora.asmgroup) 1 ONLINE ONLINE oel7clus1 Started,STABLE 2 ONLINE ONLINE oel7clus2 Started,STABLE 3 OFFLINE OFFLINE STABLE ora.asmnet1.asmnetwork(ora.asmgroup) 1 ONLINE ONLINE oel7clus1 STABLE 2 ONLINE ONLINE oel7clus2 STABLE 3 OFFLINE OFFLINE STABLE ora.cvu 1 ONLINE ONLINE oel7clus1 STABLE ora.oel7clus1.vip 1 ONLINE ONLINE oel7clus1 STABLE ora.oel7clus2.vip 1 ONLINE ONLINE oel7clus2 STABLE ora.qosmserver 1 ONLINE ONLINE oel7clus1 STABLE ora.scan1.vip 1 ONLINE ONLINE oel7clus2 STABLE ora.scan2.vip 1 ONLINE ONLINE oel7clus1 STABLE ora.scan3.vip 1 ONLINE ONLINE oel7clus1 STABLE -------------------------------------------------------------------------------- oracle@oel7clus1 /u02/app/oracle/admin/DEV/scripts :
- Next we want to add the new resource and once done start it:
- Updating /etc/oratab
- Now I know a number of DBAs prefer not to use the /etc/oratab file, but rather set things manually on their environment. I am not sure why you want to make things difficult, if you just add one entry into the /etc/oratab file for your database, you can easily move between environments using the ". oraenv" command.
- Now in my /etc/oratab files I have the following entries at the bottom:
- Node 1: oel7clus1
... +ASM1:/u01/app/19.3.0:N DEV:/u02/app/oracle/product/19.3.0/dbhome_1:N
- Node 2: oel7clus2
... +ASM2:/u01/app/19.3.0:N DEV:/u02/app/oracle/product/19.3.0/dbhome_1:N
- Node 1: oel7clus1
- Next we can test relocating our database resource.
- There is a few options here, but the first is you can just force the relocate of the database DEV.db resource and it will move the Virtual IP we added earlier as well as there is a HARD dependancy between them.
oracle@oel7clus1 /u02/app/oracle/admin/DEV/scripts : crsctl relocate resource DEV.db -f CRS-2673: Attempting to stop 'DEV.db' on 'oel7clus1' CRS-2677: Stop of 'DEV.db' on 'oel7clus1' succeeded CRS-2673: Attempting to stop 'oel7clus-dbvip' on 'oel7clus1' CRS-2677: Stop of 'oel7clus-dbvip' on 'oel7clus1' succeeded CRS-2672: Attempting to start 'oel7clus-dbvip' on 'oel7clus2' CRS-2676: Start of 'oel7clus-dbvip' on 'oel7clus2' succeeded CRS-2672: Attempting to start 'DEV.db' on 'oel7clus2' CRS-2676: Start of 'DEV.db' on 'oel7clus2' succeeded
- Or you can relocate the Virtual IP address (oel7clus-dbvip) and then the database will be moved as well due to the dependancy:
Now in the example below we run on the second node (oel7clus2) and we set the environment then execute the crsctl status resource command on the DEV.db and oel7clus-dbvip resources. We can see they are running on the second node, so the next step is to execute the relocate command on the oel7clus-dbvip Virtual IP:
oracle@oel7clus2 ~ : . oraenv ORACLE_SID = [oracle] ? +ASM2 The Oracle base has been changed from /u01/app/oracle to /u01/app/grid oracle@oel7clus2 ~ : crsctl status resource DEV.db -t -------------------------------------------------------------------------------- Name Target State Server State details -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- DEV.db 1 ONLINE ONLINE oel7clus2 STABLE -------------------------------------------------------------------------------- oracle@oel7clus2 ~ : crsctl status resource oel7clus-dbvip -t -------------------------------------------------------------------------------- Name Target State Server State details -------------------------------------------------------------------------------- Cluster Resources -------------------------------------------------------------------------------- oel7clus-dbvip 1 ONLINE ONLINE oel7clus2 STABLE -------------------------------------------------------------------------------- oracle@oel7clus2 ~ : oracle@oel7clus2 ~ : crsctl relocate resource oel7clus-dbvip -f CRS-2673: Attempting to stop 'DEV.db' on 'oel7clus2' CRS-2677: Stop of 'DEV.db' on 'oel7clus2' succeeded CRS-2673: Attempting to stop 'oel7clus-dbvip' on 'oel7clus2' CRS-2677: Stop of 'oel7clus-dbvip' on 'oel7clus2' succeeded CRS-2672: Attempting to start 'oel7clus-dbvip' on 'oel7clus1' CRS-2676: Start of 'oel7clus-dbvip' on 'oel7clus1' succeeded CRS-2672: Attempting to start 'DEV.db' on 'oel7clus1' CRS-2676: Start of 'DEV.db' on 'oel7clus1' succeeded
- There is a few options here, but the first is you can just force the relocate of the database DEV.db resource and it will move the Virtual IP we added earlier as well as there is a HARD dependancy between them.
- Now at this stage we have the database added as a resource and it has a dependancy on the Virtual IP - oel7clus-dbvip. Now you can add a listener that use this virtual IP and move that between the nodes as well. I will leave this to you to perform this task if you require it, but next we will focus on adding Dbvisit Standby to the configuration, adding its components to Clusterware as resources as well.
Step 10: Installing Dbvisit Standby
In this section we will perform a Dbvisit Standby installation on the two node cluster but also on a single server that will be used as the standby server.
- We download the latest version of the Dbvisit Standby which at the time of writing this is: dbvisit-standby9.0.02-el7.zip
- The file is copied to the 1st node in the cluster (oel7clus1) under the user "oracle" home to a sub folder 9.0. This is just a temporary folder from where we will perform the installation.
- Below are the steps followed to install the Dbvisit Standby Core components onto the shared storage /u02
- Example:
oracle@oel7clus1 ~ : cd 9.0 oracle@oel7clus1 ~/9.0 : ls dbvisit-standby9.0.02-el7.zip oracle@oel7clus1 ~/9.0 : unzip dbvisit-standby9.0.02-el7.zip Archive: dbvisit-standby9.0.02-el7.zip inflating: dbvisit-standby9.0.02-el7.tar oracle@oel7clus1 ~/9.0 : tar xf dbvisit-standby9.0.02-el7.tar oracle@oel7clus1 ~/9.0 : ls dbvisit dbvisit-standby9.0.02-el7.tar dbvisit-standby9.0.02-el7.zip INSTALL.txt README.txt oracle@oel7clus1 ~/9.0 : cd dbvisit/installer/ oracle@oel7clus1 ~/9.0/dbvisit/installer : ./install-dbvisit ----------------------------------------------------------- Welcome to the Dbvisit software installer. ----------------------------------------------------------- It is recommended to make a backup of our current Dbvisit software location (Dbvisit Base location) for rollback purposes. Installer Directory /home/oracle/9.0/dbvisit >>> Please specify the Dbvisit installation directory (Dbvisit Base). The various Dbvisit products and components - such as Dbvisit Standby, Dbvisit Dbvnet will be installed in the appropriate subdirectories of this path. Enter a custom value or press ENTER to accept default [/usr/dbvisit]: > /u02/dbvisit DBVISIT_BASE = /u02/dbvisit ----------------------------------------------------------- Component Installer Version Installed Version ----------------------------------------------------------- standby 9.0.02_0_gbd40c486 not installed dbvnet 9.0.02_0_gbd40c486 not installed dbvagent 9.0.02_0_gbd40c486 not installed dbvserver 9.0.02_0_gbd40c486 not installed observer 1.02 not installed ----------------------------------------------------------- What action would you like to perform? 1 - Install component(s) 2 - Uninstall component(s) 3 - Exit Your choice: 1 Choose component(s): 1 - Core Components (Dbvisit Standby Cli, Dbvnet, Dbvagent) 2 - Dbvisit Standby Core (Command Line Interface) 3 - Dbvnet (Dbvisit Network Communication) 4 - Dbvagent (Dbvisit Agent) 5 - Dbvserver (Dbvisit Central Console) - Not available on Solaris/AIX 6 - Dbvisit Observer (Automatic Failover Option) - Not available on Solaris/AIX Press ENTER to exit Installer Your choice: 1 ----------------------------------------------------------- Summary of the Dbvisit STANDBY configuration ----------------------------------------------------------- DBVISIT_BASE /u02/dbvisit Press ENTER to continue ----------------------------------------------------------- About to configure DBVISIT DBVNET ----------------------------------------------------------- >>> Please specify the Local host name to be used by Dbvnet on this server. Dbvnet will be listening on the local IP Address on this server which resolve to the host name specified here. If using a cluster or virtual IP make sure the host name or alias specified here resolve to the IP address local to where dbvnet is installed. The host name should resolve to IPv4 address, if not you can use an IPv4 IP address instead of host name. Enter a custom value or press ENTER to accept default [oel7clus1]: > oel7clus-dbvip >>> Please specify the Local Dbvnet PORT to be used. Dbvnet will be listening on the specified port for incoming connections from remote dbvnet connections. Please make sure that this port is not already in use or blocked by any firewall. You may choose any value between 1024 and 65535, however the default of 7890 is recommended. Enter a custom value or press ENTER to accept default [7890]: > 7890 >>> Please specify the Remote host name to be used by Dbvnet. By default Dbvnet will use this remote hostname for any remote connections. Dbvnet must be installed and configured on the specified remote host. If using a cluster or virtual IP make sure the host name or alias specified here resolve to the IP address local to where dbvnet is installed. If you are unsure about the remote host name during installation, use the default value which will be the current local hostname. The host name should resolve to IPv4 address, if not you can use an IPv4 IP address instead of host name. Enter a custom value or press ENTER to accept default [oel7clus1]: > oel7fs1 >>> Please specify the Remote Dbvnet PORT to be used. Dbvnet will connect to the remote server on this specified port. On the remote host Dbvnet will be listening on the specified port for incoming connections. Please make sure that this port is not already in use or blocked by any firewall. You may choose any value between 1024 and 65535, however the default of 7890 is recommended. Enter a custom value or press ENTER to accept default [7890]: > 7890 >>> Please specify the Dbvnet Passphrase to be used for secure connections. The passphrase provided must be the same in both the local and remote Dbvnet installations. It is used to establish a secure (encrypted) Dbvnet connections Enter a custom value: > kiwi123 ----------------------------------------------------------- Summary of the Dbvisit DBVNET configuration ----------------------------------------------------------- DBVISIT_BASE /u02/dbvisit DBVNET_LOCAL_HOST oel7clus-dbvip DBVNET_LOCAL_PORT 7890 DBVNET_REMOTE_HOST oel7fs1 DBVNET_REMOTE_PORT 7890 DBVNET_PASSPHRASE kiwi123 Press ENTER to continue ----------------------------------------------------------- About to configure DBVISIT DBVAGENT ----------------------------------------------------------- >>> Please specify the host name to be used for the Dbvisit Agent. The Dbvisit Agent (Dbvagent) will be listening on this local address. If you are using the Dbvserver (GUI) - connections from the GUI will be established to the Dbvisit Agent. The Dbvisit Agent address must be visible from the Dbvserver (GUI) installation. If using a cluster or virtual IP make sure the host name or alias specified here resolve to the IP address local to where dbvnet is installed. The host name should resolve to IPv4 address, if not you can use an IPv4 IP address instead of host name. Enter a custom value or press ENTER to accept default [oel7clus1]: > oel7clus-dbvip >>> Please specify the listening PORT number for Dbvagent. The Dbvisit Agent (Dbvagent) will be listening on the specified port for incoming requests from the GUI (Dbvserver). Please make sure that this port is not already in use or blocked by any firewall. You may choose any value between 1024 and 65535, however the default of 7891 is recommended. Enter a custom value or press ENTER to accept default [7891]: > 7891 >>> Please specify passphrase for Dbvagent Each Dbvisit Agent must have a passpharse specified. This passphrase does not have to match between all the servers. It will be used to establish a secure connection between the GUI (Dbvserver) and the Dbvisit Agent. Enter a custom value: > kiwi123 ----------------------------------------------------------- Summary of the Dbvisit DBVAGENT configuration ----------------------------------------------------------- DBVISIT_BASE /u02/dbvisit DBVAGENT_LOCAL_HOST oel7clus-dbvip DBVAGENT_LOCAL_PORT 7891 DBVAGENT_PASSPHRASE kiwi123 Press ENTER to continue ----------------------------------------------------------- About to install Dbvisit STANDBY ----------------------------------------------------------- NTF repo installed. Component standby installed. ----------------------------------------------------------- About to install Dbvisit DBVNET ----------------------------------------------------------- Component dbvnet installed. ----------------------------------------------------------- About to install Dbvisit DBVAGENT ----------------------------------------------------------- Component dbvagent installed. ----------------------------------------------------------- Component Installer Version Installed Version ----------------------------------------------------------- standby 9.0.02_0_gbd40c486 9.0.02_0_gbd40c486 dbvnet 9.0.02_0_gbd40c486 9.0.02_0_gbd40c486 dbvagent 9.0.02_0_gbd40c486 9.0.02_0_gbd40c486 dbvserver 9.0.02_0_gbd40c486 not installed observer 1.02 not installed ----------------------------------------------------------- What action would you like to perform? 1 - Install component(s) 2 - Uninstall component(s) 3 - Exit Your choice: 3 >>> Installation completed Install log /tmp/dbvisit_install.log.201909101237. oracle@oel7clus1 ~/9.0/dbvisit/installer : cd /u02/dbvisit oracle@oel7clus1 /u02/dbvisit : ls dbvagent dbvnet standby oracle@oel7clus1 /u02/dbvisit : tree -d . . ├── dbvagent │ ├── conf │ ├── doc │ ├── log │ └── tmp ├── dbvnet │ ├── conf │ ├── doc │ └── log └── standby ├── conf ├── doc ├── gs ├── lib ├── log ├── pid ├── tmp └── trace 18 directories oracle@oel7clus1 /u02/dbvisit :
- Now we have Dbvisit Standby installed on the shared storage, we can test by starting the Dbvagent and Dbvnet components:
oracle@oel7clus1 /u02/dbvisit : cd dbvnet oracle@oel7clus1 /u02/dbvisit/dbvnet : ./dbvnet -d start Dbvnet daemon started. oracle@oel7clus1 /u02/dbvisit/dbvnet : cd ../dbvagent/ oracle@oel7clus1 /u02/dbvisit/dbvagent : ./dbvagent -d start Dbvagent daemon started. oracle@oel7clus1 /u02/dbvisit/dbvagent : ps -ef|egrep 'dbvnet|dbvagent' |grep -v grep oracle 32446 1 0 12:44 ? 00:00:00 ./dbvnet -d start oracle 32569 1 0 12:44 ? 00:00:00 ./dbvagent -d start
- Next we need to install Dbvisit Standby on the Standby server, in this case it is called "oel7fs1" - it is a standalone server with Oracle Database 19c SE2 installed.
- The steps are similar to above, but in this case we also install the Web server component - Dbvserver.
- Note: in 9.0.02 we introduced the automatic failover option, this option is not installed in this configuration.
- Example installation steps:
oracle@oel7fs1 ~/9.0 : ls -ld /usr/dbvisit drwxr-xr-x. 2 oracle oinstall 4096 Sep 10 09:46 /usr/dbvisit oracle@oel7fs1 ~/9.0 : ls -l total 81572 -rw-r--r--. 1 oracle oinstall 83529352 Sep 10 12:48 dbvisit-standby9.0.02-el7.zip oracle@oel7fs1 ~/9.0 : unzip dbvisit-standby9.0.02-el7.zip Archive: dbvisit-standby9.0.02-el7.zip inflating: dbvisit-standby9.0.02-el7.tar oracle@oel7fs1 ~/9.0 : tar xf dbvisit-standby9.0.02-el7.tar oracle@oel7fs1 ~/9.0 : cd dbvisit/installer/ oracle@oel7fs1 ~/9.0/dbvisit/installer : ./install-dbvisit ----------------------------------------------------------- Welcome to the Dbvisit software installer. ----------------------------------------------------------- It is recommended to make a backup of our current Dbvisit software location (Dbvisit Base location) for rollback purposes. Installer Directory /home/oracle/9.0/dbvisit >>> Please specify the Dbvisit installation directory (Dbvisit Base). The various Dbvisit products and components - such as Dbvisit Standby, Dbvisit Dbvnet will be installed in the appropriate subdirectories of this path. Enter a custom value or press ENTER to accept default [/usr/dbvisit]: > DBVISIT_BASE = /usr/dbvisit ----------------------------------------------------------- Component Installer Version Installed Version ----------------------------------------------------------- standby 9.0.02_0_gbd40c486 not installed dbvnet 9.0.02_0_gbd40c486 not installed dbvagent 9.0.02_0_gbd40c486 not installed dbvserver 9.0.02_0_gbd40c486 not installed observer 1.02 not installed ----------------------------------------------------------- What action would you like to perform? 1 - Install component(s) 2 - Uninstall component(s) 3 - Exit Your choice: 1 Choose component(s): 1 - Core Components (Dbvisit Standby Cli, Dbvnet, Dbvagent) 2 - Dbvisit Standby Core (Command Line Interface) 3 - Dbvnet (Dbvisit Network Communication) 4 - Dbvagent (Dbvisit Agent) 5 - Dbvserver (Dbvisit Central Console) - Not available on Solaris/AIX 6 - Dbvisit Observer (Automatic Failover Option) - Not available on Solaris/AIX Press ENTER to exit Installer Your choice: 1 ----------------------------------------------------------- Summary of the Dbvisit STANDBY configuration ----------------------------------------------------------- DBVISIT_BASE /usr/dbvisit Press ENTER to continue ----------------------------------------------------------- About to configure DBVISIT DBVNET ----------------------------------------------------------- >>> Please specify the Local host name to be used by Dbvnet on this server. Dbvnet will be listening on the local IP Address on this server which resolve to the host name specified here. If using a cluster or virtual IP make sure the host name or alias specified here resolve to the IP address local to where dbvnet is installed. The host name should resolve to IPv4 address, if not you can use an IPv4 IP address instead of host name. Enter a custom value or press ENTER to accept default [oel7fs1]: > oel7fs1 >>> Please specify the Local Dbvnet PORT to be used. Dbvnet will be listening on the specified port for incoming connections from remote dbvnet connections. Please make sure that this port is not already in use or blocked by any firewall. You may choose any value between 1024 and 65535, however the default of 7890 is recommended. Enter a custom value or press ENTER to accept default [7890]: > 7890 >>> Please specify the Remote host name to be used by Dbvnet. By default Dbvnet will use this remote hostname for any remote connections. Dbvnet must be installed and configured on the specified remote host. If using a cluster or virtual IP make sure the host name or alias specified here resolve to the IP address local to where dbvnet is installed. If you are unsure about the remote host name during installation, use the default value which will be the current local hostname. The host name should resolve to IPv4 address, if not you can use an IPv4 IP address instead of host name. Enter a custom value or press ENTER to accept default [oel7fs1]: > oel7clus-dbvip >>> Please specify the Remote Dbvnet PORT to be used. Dbvnet will connect to the remote server on this specified port. On the remote host Dbvnet will be listening on the specified port for incoming connections. Please make sure that this port is not already in use or blocked by any firewall. You may choose any value between 1024 and 65535, however the default of 7890 is recommended. Enter a custom value or press ENTER to accept default [7890]: > 7890 >>> Please specify the Dbvnet Passphrase to be used for secure connections. The passphrase provided must be the same in both the local and remote Dbvnet installations. It is used to establish a secure (encrypted) Dbvnet connections Enter a custom value: > kiwi123 ----------------------------------------------------------- Summary of the Dbvisit DBVNET configuration ----------------------------------------------------------- DBVISIT_BASE /usr/dbvisit DBVNET_LOCAL_HOST oel7fs1 DBVNET_LOCAL_PORT 7890 DBVNET_REMOTE_HOST oel7clus-dbvip DBVNET_REMOTE_PORT 7890 DBVNET_PASSPHRASE kiwi123 Press ENTER to continue ----------------------------------------------------------- About to configure DBVISIT DBVAGENT ----------------------------------------------------------- >>> Please specify the host name to be used for the Dbvisit Agent. The Dbvisit Agent (Dbvagent) will be listening on this local address. If you are using the Dbvserver (GUI) - connections from the GUI will be established to the Dbvisit Agent. The Dbvisit Agent address must be visible from the Dbvserver (GUI) installation. If using a cluster or virtual IP make sure the host name or alias specified here resolve to the IP address local to where dbvnet is installed. The host name should resolve to IPv4 address, if not you can use an IPv4 IP address instead of host name. Enter a custom value or press ENTER to accept default [oel7fs1]: > oel7fs1 >>> Please specify the listening PORT number for Dbvagent. The Dbvisit Agent (Dbvagent) will be listening on the specified port for incoming requests from the GUI (Dbvserver). Please make sure that this port is not already in use or blocked by any firewall. You may choose any value between 1024 and 65535, however the default of 7891 is recommended. Enter a custom value or press ENTER to accept default [7891]: > 7891 >>> Please specify passphrase for Dbvagent Each Dbvisit Agent must have a passpharse specified. This passphrase does not have to match between all the servers. It will be used to establish a secure connection between the GUI (Dbvserver) and the Dbvisit Agent. Enter a custom value: > kiwi123 ----------------------------------------------------------- Summary of the Dbvisit DBVAGENT configuration ----------------------------------------------------------- DBVISIT_BASE /usr/dbvisit DBVAGENT_LOCAL_HOST oel7fs1 DBVAGENT_LOCAL_PORT 7891 DBVAGENT_PASSPHRASE kiwi123 Press ENTER to continue ----------------------------------------------------------- About to install Dbvisit STANDBY ----------------------------------------------------------- NTF repo installed. Component standby installed. ----------------------------------------------------------- About to install Dbvisit DBVNET ----------------------------------------------------------- Component dbvnet installed. ----------------------------------------------------------- About to install Dbvisit DBVAGENT ----------------------------------------------------------- Component dbvagent installed. ----------------------------------------------------------- Component Installer Version Installed Version ----------------------------------------------------------- standby 9.0.02_0_gbd40c486 9.0.02_0_gbd40c486 dbvnet 9.0.02_0_gbd40c486 9.0.02_0_gbd40c486 dbvagent 9.0.02_0_gbd40c486 9.0.02_0_gbd40c486 dbvserver 9.0.02_0_gbd40c486 not installed observer 1.02 not installed ----------------------------------------------------------- What action would you like to perform? 1 - Install component(s) 2 - Uninstall component(s) 3 - Exit Your choice: 1 Choose component(s): 1 - Core Components (Dbvisit Standby Cli, Dbvnet, Dbvagent) 2 - Dbvisit Standby Core (Command Line Interface) 3 - Dbvnet (Dbvisit Network Communication) 4 - Dbvagent (Dbvisit Agent) 5 - Dbvserver (Dbvisit Central Console) - Not available on Solaris/AIX 6 - Dbvisit Observer (Automatic Failover Option) - Not available on Solaris/AIX Press ENTER to exit Installer Your choice: 5 ----------------------------------------------------------- About to configure DBVISIT DBVSERVER ----------------------------------------------------------- >>> Please specify the host name to be used for Dbvserver The Dbvisit Web Server (Dbvserver) will be listening on this local address. If using a cluster or virtual IP make sure the host name or alias specified here resolve to the IP address local to where Dbvserver is installed. If you are unsure about the remote host name during installation, use the default value which will be the current local hostname. The host name should resolve to IPv4 address, if not you can use an IPv4 IP address instead of host name. Enter a custom value or press ENTER to accept default [oel7fs1]: > oel7fs1 >>> Please specify the listening port number for Dbvserver on the local server You may choose any value between 1024 and 65535. The default recommended value is 4433. Note: if you can not access this port after the installation has finished, then please double-check your server firewall settings to ensure the selected port is open. Enter a custom value or press ENTER to accept default [4433]: > 4433 >>> Please specify the host name (or IPv4 address) to be used for Dbvserver public interface In most cases this will be the same as the listener address, if not sure use the same value as the listener address. The Dbvisit Web Server (Dbvserver) will be listening on the local listener address. The public address can be set to an external IP example a firewall address in case the Central Console (Dbvserver) and agents (Primary and Standby Database servers) have a firewall inbetween them. The public interface address will be passed to the agents during communication for sending information back. If you are unsure about the public host address, use the default value which will be the current local hostname. The host name should resolve to IPv4 address, if not you can use an IPv4 IP address instead of host name. Enter a custom value or press ENTER to accept default [oel7fs1]: > oel7fs1 ----------------------------------------------------------- Summary of the Dbvisit DBVSERVER configuration ----------------------------------------------------------- DBVISIT_BASE /usr/dbvisit DBVSERVER_LOCAL_HOST oel7fs1 DBVSERVER_LOCAL_PORT 4433 DBVSERVER_PUBLIC_HOST oel7fs1 Press ENTER to continue ----------------------------------------------------------- About to install Dbvisit DBVSERVER ----------------------------------------------------------- Component dbvserver installed. ----------------------------------------------------------- Component Installer Version Installed Version ----------------------------------------------------------- standby 9.0.02_0_gbd40c486 9.0.02_0_gbd40c486 dbvnet 9.0.02_0_gbd40c486 9.0.02_0_gbd40c486 dbvagent 9.0.02_0_gbd40c486 9.0.02_0_gbd40c486 dbvserver 9.0.02_0_gbd40c486 9.0.02_0_gbd40c486 observer 1.02 not installed ----------------------------------------------------------- What action would you like to perform? 1 - Install component(s) 2 - Uninstall component(s) 3 - Exit Your choice: 3 >>> Installation completed Install log /tmp/dbvisit_install.log.201909101248. oracle@oel7fs1 ~/9.0/dbvisit/installer : oracle@oel7fs1 ~/9.0/dbvisit/installer : cd /usr/dbvisit oracle@oel7fs1 /usr/dbvisit : ls dbvagent dbvnet dbvserver standby oracle@oel7fs1 /usr/dbvisit : tree -d . . ├── dbvagent │ ├── conf │ ├── doc │ ├── log │ └── tmp ├── dbvnet │ ├── conf │ ├── doc │ └── log ├── dbvserver │ ├── conf │ ├── doc │ └── log └── standby ├── conf ├── doc ├── gs ├── lib ├── log ├── pid ├── tmp └── trace 22 directories oracle@oel7fs1 /usr/dbvisit : cd dbvnet oracle@oel7fs1 /usr/dbvisit/dbvnet : ./dbvnet -d start Dbvnet daemon started. oracle@oel7fs1 /usr/dbvisit/dbvnet : cd ../dbvagent oracle@oel7fs1 /usr/dbvisit/dbvagent : ./dbvagent -d start Dbvagent daemon started. oracle@oel7fs1 /usr/dbvisit/dbvagent : cd ../dbvserver oracle@oel7fs1 /usr/dbvisit/dbvserver : ./dbvserver -d start Dbvserver daemon started. oracle@oel7fs1 /usr/dbvisit/dbvserver :
- You will also notice in the last steps above we started the Dbvisit Standby components (dbvnet, dbvagent and dbvserver).
- Next we can now navigate to the Dbvisit Standby GUI (Central Console) to create a new Dbvisit Standby Configuration (DDC) and then the Standby Database.
- Here are a few extracts to explain the configuration. Please note for more details on adding hosts, creating the configuration and the standby database, please see the Dbvisit Standby documentation.
- Host Overview:
- DDC Configuration:
- Now remember the primary is a cluster and shared storage is used. Dbvisit Standby was installed in /u02/dbvisit and on the standby we installed into the default /usr/dbvisit
- Remember during the DDC creation that you pick the correct DBVISIT_BASE and ORACLE_HOME values as the primary and standby may be different
- Standby Database Creation:
- Follow the standard procedure to create the standby database. Now my standby server filesystem look like below:
aelsnz@oel7fs1 ~ : df -h Filesystem Size Used Avail Use% Mounted on devtmpfs 1.4G 0 1.4G 0% /dev tmpfs 1.4G 0 1.4G 0% /dev/shm tmpfs 1.4G 8.6M 1.4G 1% /run tmpfs 1.4G 0 1.4G 0% /sys/fs/cgroup /dev/sda3 6.3G 3.9G 2.2G 65% / /dev/sda1 488M 278M 175M 62% /boot tmpfs 271M 0 271M 0% /run/user/0 tmpfs 271M 0 271M 0% /run/user/1000 /dev/mapper/oravg-orahomelv 30G 7.1G 21G 26% /u01 /dev/mapper/oravg-oradblv 20G 65M 19G 1% /u02
- Now the Oracle Home is /u01/app/oracle/product/19.3.0/dbhome_1
- The database will be stored in /u02 and making use of Oracle Managed Files (OMF) - so the following locations are used:
- db_create_file_dest = /u02/oracle/oradata
- fast_recovery_area = /u02/oracle/fast_recovery_area
- Once the standby database creation is complete I have the following task details:
- Now that a fully functional primary/standby configuration in Dbvisit Standby, below are just a sample of the Database Actions screen where I showed that logs were shipped and applied.
Step 11: Adding Dbvnet and Dbvagent as Cluster Resources
The next step is to add Dbvnet and Dbvagent as cluster resources with a dependancy on the cluster virtual IP created earlier.
This can be done by adding an action script for Dbvnet and Dbvagent and then adding them as cluster resources.
Here are two sample scripts:
dbvnet-crs.sh
#!/usr/bin/bash
# set -x
#
# Dbvnet Action Script
##############################################
# Function to change database environments
# Description:
# Use oraenv to set the environment if needed
# This is optional to set the environment
##############################################
## set following to ensure oraenv is picked up from /usr/local/bin
export PATH=/usr/local/bin:$PATH
NUM=0
set_env ()
{
export ORAENV_ASK=NO
export ORACLE_SID=$1
. oraenv >> /dev/null
export ORAENV_ASK=YES
export USER=oracle
}
#################
## Main Section
#################
# This is logged to CRSD agent log file
echo "`date` Action script '$_CRS_ACTION_SCRIPT' for resource [$_CRS_NAME] called for action $1"
# set environment
set_env QA
cd /u02/dbvisit/dbvnet
case "$1" in
'start')
./dbvnet -d start
RET=0
echo "Running start dbvnet resource with return code $RET"
;;
'stop')
NUM=`ps -ef | grep dbvnet | egrep -v 'grep|dbvnet-crs|resource' | wc -l`
if [ $NUM = 0 ]; then
## do a cleanup of pid
./dbvnet -d stop
RET=0
else
## now stop the dbvnet
./dbvnet -d stop
NUM=`ps -ef | grep dbvnet | grep -v grep | wc -l`
if [ $NUM = 0 ]; then
RET=0
else
RET=1
fi
fi
echo "Running stop dbvnet resource with return code $RET"
;;
'check')
NUM=`ps -ef | grep dbvnet | egrep -v 'grep|dbvnet-crs|resource' | wc -l`
if [ $NUM = 0 ]; then
## return code 1 for check means OFFLINE
RET=1
else
## return code 0 for check means ONLINE
RET=0
fi
echo "Running check dbvnet resource with return code $RET"
;;
'clean')
for c1 in `ps -ef|grep dbvnet |egrep -v 'grep|dbvnet-crs|resource'| awk '{print $2}'` ;
do
echo "...force kill dbvnet pid $c1"
kill -9 $c1
done
## do some cleanup
./dbvnet -d stop
RET=0
echo "Running clean dbvnet resource with return code $RET"
;;
*)
echo "Usage: ./dbvnet-csr.sh start|stop|clean|clean"
RET=0
;;
esac
if [ $RET -eq 0 ]; then
exit 0
else
exit 1
fi
dbvagent-crs.sh
#!/usr/bin/bash
# set -x
#
# Dbvnet Action Script
##############################################
# Function to change database environments
# Description:
# Use oraenv to set the environment if needed
# This is optional to set the environment
##############################################
## set following to ensure oraenv is picked up from /usr/local/bin
export PATH=/usr/local/bin:$PATH
NUM=0
set_env ()
{
export ORAENV_ASK=NO
export ORACLE_SID=$1
. oraenv >> /dev/null
export ORAENV_ASK=YES
export USER=oracle
}
#################
## Main Section
#################
# This is logged to CRSD agent log file
echo "`date` Action script '$_CRS_ACTION_SCRIPT' for resource [$_CRS_NAME] called for action $1"
# set environment
set_env QA
cd /u02/dbvisit/dbvagent
case "$1" in
'start')
./dbvagent -d start
RET=0
echo "Running start dbvagent resource with return code $RET"
;;
'stop')
NUM=`ps -ef | grep dbvagent | egrep -v 'grep|dbvagent-crs|resource' | wc -l`
if [ $NUM = 0 ]; then
## do a cleanup of pid
./dbvagent -d stop
RET=0
else
## now stop the dbvagent
./dbvagent -d stop
NUM=`ps -ef | grep dbvagent | grep -v grep | wc -l`
if [ $NUM = 0 ]; then
RET=0
else
RET=1
fi
fi
echo "Running stop dbvagent resource with return code $RET"
;;
'check')
NUM=`ps -ef | grep dbvagent | egrep -v 'grep|dbvagent-crs|resource' | wc -l`
if [ $NUM = 0 ]; then
## return code 1 for check means OFFLINE
RET=1
else
## return code 0 for check means ONLINE
RET=0
fi
echo "Running check dbvagent resource with return code $RET"
;;
'clean')
for c1 in `ps -ef|grep dbvagent |egrep -v 'grep|dbvagent-crs|resource'| awk '{print $2}'` ;
do
echo "...force kill dbvagent pid $c1"
kill -9 $c1
done
## do some cleanup
./dbvagent -d stop
RET=0
echo "Running clean dbvagent resource with return code $RET"
;;
*)
echo "Usage: ./dbvagent-csr.sh start|stop|check|clean"
RET=0
;;
esac
if [ $RET -eq 0 ]; then
exit 0
else
exit 1
fi
Below is example commands of adding these two components - Note that while doing this you have to stop dbvnet and dbvagent, then after adding them as cluster resources, you can start them using the crsctl start resource command.
crsctl add resource dbvnet -type cluster_resource -attr "ACTION_SCRIPT=/u02/dbvisit/dbvnet/dbvnet-crs.sh, RESTART_ATTEMPTS=3, START_TIMEOUT=60, STOP_TIMEOUT=60, CHECK_INTERVAL=10,
START_DEPENDENCIES='hard(oel7clus-dbvip) pullup(oel7clus-dbvip)'
,STOP_DEPENDENCIES='hard(oel7clus-dbvip)'
ACL='owner:oracle:rwx,pgrp:oinstall:rwx,other::r--'
PLACEMENT='favored'
HOSTING_MEMBERS='oel7clus1'"
crsctl add resource dbvagent -type cluster_resource -attr "ACTION_SCRIPT=/u02/dbvisit/dbvagent/dbvagent-crs.sh, RESTART_ATTEMPTS=3, START_TIMEOUT=60, STOP_TIMEOUT=60, CHECK_INTERVAL=10,
START_DEPENDENCIES='hard(oel7clus-dbvip) pullup(oel7clus-dbvip)'
,STOP_DEPENDENCIES='hard(oel7clus-dbvip)'
ACL='owner:oracle:rwx,pgrp:oinstall:rwx,other::r--'
PLACEMENT='favored'
HOSTING_MEMBERS='oel7clus1'"
Step 12: Adding a Pre/Post Processing script
One important step you will need to perform is to add a Dbvisit Standby pre/post-processing script. This is a script that gets executed prior to or after certain Dbvisit Functions. Example, if you are performing a Graceful Switchover and you want to perform certain tasks prior or after the switchover, like moving application connection details or updating DNS or anything can script, you can use this script to help you achieve this.
Now, why do you need one in this scenario you may ask, well the Database Resource created above DEV.db is now going to automatically failover or restart if you perform an SQLPLUS or RMAN restart of the database and not use the Cluster Resource command (crsctl start|stop resource DEV.db).
This becomes important when you want to perform a Graceful Switchover, during which the primary database is stopped and converted to a standby. Now if you do not disable the cluster resource during this operation, Clusterware will attempt to restart the database or will fail-over to the second node.
Now, this can easily be avoided by setting the cluster resource attribute ENABLED=0 and then once the Graceful Switchover is complete set it back to one.
This can easily be done by a pre/post-processing script. Now a sample is provided in the DBVISIT_BASE/doc subfolder of any installation. You can take this file move it to shared storage in the cluster - we recommend DBVISIT_BASE/standby or DBVISIT_BASE/conf
Here is a short extract of this script in the above example, showing the setting of the resource to ENABLED=0
#!/bin/bash
#
# Description:
# ===========
# This script is provided by Dbvisit Software Limited as an example
# pre/post processing script. This script can be used to perform
# custom tasks (pre) before Dbvisit Standby execute, or (post) following
# a successful execution.
#
# NOTE: This script is a basic template and you will need to add custom code
# to this script to perform the required tasks you want to perform as
# part of pre or post dbvisit processing.
#
# This script should be added to the Dbvisit Database Configuration (DDC) file
# Example: PRE_POST_PROCESSING=/usr/local/bin/pre-post-dbvisit.sh
#
# Dbvisit STandby will pass two parameters to this script
# Parameter 1 can be one of two possible values: [pre|post]
# Parameter 2 can be one number from 1 to 8: [1|2|3|4|5|6|7|8]
#
# Usage: ./pre-post-dbvisit.sh [pre|post] [1|2|3|4|5|6|7|8]
#
# Example: ./pre-post-dbvisit.sh pre 1
#
# Parameter explanation:
# =====================
# pre = Pre Dbvisit Processing step
# post = Post Dbvisit Processing step
# 1 = Dbvisit is executed on primary and will be sending logs to standby
# 2 = Dbvisit is executed on standby and will be applying logs
# 3 = Dbvisit Graceful Switchover is executed on primary
# 4 = Dbvisit Graceful Switchover is executed on standby
# 5 = Dbvisit Activation is executed on standby
# 6 = Standby Database is opened read-only
# 7 = Dbvisit Create Standby Database is executed on primary (post only)
# 8 = Dbvisit Create Standby Database is executed on standby (post only)
#
# Note: This script is an example only and should be tested first before
# it is used in a production environment.
#
# To enable debugging, uncomment following line:
# set -x
#
###########################################
# Function to echo usage
###########################################
usage ()
{
program=`basename $0`
cat <<EOF
Usage:
${program} [pre|post] [1|2|3|4|5|6|7|8]
[pre|post} - Specify if Pre or Post processing is being performed
[1|2|3|4|5|6|7|8] - Indicate Dbvisit operation being performed
-h - display the usage
pre = Pre Dbvisit Processing step
post = Post Dbvisit Processing step
1 = Dbvisit is executed on primary and will be sending logs to standby
2 = Dbvisit is executed on standby and will be applying logs
3 = Dbvisit Graceful Switchover is executed on primary
4 = Dbvisit Graceful Switchover is executed on standby
5 = Dbvisit Activation is executed on standby
6 = Standby Database is opened read-only
7 = Dbvisit Create Standby Database is executed on primary (post only)
8 = Dbvisit Create Standby Database is executed on standby (post only)
Example, to perform a task on the standby server following activation:
${program} post 5
EOF
exit 1
}
###########################################
# Function to setup parameter variables
# Description:
# Specify any global settings in this section
###########################################
setup_parameters ()
{
## Below are some example values
export PATH=$PATH:/bin:/usr/bin:/usr/local/bin
export NLS_DATE_FORMAT='DD/MM/YYYY HH24:MI:SS'
SERVER=`uname -n`
dbvisit_base=/u02/dbvisit
logfile=${dbvisit_base}/standby/log/pre-post-dbvisit.log
}
################################################
# Function to change database environments
# Description:
# Use oraenv to set the environment if needed
# This is optional to set the environment
################################################
set_env ()
{
export ORAENV_ASK=NO
export ORACLE_SID=$1
. oraenv >> /dev/null
export ORAENV_ASK=YES
}
###########################################
# Function to Add text to logfile
###########################################
addLog()
{
# comment 1st line below to remove echo back to user and to only write to log
echo "`date +%d-%h-%Y:%H:%M:%S` : ${1}"
echo "`date +%d-%h-%Y:%H:%M:%S` : ${1}" >> ${logfile}
}
### Add your custom functions here
###
disable_cluster_resource () {
## This example we are testing if runninig on clusterware, if resource exist
## disable it, as shutdown of database can cause resource to restart it again....
server_name=`uname -n`
if [ "${server_name}" = "dbvclus1.dbvisit.co.nz" ]; then
set_env +ASM1
else
set_env +ASM2
fi
crsctl modify resource DEV.db -attr "ENABLED=0"
}
enable_cluster_resource () {
## This example we are testing if runninig on clusterware, if resource exist
## disable it, as shutdown of database can cause resource to restart it again....
## at end of operation we want to enable it again
server_name=`uname -n`
if [ "${server_name}" = "dbvclus1.dbvisit.co.nz" ]; then
set_env +ASM1
else
set_env +ASM2
fi
crsctl modify resource DEV.db -attr "ENABLED=1"
}
##########################
##########################
####
#### MAIN PROGRAM SECTION
####
##########################
##########################
## First call function to set the environment or specific global variables
setup_parameters
## Test input values, must be 2 values, if not exit
if test $# -lt 2
then
echo "Cannot execute, 2 values must be specified as input..."
usage
exit 99
fi
if [ ${1} = "pre" ] || [ ${1} = "post" ]
then
if ! [[ $2 =~ ^[1-8]$ ]]
then
echo "2nd Parameter must be a number between 1 and 8"
usage
exit 97
fi
else
echo "1st Parameter must be \"pre\" or \"post\"";
usage
exit 98
fi
### Now that values are valid, execute required code block
case ${1} in
pre)
addLog "Start Pre Processing"
case ${2} in
1)
addLog "Primary Database Send"
## add custom code here or call custom function
;;
2)
addLog "Standby Database Apply"
## add custom code here or call custom function
;;
3)
addLog "Primary Server Graceful Switchover"
if [ "${SERVER}" = "dbvclus1.dbvisit.co.nz" ] || [ "${SERVER}" = "dbvclus2.dbvisit.co.nz" ]; then
disable_cluster_resource "DEV.db"
fi
## add custom code here or call custom function
;;
4)
addLog "Standby Server Graceful Switchover"
## add custom code here or call custom function
if [ "${SERVER}" = "dbvclus1.dbvisit.co.nz" ] || [ "${SERVER}" = "dbvclus2.dbvisit.co.nz" ]; then
disable_cluster_resource "DEV.db"
fi
;;
5)
addLog "Standby Database Activate"
## add custom code here or call custom function
;;
6)
addLog "Standby Database Read-Only"
## add custom code here or call custom function
;;
esac
;;
post)
addLog "Start post Processing"
case ${2} in
1)
addLog "Primary Database Send"
## add custom code here or call custom function
;;
2)
addLog "Standby Database Apply"
## add custom code here or call custom function
# Uncomment/edit lines below to compress archive logs in ARCHDEST
# on the standby when logs are transferred uncompressed
# Perl script compress_archdest must be located in the same
# directory as the pre-post processing shell script
# Param ${3} is passed from dbvctl and is used by script
# compress_archdest.pl
#compress_method="/bin/gzip"
# uncompliled script requires perl to be installed
#perl compress_archdest.pl ${3} --compress_method "$compress_method" >> ${logfile}
# Script compress_archdest.pl can be compiled
#compress_archdest ${3} --compress_method "$compress_method" >> ${logfile}
;;
3)
addLog "Primary Server Graceful Switchover"
## add custom code here or call custom function
if [ "${SERVER}" = "dbvclus1.dbvisit.co.nz" ] || [ "${SERVER}" = "dbvclus2.dbvisit.co.nz" ]; then
enable_cluster_resource "DEV.db"
fi
;;
4)
addLog "Standby Server Graceful Switchover"
## add custom code here or call custom function
addLog "Primary Server Graceful Switchover"
if [ "${SERVER}" = "dbvclus1.dbvisit.co.nz" ] || [ "${SERVER}" = "dbvclus2.dbvisit.co.nz" ]; then
enable_cluster_resource "DEV.db"
fi
;;
5)
addLog "Standby Database Activate"
## add custom code here or call custom function
;;
6)
addLog "Standby Database Read-Only"
## add custom code here or call custom function
;;
7)
addLog "Primary Server Create Standby Database"
## add custom code here or call custom function
;;
8)
addLog "Standby Server Create Standby Database"
## add custom code here or call custom function
;;
esac
;;
esac
Testing the pre/post processing script is easy, you can just pass in the parameters and then review the ENABLED attribute of the DEV.db resource.
Example:
oracle@dbvclus1 /u02/dbvisit/standby/conf : ./pre_post_processing.sh pre 3 11-Sep-2019:14:22:21 : Start Pre Processing 11-Sep-2019:14:22:21 : Primary Server Graceful Switchover oracle@dbvclus1 /u02/dbvisit/standby/conf : crsctl status resource DEV.db -f|grep ENABLED ENABLED=0 oracle@dbvclus1 /u02/dbvisit/standby/conf : ./pre_post_processing.sh post 3 11-Sep-2019:14:22:33 : Start post Processing 11-Sep-2019:14:22:33 : Primary Server Graceful Switchover oracle@dbvclus1 /u02/dbvisit/standby/conf : crsctl status resource DEV.db -f|grep ENABLED ENABLED=1