Plugins to Monitor Oracle with Zabbix

ZBXORA

Zabbix Oracle monitoring plugin Downloadable from https://github.com/ikzelf/zbxora

Written in python, tested with python 2.6 and 2.7. Using cx_Oracle purpose is monitoring an Oracle database in an efficient way. Optionally calling zabbix_sender to upload data

Supports Oracle 9,10,11,12 RAC,asm and plugin databases Tested with Oracle 11,12 RAC,standby,asm and plugin databases

 

ZabbixDBA

ZabbixDBA is fast, flexible, and continuously developing plugin to monitor your RDBMS.

ZabbixDBA uses threading of DBI connections which is good for monitoring of multiple database instances simultaneously. Just configure it, run daemon and it will do all the job. Currently there are template and query set only for Oracle database, but Perl DBI supports any type of RDBMS:

  • Oracle
  • MySQL
  • MS SQL
  • PostgreSQL
  • DB2, etc.

 

DABABIX

DaBaBix is a Zabbix database monitoring agent. It combines the features of postbix and orabbix, but has been rewritten and improved to monitor different database types (currently postgre/mysql/mssql/oracle) and instances with a single daemon instance based on apache daemons (unix/windows service).

ORABBIX

Orabbix was made to monitor oracle instances with zabbix, with this you are going to acquire data from every oracle instances that you want to monitor and then zabbix server is going to produce graphs and collect data. For most of data collected are present some trigger that send mail for each trouble funded or performance problem. It’s incredibly useful to collect data and produce SLA or to have a workload history of your DB.

 

 

 


“Dumping current patch information” in Alert Log 12c can lead to a misinterpretation

Oracle Database 12.1 shows information on applied patches in the alert log during instance startup.

A question come up: The msg info in alert.log (below) is about patch applied on OH only or this information is related to patch applied on Database?

alert.log
...
2016-09-01 18:07:52.476000 -03:00
===========================================================
Dumping current patch information
===========================================================
Patch Id: 23144544
Patch Description: DATABASE BUNDLE PATCH: 12.1.0.2.160719 (23144544)
Patch Apply Time: 2016-08-31 17:44:09 GMT-03:00
...

To get the answer I installed a OH 12.1 without create a database and apply the DATABASE BUNDLE PATCH: 12.1.0.2.160719.

I created a CDB database  with DBCA using template General purpose or transaction processing.

During a database creation  was looking at alert log and I noticed two things.

First the alert log show:

===========================================================
Dumping current patch information
===========================================================
No patches have been applied
===========================================================

Then at end of database creation  alert log show:

...
2016-09-01 18:07:52.476000 -03:00
===========================================================
Dumping current patch information
===========================================================
Patch Id: 23144544
Patch Description: DATABASE BUNDLE PATCH: 12.1.0.2.160719 (23144544)
Patch Apply Time: 2016-08-31 17:44:09 GMT-03:00
...

The patch has been applied on my database during database creation?

The anwser is NO.
To my surprise, the “Dumping current patch information” is about Patches applied on OH only.

Why  “No Patches” the after some time “Show Patches”? 

The patch information is populated by calling DBMS_QOPATCH. It is populated if the database was started in a open mode, as it’s not possible to execute any package with database not opened in read-write.
At first moment database was started in a non-open mode , then NO PATCHES, when database was started in OPEN mode then alertlog was pouplated with the info.

Why this can lead a misinterpretation? 

DBMS_QOPATCH was introduced in 12c to query about Patch information  applied on database, but the info on alert.log is not about patch applied on database, the info is about patch applied on OH only, it can lead us to think wich database have listed patch on alertlog applied.

(I think this is some bug or something else, because it makes no sense to me)

What is the safe mode to check if database have a patch applied?

 

conn as sysdba

SQL> set serverout on
SQL> exec dbms_qopatch.get_sqlpatch_status;

PL/SQL procedure successfully completed.

No outuput then No Patch applied.

After create database I checked if patch was applied and none patch was applied, but the “Dumping current patch information”  show all applied patch in the alertlog at startup of instance.

 

How to apply a BP or PSU on Database 12c?

By executing datapatch utility.

./datapatch -h
SQL Patching tool version 12.1.0.2.0 on Thu Sep 1 19:35:49 2016
Copyright (c) 2016, Oracle. All rights reserved.

sqlpatch usage:
All arguments are optional, if there are no arguments sqlpatch
will automatically determine which SQL scripts need to be run in
order to complete the installation of any SQL patches.

Optional arguments:
-db <db name>
 Use the specified database rather than $ORACLE_SID
-bundle_series <bundle_series>
 Specify if the patch is a bundle patch
 Should also be accompanied by -force option
 if -bundle_series option is specified,only 1 patch will
 be considered by the -force command
-apply <patch1,patch2,...,patchn>
 Only consider the specified patch list for apply operations
-rollback <patch1,patch2,...,patchn>
 Only consider the specified patch list for rollback operations
-upgrade_mode_only
 Only consider patches that require upgrade mode
-force
 Run the apply and/or rollback scripts even if not necessary
 per the SQL registry
-pdbs <pdb1,pdb2,...,pdbn>
 Only consider the specified list of PDBs for patching. All
 other PDBs will not be patched
-prereq
 Run prerequisite checks only, do not actually run any scripts
-oh <oracle_home value>
 Use the specified directory to check for installed patches
-verbose
 Output additional information used for debugging
-help
 Output usage information and exit
-version
 Output build information and exit

Please refer to Note: Datapatch: Database 12c Post Patch SQL Automation
(Doc ID 1585822.1)

Let’s  apply the patch.

cd $ORACLE_HOME/OPatch
./datapatch -verbose

Suprise: When execute ./datapatch -verbose it has no warning or confirmation, the datapatch automatically  will apply patch in  all CDBs and in all opened PDBs of current OH environment.

The same concern that we have about “dd” command, we must have about datapatch command. Hope Oracle fix it.

Output:

./datapatch -verbose
SQL Patching tool version 12.1.0.2.0 on Thu Sep 1 18:28:47 2016
Copyright (c) 2016, Oracle. All rights reserved.

Log file for this invocation: /u01/app/oracle/cfgtoollogs/sqlpatch/
sqlpatch_23003606_2016_09_01_18_28_47/sqlpath_invocation.log

Connecting to database...OK
Note: Datapatch will only apply or rollback SQL fixes for PDBs
 that are in an open state, no patches will be applied to closed PDBs.
 Please refer to Note: Datapatch: Database 12c Post Patch SQL Automation
 (Doc ID 1585822.1)
Bootstrapping registry and package to current versions...done
Determining current state...done

Current state of SQL patches:
Bundle series DBBP:
 ID 160719 in the binary registry and not installed in any PDB

Adding patches to installation queue and performing prereq checks...
Installation queue:
 For the following PDBs: CDB$ROOT PDB$SEED
 Nothing to roll back
 The following patches will be applied:
 23144544 (DATABASE BUNDLE PATCH: 12.1.0.2.160719 (23144544))

Installing patches...
Patch installation complete. Total patches installed: 2

Validating logfiles...
Patch 23144544 apply (pdb CDB$ROOT): SUCCESS
 logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/23144544/20355564/23144544_apply_
PRDCDB_CDBROOT_2016Sep01_18_29_26.log (no errors)
Patch 23144544 apply (pdb PDB$SEED): SUCCESS
 logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/23144544/20355564/23144544_apply_
PRDCDB_PDBSEED_2016Sep01_18_32_36.log (no errors)
SQL Patching tool complete on Thu Sep 1 18:34:54 2016

datapatch always validate if a BP or PSU already was applied  as show below:

./datapatch -verbose
SQL Patching tool version 12.1.0.2.0 on Thu Sep 1 18:35:21 2016
Copyright (c) 2016, Oracle. All rights reserved.

Log file for this invocation: /u01/app/oracle/cfgtoollogs/sqlpatch/
sqlpatch_23593462_2016_09_01_18_35_21/sqlpatch_invocation.log

Connecting to database...OK
Note: Datapatch will only apply or rollback SQL fixes for PDBs
 that are in an open state, no patches will be applied to closed PDBs.
 Please refer to Note: Datapatch: Database 12c Post Patch SQL Automation
 (Doc ID 1585822.1)
Bootstrapping registry and package to current versions...done
Determining current state...done

Current state of SQL patches:
Bundle series DBBP:
 ID 160719 in the binary registry and ID 160719 in PDB CDB$ROOT, 
 ID 160719 in PDB PDB$SEED

Adding patches to installation queue and performing prereq checks...
Installation queue:
 For the following PDBs: CDB$ROOT PDB$SEED
 Nothing to roll back
 Nothing to apply

SQL Patching tool complete on Thu Sep 1 18:35:53 2016

 

Checking what patches have been applied (or rolled back) on database

SQL> set serverout on
SQL> exec dbms_qopatch.get_sqlpatch_status;

Patch Id : 23144544
 Action : APPLY
 Action Time : 01-SEP-2016 18:34:46
 Description : DATABASE BUNDLE PATCH: 12.1.0.2.160719 (23144544)
 Logfile :
/u01/app/oracle/cfgtoollogs/sqlpatch/23144544/20355564/23144544_apply_PRDCDB_CDB
ROOT_2016Sep01_18_29_26.log
 Status : SUCCESS

PL/SQL procedure successfully completed.

 

Some notes:

  • Patch Information in Alert Log is about OH only
  • Creating a Database a CDB or Non-CDB database using a default Template it does not apply any Patch, you must apply patch manually after database creation.
  • datapatch utility is dangerous because has no warning or confirmation when executed.

 


PRCR-1079 CRS-5017 ORA-01017 – DBCA Fails to create a Database in Oracle Restart Environment 12c.

The Oracle Grid Infrastructure software owner (typically, grid) must be a member of the OSDBA group. Membership in the OSDBA group enables access to the files managed by Oracle ASM. If you have a separate OSDBA group for Oracle ASM, then the Oracle Restart software owner must be a member of the OSDBA group for each database and the OSDBA group for Oracle ASM.

http://docs.oracle.com/database/121/LADBI/usr_grps.htm#BABFECII

Issue:

If grid user is not a member of dba os group, then DBCA will fails at end of database creation and DBCA will perform rollback of database creation (database will not be created).

The error below will be raised:

srvctl start database -d prdcdb
PRCR-1079 : Failed to start resource ora.prdcdb.db
ORA-01017: invalid username/password; logon denied
CRS-5017: The resource action "ora.prdcdb.db start" encountered the following
error: ORA-01017: invalid username/password; logon denied
. For details refer to "(:CLSN00107:)" in "/u01/app/grid/diag/crs/db-oracle/crs/
trace/ohasd_oraagent_grid.trc".

 

Solution:

To fix the issue you must add  user “grid” to OSDBA group “dba” and relink RDBMS Binaries.

# id grid
uid=205(grid) gid=202(oinstall) groups=208(asmdba),209(asmadmin),210(asmoper)
# usermod -a -G dba grid (linux)
# id grid
uid=205(grid) gid=202(oinstall) groups=203(dba),208(asmdba),209(asmadmin),
210(asmoper)


# su - oracle
oracle@db-oracle:/home/oracle> export AIXTHREAD_SCOPE=S
oracle@db-oracle:/home/oracle> export ORACLE_BASE=/u01/app/oracle
oracle@db-oracle:/home/oracle> export ORACLE_HOME=$ORACLE_BASE/product/12.1.0/
dbhome_12102
oracle@db-oracle:/home/oracle> export PATH=$ORACLE_HOME/bin:$PATH

oracle@db-oracle:/home/oracle> relink all
writing relink log to: /u01/app/oracle/product/12.1.0/dbhome_12102/
install/relink.log


If there are no errors reported in $ORACLE_HOME/install/relink.log, then retry to create the database with dbca.

 

 

 


ACFS-9459: ADVM/ACFS is not supported on this OS version: ‘unknown’

Oracle Grid Infrastructure is supported on Operation System, but ACFS isn’t. Why?

OLYMPUS DIGITAL CAMERA

The Grid Infrastructure is a Software that use OS Library, so Grid Infrastructure isn’t Kernel dependent but OS Package dependent.

ACFS is a module of Grid Infrastructure that have drivers installed/configured  into OS Kernel, then ACFS is Kernel Dependent.

An Oracle ACFS file system is installed as a dynamically loadable vendor operating system (OS) file system driver and tool set that is developed for each supported operating system platform. The driver is implemented as a Virtual File System (VFS) and processes all file and directory operations directed to a specific file system.

The ACFS  is composed by three components:

The Oracle ACFS, Oracle Kernel Services (OKS) and Oracle ADVM drivers, they are dynamically loaded when the Oracle ASM instance is started.

  • Oracle ACFS
    This driver processes all Oracle ACFS file and directory operations.
  • Oracle ADVM
    This driver provides block device services for Oracle ASM volume files that are used by file systems for creating file systems.
  • Oracle Kernel Services Driver (OKS)
    This driver provides portable driver services for memory allocation, synchronization primitives, and distributed locking services to Oracle ACFS and Oracle ADVM.

Before upgrade your OS Kernel you must place into your check list the ACFS Drivers, as you do with others OS Drives.

How to Oracle support the ACFS into future Kernels?

By releasing Patch Set Updates (PSUs), they are proactive cumulative patches containing recommended bug fixes that are released on a regular and predictable schedule.

How it’s works?

Check this EXAMPLE below:

acfs_release_date

Note: This table is only a example where these kernel and dates are fictitious.

On image above we can see.

In jan/2016 the  GI 11.2.0.4 and 12.0.1.2 don’t need apply PSU because already have supported drives into base release, but 11.2.0.3 need PSU 11.2.0.3.2 to get supported drivers to kernel 2.6.32-100.

In Feb/2016 was released the kernel 2.6.65-30, so ACFS Deployment take some time until developers build new ACFS Drives. So, in FeB/2016 none release is supported for the kernel 2.6.65-30.

In Mar/2016 Oracle release the PSU where all ACFS versions was supported under that PSU.

With example above we can conclude: Does not matter what Grid Infrastructure base version  (such as 11.2.0.4 or 12.0.1.2 ) you are using,  what matter is, the Grid Infrastructure must have supported  ACFS Drivers for that Kernel.

Where to find Certification Matrix for ACFS ?

Oracle Support have a detailed MOS Note : ACFS Support On OS Platforms (Certification Matrix). (Doc ID 1369107.1)

You need ALWAYS check above mos note before any kernel updates for environments that have ACFS Configured.

 


RHEL/OEL 7 – NTPD replaced by Chrony

Despite configuring NTP during the installation process, NTPD is not installed /configured/running after the installation completes.

date_time

New fresh installation of  RHEL/OEL 7 the default NTP Client is the CHRONY.

What is the CHRONY?
Chrony was introduced as new NTP client provided in the chrony package. Chrony does not provides all features available in old ntp client (ntp). So ntp is still provided due to compatibility.

During Oracle Grid Infrastructure Installation the runInstaller will   fail during  prerequisite  check due ntp not configured.

NTP not configured. Network Time Protocol (NTP)  - Failed 
PRVF-7590 : "ntpd" is not running on node "xxx"

Oracle recommend to use NTPD and disable CHRONYD.

Just follow step below to Install/Configure/Enable NTPD

1. Check Chronyd service

# systemctl status chronyd.service
● chronyd.service - NTP client/server
 Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled; vendor preset: enabled)
 Active: active (running) since Mon 2016-05-30 01:15:33 BRT; 5s ago
 Process: 19464 ExecStartPost=/usr/libexec/chrony-helper update-daemon (code=exited, status=0/SUCCESS)
 Process: 19456 ExecStart=/usr/sbin/chronyd $OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 19459 (chronyd)
 CGroup: /system.slice/chronyd.service
 └─19459 /usr/sbin/chronyd

May 30 01:15:33 node1 systemd[1]: Starting NTP client/server...
May 30 01:15:33 node1 chronyd[19459]: chronyd version 2.1.1 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +DEBUG +ASYNCDNS +IPV6 +SECHASH)
May 30 01:15:33 node1 chronyd[19459]: Frequency -23.731 +/- 0.023 ppm read from /var/lib/chrony/drift
May 30 01:15:33 node1 systemd[1]: Started NTP client/server.

2. Stop and disable Chronyd service

# systemctl stop chronyd.service
# systemctl disable chronyd.service
Removed symlink /etc/systemd/system/multi-user.target.wants/chronyd.service.

3. Install ntpd package

# yum install ntp -y

4. Add “-x ” option into  the “/etc/sysconfig/ntpd” file.

OPTIONS="-x -u ntp:ntp -p /var/run/ntpd.pid"

5. Enable and Start NTPD

#systemctl enable ntpd.service
Created symlink from /etc/systemd/system/multi-user.target.wants/ntpd.service to /usr/lib/systemd/system/ntpd.service.


# systemctl start ntpd.service

# systemctl status ntpd.service
● ntpd.service - Network Time Service
 Loaded: loaded (/usr/lib/systemd/system/ntpd.service; enabled; vendor preset: disabled)
 Active: active (running) since Mon 2016-05-30 01:23:09 BRT; 9s ago
 Process: 23048 ExecStart=/usr/sbin/ntpd -u ntp:ntp $OPTIONS (code=exited, status=0/SUCCESS)
 Main PID: 23050 (ntpd)
 CGroup: /system.slice/ntpd.service
 └─23050 /usr/sbin/ntpd -u ntp:ntp -x -u ntp:ntp -p /var/run/ntpd.pid

May 30 00:27:07 node1 ntpd[2829]: Listen normally on 10 virbr0 192.168.122.1 UDP 123
May 30 00:27:07 node1 ntpd[2829]: Listen normally on 11 lo ::1 UDP 123
May 30 00:27:07 node1 ntpd[2829]: Listen normally on 12 eno16777984 fe80::250:56ff:fe90:21e9 UDP 123
May 30 00:27:07 node1 ntpd[2829]: Listen normally on 13 eno33559296 fe80::250:56ff:fe90:d421 UDP 123
May 30 00:27:07 node1 ntpd[2829]: Listen normally on 14 eno50338560 fe80::250:56ff:fe90:f203 UDP 123
May 30 00:27:07 node1 ntpd[2829]: Listening on routing socket on fd #31 for interface updates
May 30 00:27:07 node1 systemd[1]: Started Network Time Service.
May 30 00:27:08 node1 ntpd[2829]: 0.0.0.0 c016 06 restart
May 30 00:27:08 node1 ntpd[2829]: 0.0.0.0 c012 02 freq_set ntpd 0.000 PPM
May 30 00:27:08 node1 ntpd[2829]: 0.0.0.0 c011 01 freq_not_set

Known issues that are solved using above procedure.

# ntpq -p
localhost: timed out, nothing received
***Request timed out
# ntpstat 
Unable to talk to NTP daemon.  Is it running? 

 

 

 


Starting with Oracle In-Memory (aka IM) new Feature


I recommend that DBA be aware of this feature, because this feature will really bring a HUGE difference in performance matter to our databases.

Note: This feature is available starting with Oracle Database 12c Release 1 (12.1.0.2).

Good to Know

  • Is a Option of Oracle Enterprise Edition (12.1.0.2 and above). - Additional license required
  • It is Not in Memory Database - it's an accelerator to our current database
  • It is Not Column Store Database - it allows keeping some of our data in column store which is non-persistent
  • It has nothing to do with Oracle Times-Ten or Oracle Coherence
  • No additional hardware requirement, except additional OS RAM available

Price: Buy it

 

I'm posting useful videos links from Oracle Learning Library.

 

White paper: When to Use Oracle Database In-Memory (PDF)

White paper: Oracle Database In-Memory Advisor Best Practices (PDF)

Blog Official of In-Memory Feature

 


ORAchk Health Checks For The Oracle Stack (ORACHK 2.2.4 and above)


Oracle Database 12c (12.0.1.*) and 11g (11.2.0.4) comes with new feature called RACcheck.
Although the RACcheck already exists before these releases. (In 2011 I posted about benefits of RACcheck)

Brief of RACcheck.
.....
RACcheck - The Oracle RAC Configuration Audit Tool
RACcheck is designed to audit vital configuration settings for the Oracle Database, single instance databases, as well as Oracle Real Application Clusters (Oracle RAC) databases. It also includes checks for Oracle Clusterware, Oracle Automatic Storage Management (Oracle ASM) and Oracle Grid Infrastructure.

RACcheck provides best practices recommedations considering the whole stack, including Maximum Availability Architecture (MAA) configurations and is therefore the ideal tool for regular health checks as well as pre- and post-upgrade best practices assessments.
.....

Now Oracle replace/renamed Oracle RACcheck to ORAchk.

ORAchk- Health Checks for the Oracle Stack

ORAchk replaces the popular RACcheck to support a wider range of products. ORAchk version 2.2.4 is now available for download and includes the following key features:

  • RACcheck renamed to ORAchk
  • ORAchk daemon auto-start mode after node reboot (init integration)
  • Merge multiple ORAchk collection reports
  • Upload of installed patches to database
  • Collection Manager for ORAchk, RACcheck and Exachk (Document 1602329.1)
  • ORAchk signature file in /tmp on all nodes to verify last ORAchk run
  • New checks and bug fixes, including
  • 30 Oracle Ebusiness AP module data integrity checks
  • 12 new Database checks
  • 8 new Solaris system checks
  • Supported Platforms

  • Linux x86-64* (Enterprise Linux, RedHat and SuSE 9, SuSE 10 & SuSE 11)
  • Oracle Solaris SPARC (Solaris 10 and 11)
  • Oracle Solaris x86-64 (Solaris 10 and 11)
  • AIX **
  • HPUX**
  • * 32-bit platforms not supported, no planned support for Linux Itanium
    **Requires BASH Shell 3.2 or higher to be installed

    Supported Oracle Releases

  • 10gR2
  • 11gR1
  • 11gR2
  • 12cR1
  • When to Run ORAchk

  • After initial Oracle RAC deployment
  • Before planned system maintenance
  • After planned system maintenance
  • At least once every three months
  • Install/Configure

    It is recommended to run the tool as the database software owner (e.g. oracle). The user may run the tool as the Grid Infrastructure software owner (e.g. grid) and it will collect the same data but database credentials must manually be supplied to perform the database related audit checks. Typically when run as oracle the customer will have OS authentication set up for the oracle database software owner and the database login credentials will not be needed.

    Download ORAchk

    Stage Location:
    It is recommended that the kit be staged and operated from a local filesystem on a single database server in order to provide the best performance possible.


    $ mkdir -p /u01/app/oracle/orachk

    [oracle@node11g01 install]$ cd /u01/app/oracle/orachk

    [oracle@node11g01 orachk]$ unzip orachk.zip
    Archive: orachk.zip
    inflating: raccheck
    inflating: rules.dat
    inflating: collections.dat
    inflating: readme.txt
    inflating: orachk
    creating: .cgrep/
    inflating: .cgrep/ogghc_12101.sql
    inflating: .cgrep/lcgrep4
    inflating: .cgrep/checkDiskFGMapping.sh
    inflating: .cgrep/ogghc_11204.sql
    inflating: .cgrep/lcgreps9
    inflating: .cgrep/ogghc_11203.sql
    inflating: .cgrep/scgrepx86
    inflating: .cgrep/acgrep
    inflating: .cgrep/oracle-upstarttmpl.conf
    inflating: .cgrep/check_reblance_free_space.sql
    inflating: .cgrep/CollectionManager_App.sql
    inflating: .cgrep/exalogic_zfs_checks.aksh
    inflating: .cgrep/hiacgrep
    inflating: .cgrep/init.tmpl
    inflating: .cgrep/lcgreps10
    inflating: .cgrep/preupgrd.sql
    inflating: .cgrep/diff_collections.pl
    inflating: .cgrep/merge_collections.pl
    inflating: .cgrep/ggdiscovery.sh
    creating: .cgrep/profiles/
    inflating: .cgrep/profiles/DA94919CD0DE0913E04312C0E50A7996.prf
    inflating: .cgrep/profiles/D49C0FBF8FBF4B1AE0431EC0E50A0F24.prf
    extracting: .cgrep/profiles/F13E11974A282AB3E04312C0E50ABCBF.prf
    inflating: .cgrep/profiles/EF6C016813C51366E04313C0E50AE11F.prf
    inflating: .cgrep/profiles/D8367AD6754763FEE04312C0E50A6FCB.prf
    inflating: .cgrep/profiles/DF65D6117CB41054E04312C0E50A69D1.prf
    inflating: .cgrep/profiles/EA5EE324E7E05128E04313C0E50A4B2A.prf
    inflating: .cgrep/profiles/E1BF012E8F210839E04313C0E50A7B68.prf
    inflating: .cgrep/profiles/D462A6F7E9C340FDE0431EC0E50ABE12.prf
    inflating: .cgrep/profiles/D49AD88F8EE75CD8E0431EC0E50A0BC3.prf
    inflating: .cgrep/profiles/E2E972DDE1E14493E04312C0E50A1AB1.prf
    inflating: .cgrep/profiles/F32F44CE0BCD662FE04312C0E50AB058.prf
    inflating: .cgrep/profiles/E8DF76E07DD82E0DE04313C0E50AA55D.prf
    inflating: .cgrep/profiles/D49B218473787400E0431EC0E50A0BB9.prf
    inflating: .cgrep/profiles/D49C0AB26A6D45A8E0431EC0E50ADE06.prf
    inflating: .cgrep/profiles/DFE9C207A8F2428CE04313C0E50A6B0A.prf
    inflating: .cgrep/profiles/D49C4F9F48735396E0431EC0E50A9A0B.prf
    inflating: .cgrep/profiles/D49BDC2EC9E624AEE0431EC0E50A3E12.prf
    inflating: .cgrep/profiles/DF65D0F7FB6F1014E04312C0E50A7808.prf
    inflating: .cgrep/scnhealthcheck.sql
    inflating: .cgrep/pxhcdr.sql
    inflating: .cgrep/lcgrep5
    inflating: .cgrep/scgrep
    inflating: .cgrep/raw_data_browser.pl
    inflating: .cgrep/profiles.dat
    inflating: .cgrep/rack_comparison.py
    inflating: .cgrep/versions.dat
    inflating: .cgrep/create_version.pl
    inflating: .cgrep/lcgreps11
    inflating: .cgrep/utluppkg.sql
    inflating: .cgrep/utlusts.sql
    inflating: .cgrep/reset_crshome.pl
    inflating: .cgrep/asrexacheck
    inflating: .cgrep/lcgrep6
    inflating: .cgrep/utlu112i.sql
    inflating: UserGuide.txt

    Running ORAchk Interactively

    [oracle@node11g01 orachk]$ ./orachk

    CRS stack is running and CRS_HOME is not set. Do you want to set CRS_HOME to /u01/app/11.2.0/grid?[y/n][y]

    Checking ssh user equivalency settings on all nodes in cluster

    Node node11g02 is configured for ssh user equivalency for oracle user

    Searching for running databases . . . . .

    . .
    List of running databases registered in OCR
    1. dborcl
    2. None of above

    Select databases from list for checking best practices. For multiple databases, select 1 for All or comma separated number like 1,2 etc [1-2][1].1
    . .

    Checking Status of Oracle Software Stack - Clusterware, ASM, RDBMS

    . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
    -------------------------------------------------------------------------------------------------------
    Oracle Stack Status
    -------------------------------------------------------------------------------------------------------
    Host Name CRS Installed ASM HOME RDBMS Installed CRS UP ASM UP RDBMS UP DB Instance Name
    -------------------------------------------------------------------------------------------------------
    node11g01 Yes Yes Yes Yes Yes Yes dborcl_1
    node11g02 Yes Yes Yes Yes Yes Yes dborcl_2
    -------------------------------------------------------------------------------------------------------

    Copying plug-ins

    . . . . . . . . . . . . . . . . . . . . . . . . . . .

    . . . . . .

    17 of the included audit checks require root privileged data collection . If sudo is not configured or the root password is not available, audit checks which require root privileged data collection can be skipped.

    1. Enter 1 if you will enter root password for each host when prompted

    2. Enter 2 if you have sudo configured for oracle user to execute root_orachk.sh script

    3. Enter 3 to skip the root privileged collections

    4. Enter 4 to exit and work with the SA to configure sudo or to arrange for root access and run the tool later.

    Please indicate your selection from one of the above options for root access[1-4][1]:- 2

    *** Checking Best Practice Recommendations (PASS/WARNING/FAIL) ***

    Collections and audit checks log file is
    /u01/app/oracle/orachk/orachk_node11g01_dborcl_031814_170552/log/orachk.log
    .
    .
    .
    .
    ---------------------------------------------------------------------------------
    CLUSTERWIDE CHECKS
    ---------------------------------------------------------------------------------
    ---------------------------------------------------------------------------------

    Detailed report (html) - /u01/app/oracle/orachk/orachk_node11g01_dborcl_031814_170552/orachk_node11g01_dborcl_031814_170552.html

    UPLOAD(if required) - /u01/app/oracle/orachk/orachk_node11g01_dborcl_031814_170552.zip


    Below the report generated in this test.
    Oracle RAC Assessment Report

    orachk usage options

    [oracle@node11g01 orachk]$ ./orachk -h
    Usage : ./orachk [-abvhpfmsuSo:c:t:]
    -a All (Perform best practice check and recommended patch check)
    -b Best Practice check only. No recommended patch check
    -h Show usage
    -v Show version
    -p Patch check only
    -m exclude checks for Maximum Availability Architecture (MAA) scorecards(see user guide for more details)
    -u Run orachk to check pre-upgrade or post-upgrade best practices for 11.2.0.3,11.2.0.4.0 and 12.1.0.1
    -o pre or -o post is mandatory with -u option like ./orachk -u -o pre
    -f Run Offline.Checks will be performed on data already collected from the system
    -o Argument to an option. if -o is followed by v,V,Verbose,VERBOSE or Verbose, it will print checks which passs on the screen
    if -o option is not specified,it will print only failures on screen. for eg: orachk -a -o v

    -clusternodes
    Pass comma separated node names to run orachk only on subset of nodes.
    -dbnames
    Pass comma separated database names to run orachk only on subset of databases
    -localonly
    Run orachk only on local node.
    -debug
    Run orachk in debug mode. Debug log will be generated.
    eg:- ./orachk -debug
    -nopass
    Skip PASS'ed check to print in orachk report and upload to database.

    -noscore
    Do not print healthscore in HTML report.

    -diff [-outfile ]
    Diff two orachk reports. Pass directory name or zip file or html report file as &

    -exadiff
    Compare two different Exalogic rack and see if both are from the same release.Pass directory name or zip file as & (applicable for Exalogic only)

    -c Used only under the guidance of Oracle support or development to override default components

    -
    initsetup : Setup auto restart. Auto restart functionality automatically brings up orachk daemon when node starts
    initrmsetup : Remove auto restart functionality
    initcheck : Check if auto restart functionality is setup or not
    initpresetup : Sets root user equivalency for COMPUTE, STORAGE and IBSWITCHES.(root equivalency for COMPUTE nodes is mandatory for setting up auto restart functionality)

    -d
    start : Start the orachk daemon
    start_debug : Start the orachk daemon in debug mode
    stop : Stop the orachk daemon
    status : Check if the orachk daemon is running
    info : Print information about running orachk daemon
    stop_client : Stop the orachk daemon client
    nextautorun : print the next auto run time
    -daemon
    run orachk only if daemon is running
    -nodaemon
    Dont use daemon to run orachk
    -set
    configure orachk daemon parameter like "param1=value1;param2=value2... "

    Supported parameters are:-

    AUTORUN_INTERVAL :- Automatic rerun interval in daemon mode.Set it zero to disable automatic rerun which is zero.

    AUTORUN_SCHEDULE * * * * :- Automatic run at specific time in daemon mode.
    - - - -
    ▒ ▒ ▒ ▒
    ▒ ▒ ▒ +----- day of week (0 - 6) (0 to 6 are Sunday to Saturday)
    ▒ ▒ +---------- month (1 - 12)
    ▒ +--------------- day of month (1 - 31)
    +-------------------- hour (0 - 23)

    example: orachk -set "AUTORUN_SCHEDULE=8,20 * * 2,5" will schedule runs on tuesday and friday at 8 and 20 hour.

    AUTORUN_FLAGS : orachk flags to use for auto runs.

    example: orachk -set "AUTORUN_INTERVAL=12h;AUTORUN_FLAGS=-profile sysadmin" to run sysadmin profile every 12 hours

    orachk -set "AUTORUN_INTERVAL=2d;AUTORUN_FLAGS=-profile dba" to run dba profile once every 2 days.

    NOTIFICATION_EMAIL : Comma separated list of email addresses used for notifications by daemon if mail server is configured.

    PASSWORD_CHECK_INTERVAL : Interval to verify passwords in daemon mode

    COLLECTION_RETENTION : Purge orachk collection directories and zip files older than specified days.

    -unset
    unset the parameter
    example: orachk -unset "AUTORUN_SCHEDULE"

    -get
    Print the value of parameter

    -excludeprofile
    Pass specific profile.
    List of supported profiles is same as for -profile.

    -merge
    Pass comma separated collection names(directory or zip files) to merge collections and prepare single report.
    eg:- ./orachk -merge orachk_hostname1_db1_120213_163405.zip,orachk_hostname2_db2_120213_164826.zip

    -vmguest
    Pass comma separated filenames containing exalogic guest VM list(applicable for Exalogic only)

    -hybrid [-phy]
    phy :Pass comma separated physical compute nodes(applicable for Exalogic only)
    eg:- ./orachk -hybrid -phy phy_node1,phy_node2

    -profile Pass specific profile.
    List of supported profiles:

    asm asm Checks
    clusterware Oracle clusterware checks
    compute_node Compute Node checks (Exalogic only)
    control_VM Checks only for Control VM(ec1-vm, ovmm, db, pc1, pc2). No cross node checks
    dba dba Checks
    ebs Oracle E-Business Suite checks
    el_extensive Extensive EL checks
    el_lite Exalogic-Lite Checks(Exalogic Only)
    el_rackcompare Data Collection for Exalogic Rack Comparison Tool(Exalogic Only)
    goldengate Oracle GoldenGate checks
    maa Maximum Availability Architecture Checks
    obiee obiee Checks(Exalytics Only)
    storage Oracle Storage Server Checks
    switch Infiniband switch checks
    sysadmin sysadmin checks
    timesten timesten Checks(Exalytics Only)
    virtual_infra OVS, Control VM, NTP-related and stale VNICs check (Exalogic Only)
    zfs ZFS storage appliances checks (Exalogic Only)

    -cells
    Pass comma separated storage server names to run orachk only on selected storage servers.

    -ibswitches
    Pass comma separated infiniband switch names to run orachk only on selected infiniband switches.

    -zfsnodes
    Pass comma separated ZFS storage appliance names to run orachk only on selected storage appliances.

    ORAchk Other Useful Options Not Covered Here

  • Using ORAchk Silently
  • ORAchk can be optionally run in “silent” or “non-interactive” mode in order to enable scheduling and automation
    Is required only if customer does not want to use orachk daemon functionality.

  • Using ORAchk Daemon Mode Operation
  • This functionality permit non-interactive (batch or silent mode) execution on a regular interval.

    When running ORAchk in daemon mode, the most recent and next most recent (if any) collection reports are automatically compared. If the mail address is configured a summary will be emailed along with attachments for the reports and the comparison report.

  • Report Comparisons with ORAchk
  • ORAchk has the ability to perform report comparisons between 2 ORAchk reports.
    This allows for trending of Success Factor and Best Practice changes over time, after planned maintenance, etc within a user friendly HTML report.

  • ORAchk in Upgrade Readiness Mode
  • ORAchk can be used to obtain an automated 11.2.0.3 (or above) Upgrade Readiness Assessment.
    The goal of the ORAchk Upgrade Readiness Assessment is to make the process of upgrade planning for Oracle RAC and Oracle Clusterware target versions 11.2.0.3 and above as smooth as possible by automating many of the
    manual pre and post checks detailed in various upgrade related documents.

    Refer MoS Notes for more details:
    ORAchk - Oracle Configuration Audit Tool (Doc ID 1268927.2)

    ORAchk Users Guide
    For details instructions on how to run ORAchk including troubleshooting steps, available options, etc.

    Enjoy!!!