Friday, August 3, 2012


Applying Grid Infrastructure Patch Set Update aka GI PSU to 11.2.0.1 Grid home and 11.2.0.1 RDBMS home

Environment:
2 Node RAC running SUSE Linux Enterprise Server 10 (x86_64)
Oracle Grid Infrastructure 11.2.0.1
Oracle RAC rdbms 11.2.0.1
Grid and Oracle database server software owner account is OS user “oracle”


PSU - What/ When ??

Patch Set Updates (PSUs) are proactive cumulative patches containing recommended bug fixes that are released every quarter on the Tuesday closest to the 17th of January, April, July, and October.

Identifying the right PSU for your environment

-- First, find my version of Grid Infrastructure software.

> crsctl query crs softwareversion
Oracle Clusterware version on node [labrac40] is [11.2.0.1.0]


-- Next, find the current recommended PSU for your version of software ?

Use support note, Oracle Recommended Patches -- Oracle Database [ID 756671.1]

As of date, the recommended patch for 11.2.0.1 is Patch 9655006

Grid Infrastructure
Document
Description
Rolling RAC
Patch Download
Note:9655006.8
11.2.0.1.2 Grid Infrastructure Patch Set Update (GI PSU)
Yes
Patch: 9655006


Obtain the Patch

Download the patch from support.oracle.com for the OS version that you are running. Place the downloaded files in a shared location that is accessible from both the nodes in the cluster configuration. This will make it easy as we will use the same source software for applying in all nodes of the RAC.


Patch Installation Prerequisites

Note: Always use the readme.html from the patch source. It contains complete details required for applying the patch successfully.

Latest OPatch
               
OPatch is the utility used to apply the patch. Download the latest OPatch utility from My Oracle Support patch 6880880.

Lets check my current OPatch version

> export CRS_ORA_HOME=/u00/app/11.2.0/grid
labrac40 | ORA1120 | /export/install/nels/GI_PSU_11.2.0.1.2
> export ORACLE_HOME=/u00/app/oracle/product/11.2.0.1
labrac40 | ORA1120 | /export/install/nels/GI_PSU_11.2.0.1.2
> export PATH=$PATH:$CRS_ORA_HOME/OPatch
labrac40 | ORA1120 | /export/install/nels/GI_PSU_11.2.0.1.2
> which opatch
/u00/app/11.2.0/grid/OPatch/opatch
labrac40 | ORA1120 | /export/install/nels/GI_PSU_11.2.0.1.2
> opatch version
Invoking OPatch 11.1.0.6.6

OPatch Version: 11.1.0.6.6

OPatch succeeded.


As Grid home owner, unzip the downloaded OPatch to the $CRS_ORA_HOME and $ORACLE_HOME

> unzip p6880880_112000_Linux-x86-64.zip -d $CRS_ORA_HOME

> unzip p6880880_112000_Linux-x86-64.zip -d $ORACLE_HOME


Check the OPatch version

> opatch version
OPatch Version: 11.2.0.3.0

OPatch succeeded.


OCM Configuration

During patch apply, OPatch will prompt for the location of the OCM (Oracle Configuration Manager) response file. If you do not have one, run the following command to create it.

> $CRS_ORA_HOME/OPatch/opatch/ocm/bin/emocmrsp

Make a note of the response file location as you will need it.

Validation of Oracle Inventory

Before patch application, let check the consistency of the oracle inventory. If the below command succeeds, we are good.

> $CRS_ORA_HOME/OPatch/opatch lsinventory -detail -oh $CRS_ORA_HOME
Oracle Interim Patch Installer version 11.2.0.3.0
Copyright (c) 2012, Oracle Corporation.  All rights reserved.


Oracle Home       : /u00/app/11.2.0/grid
Central Inventory : /u00/app/oraInventory
   from           : /u00/app/11.2.0/grid/oraInst.loc
OPatch version    : 11.2.0.3.0
OUI version       : 11.2.0.1.0
Log file location : /u00/app/11.2.0/grid/cfgtoollogs/opatch/opatch2012-07-18_13-35-27PM_1.log

Lsinventory Output file location : /u00/app/11.2.0/grid/cfgtoollogs/opatch/lsinv/lsinventory2012-07-18_13-35-27PM.txt

--------------------------------------------------------------------------------
Installed Top-level Products (1):

Oracle Grid Infrastructure                                           11.2.0.1.0
There are 1 products installed in this Oracle Home.


Installed Products (87):

Agent Required Support Files                                         10.2.0.4.2
Assistant Common Files                                               11.2.0.1.0
Automatic Storage Management Assistant                               11.2.0.1.0
Bali Share                                                           1.1.18.0.0
Buildtools Common Files                                              11.2.0.1.0
Character Set Migration Utility                                      11.2.0.1.0
Cluster Ready Services Files                                         11.2.0.1.0
Cluster Verification Utility Common Files                            11.2.0.1.0
Cluster Verification Utility Files                                   11.2.0.1.0
Database SQL Scripts                                                 11.2.0.1.0
Deinstallation Tool                                                  11.2.0.1.0
Enterprise Manager Common Core Files                                 10.2.0.4.2
Enterprise Manager Common Files                                      10.2.0.4.2
Enterprise Manager Minimal Integration                               11.2.0.1.0
Enterprise Manager plugin Common Files                               11.2.0.1.0
Expat libraries                                                       2.0.1.0.1
HAS Common Files                                                     11.2.0.1.0
HAS Files for DB                                                     11.2.0.1.0
Installation Common Files                                            11.2.0.1.0
Installation Plugin Files                                            11.2.0.1.0
Installer SDK Component                                              11.2.0.1.0
LDAP Required Support Files                                          11.2.0.1.0
OLAP SQL Scripts                                                     11.2.0.1.0
Oracle Clusterware RDBMS Files                                       11.2.0.1.0
Oracle Configuration Manager Deconfiguration                         10.3.1.0.0
Oracle Containers for Java                                           11.2.0.1.0
Oracle Core Required Support Files                                   11.2.0.1.0
Oracle Database 11g                                                  11.2.0.1.0
Oracle Database 11g Multimedia Files                                 11.2.0.1.0
Oracle Database Deconfiguration                                      11.2.0.1.0
Oracle Database User Interface                                       2.2.13.0.0
Oracle Database Utilities                                            11.2.0.1.0
Oracle DBCA Deconfiguration                                          11.2.0.1.0
Oracle Extended Windowing Toolkit                                    3.4.47.0.0
Oracle Globalization Support                                         11.2.0.1.0
Oracle Globalization Support                                         11.2.0.1.0
Oracle Grid Infrastructure                                           11.2.0.1.0
Oracle Help For Java                                                  4.2.9.0.0
Oracle Ice Browser                                                    5.2.3.6.0
Oracle Internet Directory Client                                     11.2.0.1.0
Oracle Java Client                                                   11.2.0.1.0
Oracle JDBC/OCI Instant Client                                       11.2.0.1.0
Oracle JDBC/THIN Interfaces                                          11.2.0.1.0
Oracle JFC Extended Windowing Toolkit                                4.2.36.0.0
Oracle JVM                                                           11.2.0.1.0
Oracle LDAP administration                                           11.2.0.1.0
Oracle Locale Builder                                                11.2.0.1.0
Oracle Multimedia                                                    11.2.0.1.0
Oracle Multimedia Client Option                                      11.2.0.1.0
Oracle Multimedia Java Advanced Imaging                              11.2.0.1.0
Oracle Multimedia Locator                                            11.2.0.1.0
Oracle Multimedia Locator RDBMS Files                                11.2.0.1.0
Oracle Net                                                           11.2.0.1.0
Oracle Net Listener                                                  11.2.0.1.0
Oracle Net Required Support Files                                    11.2.0.1.0
Oracle Netca Client                                                  11.2.0.1.0
Oracle Notification Service                                          11.2.0.0.0
Oracle Notification Service (eONS)                                   11.2.0.1.0
Oracle One-Off Patch Installer                                       11.2.0.0.2
Oracle Quality of Service Management (Client)                        11.2.0.1.0
Oracle Quality of Service Management (Server)                        11.2.0.1.0
Oracle RAC Deconfiguration                                           11.2.0.1.0
Oracle RAC Required Support Files-HAS                                11.2.0.1.0
Oracle Recovery Manager                                              11.2.0.1.0
Oracle Security Developer Tools                                      11.2.0.1.0
Oracle Text Required Support Files                                   11.2.0.1.0
Oracle Universal Installer                                           11.2.0.1.0
Oracle Wallet Manager                                                11.2.0.1.0
Parser Generator Required Support Files                              11.2.0.1.0
Perl Interpreter                                                     5.10.0.0.1
Perl Modules                                                         5.10.0.0.1
PL/SQL                                                               11.2.0.1.0
PL/SQL Embedded Gateway                                              11.2.0.1.0
Platform Required Support Files                                      11.2.0.1.0
Precompiler Required Support Files                                   11.2.0.1.0
RDBMS Required Support Files                                         11.2.0.1.0
RDBMS Required Support Files for Instant Client                      11.2.0.1.0
Required Support Files                                               11.2.0.1.0
Secure Socket Layer                                                  11.2.0.1.0
SQL*Plus                                                             11.2.0.1.0
SQL*Plus Files for Instant Client                                    11.2.0.1.0
SQL*Plus Required Support Files                                      11.2.0.1.0
SSL Required Support Files for InstantClient                         11.2.0.1.0
Sun JDK                                                              1.5.0.17.0
Universal Storage Manager Files                                      11.2.0.1.0
XDK Required Support Files                                           11.2.0.1.0
XML Parser for Java                                                  11.2.0.1.0
There are 87 products installed in this Oracle Home.


There are no Interim patches installed in this Oracle Home.


Rac system comprising of multiple nodes
  Local node = labrac40
  Remote node = labrac41

--------------------------------------------------------------------------------

OPatch succeeded.




Let’s do the same for the RDBMS home. I’m not listing the whole output again here.

> $ORACLE_HOME/OPatch/opatch lsinventory -detail -oh $ORACLE_HOME


One-off Patch Conflict Detection and Resolution

Check for conflicting patches by doing the following for each GI and RDBMS home you are planning to apply the patch on.

Use My Oracle Support Note 1061295.1 Patch Set Updates - One-off Patch Conflict Resolution to determine, for each conflicting patch, whether a conflict resolution patch is already available, and if you need to request a new conflict resolution patch or if the conflict may be ignored.

labrac40 | ORA1120 | /export/install/nels/GI_PSU_11.2.0.1.2
> opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir ./9655006
Oracle Interim Patch Installer version 11.2.0.3.0
Copyright (c) 2012, Oracle Corporation.  All rights reserved.

PREREQ session

Oracle Home       : /u00/app/oracle/product/11.2.0.1
Central Inventory : /u00/app/oraInventory
   from           : /u00/app/oracle/product/11.2.0.1/oraInst.loc
OPatch version    : 11.2.0.3.0
OUI version       : 11.2.0.1.0
Log file location : /u00/app/oracle/product/11.2.0.1/cfgtoollogs/opatch/opatch2012-07-18_13-40-19PM_1.log

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.


For each RAC home

>opatch prereq CheckConflictAgainstOHWithDetail -oh $ORACLE_HOME -phBaseDir <UNZIPPED_PATCH_LOCATION>/9655006

For each GI home

>opatch prereq CheckConflictAgainstOHWithDetail -oh $CRS_ORA_HOME -phBaseDir <UNZIPPED_PATCH_LOCATION>/9655006



Patch Installation

Note:

-- Stop EM agent if it is running on your server before you apply the patch.
-- Make sure there is enough free space available in your GI and RDBMS home volume
-- If the available free space is less, try to use ‘opatch util cleanup’ to clean old backup files from previous patching. It is usually a hidden directory in GI_HOME or ORACLE_HOME called .patch_storage

syntax:

#opatch auto <UNZIPPED_PATCH_LOCATION> -oh <path to GI home>

Apply OPatch auto as ROOT user on Grid Infrastructure home

During this process all services running on the node will be taken offline, patch applied and brought up online by Oracle

labrac40 | ORA1120 | /home/oracle
> export CRS_ORA_HOME=/u00/app/11.2.0/grid
labrac40 | ORA1120 | /home/oracle
> export ORACLE_HOME=/u00/app/oracle/product/11.2.0.1
labrac40 | ORA1120 | /home/oracle
> export PATH=$PATH:$CRS_ORA_HOME/OPatch
labrac40 | ORA1120 | /home/oracle
> 


[root@labrac40 ~]# id
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel)
[root@labrac40 ~]# opatch auto /export/install/nels/GI_PSU_11.2.0.1.2 -oh /u00/app/11.2.0/grid
Executing /usr/bin/perl /u00/app/11.2.0/grid/OPatch/crs/patch112.pl -patchdir /export/install/nels -patchn GI_PSU_11.2.0.1.2 -oh /u00/app/11.2.0/grid -paramfile /u00/app/11.2.0/grid/crs/install/crsconfig_params
opatch auto log file location is /u00/app/11.2.0/grid/OPatch/crs/../../cfgtoollogs/opatchauto2012-07-18_15-10-48.log
Detected Oracle Clusterware install
Using configuration parameter file: /u00/app/11.2.0/grid/crs/install/crsconfig_params
OPatch  is bundled with OCM, Enter the absolute OCM response file path:
/home/oracle/ocm.rsp

Unable to determine if /u00/app/11.2.0/grid is shared oracle home
Enter 'yes' if this is not a shared home or if the prerequiste actions are performed to patch this shared home (yes/no):yes
Successfully unlock /u00/app/11.2.0/grid
patch /export/install/nels/GI_PSU_11.2.0.1.2/9655006  apply successful for home  /u00/app/11.2.0/grid
patch /export/install/nels/GI_PSU_11.2.0.1.2/9654983  apply successful for home  /u00/app/11.2.0/grid
ACFS-9300: ADVM/ACFS distribution files found.
ACFS-9312: Existing ADVM/ACFS installation detected.
ACFS-9314: Removing previous ADVM/ACFS installation.
ACFS-9315: Previous ADVM/ACFS components successfully removed.
ACFS-9307: Installing requested ADVM/ACFS software.
ACFS-9308: Loading installed ADVM/ACFS drivers.
ACFS-9321: Creating udev for ADVM/ACFS.
ACFS-9323: Creating module dependencies - this may take some time.
ACFS-9327: Verifying ADVM/ACFS devices.
ACFS-9309: ADVM/ACFS installation correctness verified.
CRS-4123: Oracle High Availability Services has been started.


Apply OPatch auto as ROOT user on Oracle RDBMS home

[root@labrac40 ~]# export PATH=$PATH:$ORACLE_HOME/OPatch
[root@labrac40 ~]# which opatch
/u00/app/11.2.0/grid/OPatch/opatch
[root@labrac40 ~]# opatch auto /export/install/nels/GI_PSU_11.2.0.1.2 -oh /u00/app/oracle/product/11.2.0.1
Executing /usr/bin/perl /u00/app/11.2.0/grid/OPatch/crs/patch112.pl -patchdir /export/install/nels -patchn GI_PSU_11.2.0.1.2 -oh /u00/app/oracle/product/11.2.0.1 -paramfile /u00/app/11.2.0/grid/crs/install/crsconfig_params
opatch auto log file location is /u00/app/11.2.0/grid/OPatch/crs/../../cfgtoollogs/opatchauto2012-07-18_15-17-39.log
Detected Oracle Clusterware install
Using configuration parameter file: /u00/app/11.2.0/grid/crs/install/crsconfig_params
OPatch  is bundled with OCM, Enter the absolute OCM response file path:
/home/oracle/ocm.rsp

Unable to determine if /u00/app/oracle/product/11.2.0.1 is shared oracle home
Enter 'yes' if this is not a shared home or if the prerequiste actions are performed to patch this shared home (yes/no):yes
patch /export/install/nels/GI_PSU_11.2.0.1.2/9655006/custom/server/9655006  apply successful for home  /u00/app/oracle/product/11.2.0.1
patch /export/install/nels/GI_PSU_11.2.0.1.2/9654983  apply successful for home  /u00/app/oracle/product/11.2.0.1
[root@labrac40 ~]#


Check Applied Patch


labrac40 | ORA1120 | /u00/app/11.2.0/grid/OPatch
> $ORACLE_HOME/OPatch/opatch lsinventory -bugs_fixed | grep -i 'DATABASE PSU'
9352237    9654983   Wed Jul 18 15:20:39 EDT 2012   DATABASE PSU 11.2.0.1.1
9654983    9654983   Wed Jul 18 15:20:39 EDT 2012   DATABASE PSU 11.2.0.1.2 (INCLUDES CPUJUL2010)
9655006    9655006   Wed Jul 18 15:20:01 EDT 2012   GI PSU 11.2.0.1.2 (INCLUDES DATABASE PSU 11.2.0.1.


labrac40 | ORA1120 | /u00/app/11.2.0/grid/OPatch
> $CRS_ORA_HOME/OPatch/opatch lsinventory -bugs_fixed |grep -i 'GI PSU'
9655006    9655006   Wed Jul 18 15:20:01 EDT 2012   GI PSU 11.2.0.1.2 (INCLUDES DATABASE PSU 11.2.0.1.
9778840    9655006   Wed Jul 18 15:20:01 EDT 2012   TRACKING BUG FOR GI PSU( THIS IS REPLACEMENT FOR
                                                    APRIL GI PSU 9343627)


Perform all the above steps on the second node, one node at a time in a rolling fashion to install the patch in all the servers.

Patch Post Installation Steps

For each database instance running on the Oracle home being patched, connect to the database using SQL*Plus. Connect as SYSDBA and run the catbundle.sql script


cd $ORACLE_HOME/rdbms/admin
sqlplus /nolog
SQL> CONNECT / AS SYSDBA
SQL> @catbundle.sql psu apply
SQL> QUIT


Check the log files in $ORACLE_HOME/cfgtoollogs/catbundle for any errors



No comments:

Post a Comment