Thursday, December 22, 2011

Statspack in 11g.

You need Oracle tuning pack/diagnostic pack licences if you want to use Grid control or Database control performance analyzer.

Statspack is free utility for performance monitoring and reporting and it's there since Oracle 8i. Lets start with Installation of Statspack in 11gR2 DB.

- You can create separate tablespace for Statspack objects or use existing tablespsace too.

PERFSTATS is the owner for Statspack repository and this user will be created automatically during Installation.

SQL> connect / as sysdba
Connected.
SQL> @?/rdbms/admin/spcreate
Choose the PERFSTAT user's password
-----------------------------------
Not specifying a password will result in the installation FAILING
Enter value for perfstat_password:

Choose the Default tablespace for the PERFSTAT user
---------------------------------------------------
Below is the list of online tablespaces in this database which can
store user data. Specifying the SYSTEM tablespace for the user's
default tablespace will result in the installation FAILING, as
using SYSTEM for performance data is not supported.
Choose the PERFSTAT users's default tablespace. This is the tablespace
in which the STATSPACK tables and indexes will be created.

TABLESPACE_NAME CONTENTS STATSPACK DEFAULT TABLESPACE
------------------------------ --------- ----------------------------
SYSAUX PERMANENT *
TOOLS PERMANENT
USERS PERMANENT

Pressing <return> will result in STATSPACK's recommended default
tablespace (identified by *) being used.
Enter value for default_tablespace:
Using tablespace SYSAUX as PERFSTAT default tablespace.
 
Choose the Temporary tablespace for the PERFSTAT user
-----------------------------------------------------

Below is the list of online tablespaces in this database which can
store temporary data (e.g. for sort workareas). Specifying the SYSTEM
tablespace for the user's temporary tablespace will result in the
installation FAILING, as using SYSTEM for workareas is not supported.
Choose the PERFSTAT user's Temporary tablespace.
TABLESPACE_NAME CONTENTS DB DEFAULT TEMP TABLESPACE
------------------------------ --------- --------------------------
TEMP TEMPORARY *

Pressing <return> will result in the database's default Temporary
tablespace (identified by *) being used.

Enter value for temporary_tablespace:
....
...
No errors.
NOTE:

SPCPKG complete. Please check spcpkg.lis for any errors.
SQL>
 
It's time to connect to PERFSTAT user and take snapshot. You need snapshot before and the end of the period you want to analyze. You can schedule this to run every hour. That's what AWR does automatically for you.
$ sqlplus perfstat/perfstat
SQL> exec statspack.snap;
PL/SQL procedure successfully completed.
 
run you load or wait until period you are monitoring is over and take another snapshot.
SQL> exec statspack.snap;
PL/SQL procedure successfully completed
It's time to run Statspack reports.
SQL> @?/rdbms/admin/spreport
Current Instance
~~~~~~~~~~~~~~~~
DB Id DB Name Inst Num Instance
----------- ------------ -------- ------------
961341030 ORCL 1 ORCL
 
 
Instances in this Statspack schema
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
DB Id Inst Num DB Name Instance Host
----------- -------- ------------ ------------ ------------
961341030 1 ORCL ORCL server1
Using 961341030 for database Id
Using 1 for instance number
 
Specify the number of days of snapshots to choose from
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Entering the number of days (n) will result in the most recent
(n) days of snapshots being listed. Pressing <return> without
specifying a number lists all completed snapshots.
 
 
Listing all Completed Snapshots
Snap
Instance DB Name Snap Id Snap Started Level Comment
------------ ------------ --------- ----------------- ----- --------------------
ORCL ORCL 1 21 Dec 2011 15:00 5
2 22 Dec 2011 16:00 5Specify the Begin and End Snapshot Ids
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Enter value for begin_snap: 1
Begin Snapshot Id specified: 1
Enter value for end_snap: 2
End Snapshot Id specified: 2
 
 
Specify the Report Name
~~~~~~~~~~~~~~~~~~~~~~~
The default report file name is sp_1_2. To use this name,
press <return> to continue, otherwise enter an alternative.
Enter value for report_name: test
...
...
End of Report ( test.lst )
SQL>
This will create OS file "test.lst" and time to do analysis.



Good Luck!
 
 
 
 

Monday, November 21, 2011

11g Clone - Caution! Do NOT run manual restore commands after failed 11g duplicate

In previous versions (10g or 9i)  we used to run manual commands if something goes wrong during  RMAN duplicate restore. For example:

SQL > Alter database open resetlogs
or
SQL > recover database until cancel;

In 11g this process runs differently and you should never run manual commands if you trying to clone production DB on same server.The reason is that during 11g duplication RMAN rename the auxiliary instance name same as target instance and it may overwrite you logfiles if you try to open database with resetlog option.

Here are the steps 11g RMAN follows during 11g duplication. Please look into my blog http://vkoracle.blogspot.com/2010/11/duplicate-10gr2-rac-database-instances.html
 on database duplication.

1.Set the clone instance name same as target instance name.

2. Restore database.

3. recover database.

4. Set the instance name to new clone Instance name.

5. Open database with resetlogs.



Solution: Run 11g duplicate command again and it will resume from point of failure.

If you run into some problems then fix the problem and run the clone commands again and 11g will resume from point of failure.

Thursday, September 22, 2011

ORA-29701: unable to connect to Cluster Synchronization Service

While starting ASM on 11g grid Infrastructure I was getting following errors :

SQL> connect / as sysasm
Connected to an idle instance.
SQL> startup;
ORA-01078: failure in processing system parameters
ORA-29701: unable to connect to Cluster Synchronization Service
SQL>


Login with the account under which grid Infrastructure is install. In my case it's grid account and check if CSS is up and running.
$ ./crs_stat -t

Name           Type           Target    State     Host       
------------------------------------------------------------
ora.DATA.dg    ora....up.type OFFLINE   OFFLINE              
ora.DATA2.dg   ora....up.type OFFLINE   OFFLINE              
ora....ER.lsnr ora....er.type ONLINE    ONLINE    node2  
ora.asm        ora.asm.type   OFFLINE   OFFLINE              
ora.cssd       ora.cssd.type  ONLINE    OFFLINE              
ora.diskmon    ora....on.type ONLINE    OFFLINE       

Looks like crs not started or autostart is diabled after server reboot. Let check this one out
root@node2 # ./crsctl enable crs
CRS-4013: This command is not supported in a single-node configuration.
CRS-4000: Command Enable failed, or completed with errors.

enable and disabled options are not supported with single-node configuration. Let start all the resources manually.

grid@node2$ $ ./crsctl start resource -all
CRS-5702: Resource 'ora.LISTENER.lsnr' is already running on 'node2'
CRS-2672: Attempting to start 'ora.cssd' on 'node2'
CRS-2679: Attempting to clean 'ora.diskmon' on 'node2'
CRS-2681: Clean of 'ora.diskmon' on 'node2' succeeded
CRS-2672: Attempting to start 'ora.diskmon' on 'node2'
CRS-2676: Start of 'ora.diskmon' on 'node2' succeeded
CRS-2676: Start of 'ora.cssd' on 'node2' succeeded
CRS-2672: Attempting to start 'ora.asm' on 'node2'
CRS-2676: Start of 'ora.asm' on 'node2' succeeded
CRS-2672: Attempting to start 'ora.DATA.dg' on 'node2'
CRS-2672: Attempting to start 'ora.DATA2.dg' on 'node2'
CRS-2676: Start of 'ora.DATA.dg' on 'node2' succeeded
CRS-2676: Start of 'ora.DATA2.dg' on 'node2' succeeded

$ ps -ef | grep cssd

$ ./crs_stat -t
Name           Type           Target    State     Host       
------------------------------------------------------------
ora.DATA.dg    ora....up.type ONLINE    ONLINE    node2  
ora.DATA2.dg   ora....up.type ONLINE    ONLINE    node2  
ora....ER.lsnr ora....er.type ONLINE    ONLINE    node2  
ora.asm        ora.asm.type   ONLINE    ONLINE    node2  
ora.cssd       ora.cssd.type  ONLINE    ONLINE    node2  
ora.diskmon    ora....on.type ONLINE    ONLINE    node2  

back to business!


Friday, July 22, 2011

root.sh failed during grid Infrastructure Installation.


In my case it was a permission issue with one of the disk to verify any permission issues run the following command as grid user

 $ dd if=/dev/zero of=/dev/rdsk/emcpower6e count=1440


sh /u01/app/11.2.0/grid/root.sh failed with the message ...see "/u01/app/11.2.0/grid/cfgtoollogs/crsconfig/rootcrs_node1.log" for details and inside this logfile it failed while creating ASM.

: Failed to stop ASM
: Initial cluster configuration failed. 

Executing as grid: /u01/app/11.2.0/grid/bin/asmca -silent -diskGroupName DATA -diskList /dev/
rdsk/emcpower6b,/dev/rdsk/emcpower6d,/dev/rdsk/emcpower6e -redundancy EXTERNAL -configureLocalASM

To fix this issue.. first run deconfigure and then run root.sh again.

Here are the steps to fix this.

1. On all the nodes except Last one
     Login as root and go to GRID_HOME(/u01/app/11.2.0/grid)/crs/install
     #cd /u01/app/11.2.0/grid/crs/install
     #./rootcrs.pl -deconfig -force
       Parsing the host name
       .........
       ........
       Successfully deconfigured Oracle clusterware stack on this node

2. On Last node
       #cd /u01/app/11.2.0/grid/crs/install
       #./rootcrs.pl -deconfig -force -lastnode
         Parsing the host name
         ....
         ...
         Successfully deconfigured Oracle clusterware stack on this node


In my case it was a permission issue with one of the disk. Verify any permission issues with the following command as grid user

 $ dd if=/dev/zero of=/dev/rdsk/emcpower6e count=1440
   1440 records in
   1440 records out

3. Now run root.sh again.
   #sh /u01/app/11.2.0/grid/root.sh
    Running Oracle 11g root.sh script...
  
 The following environment variables are set as:
    ORACLE_OWNER= grid
    ORACLE_HOME=  /u01/app/11.2.0/grid
    ... 
    ...
   ASM created and started successfully.
   DiskGroup DATA created successfully.
   ...
   CRS-2672: Attempting to start 'ora.DATA.dg' on 'node1'
   CRS-2676: Start of 'ora.DATA.dg' on 'node1' succeeded