Tuesday, 4 July 2023

Oracle Grid Installation/ASM Storage for Single Instance Database

 


  • Download Grid Infrastructure software 19c with latest patch set (19.19)
  • 100 GB space required for 19c Grid Binaries
  • Grid Instance for ASM -> need to install Grid Infrastructure for Standalone Server 
  • Disk Space required for ASM Disk group as below.


Set ASM disks

# oracleasm configure -i

Create ASM disks

oracleasm createdisk OCR  /dev/ASM/LV_OCR

oracleasm createdisk DATA_DG1 /dev/ASM/LV_EASYDB_DATA

oracleasm createdisk FRA_DG1 /dev/ASM/LV_EASYDB_FRA

oracleasm createdisk REDO_DG1 /dev/ASM/LV_EASYDB_REDO1

oracleasm createdisk REDO_DG2 /dev/ASM/LV_EASYDB_REDO2

oracleasm createdisk BKP_DG1 /dev/ASM/LV_EASYDB_BKP


100 G =  OCR Disk   High Redundancy

3TB  = Archive log    External Redundancy

50 TB = Datafiles     External Redundancy   

100 G = redolog1    External Redundancy 

100 G = redolog2    External Redundancy 

1TB  = For BKP Dump External Redundancy 


Oracle ASM Storage Limits

-------------------------

Oracle ASM provides near unlimited capacity for future growth, but does have some storage limits. 

For example, Oracle ASM has the following limits on the number of disk groups, disks, and files:


• 63 disk groups in a storage system

• 10,000 Oracle ASM disks in a storage system

• 1 million files for each disk group


Without any Oracle Exadata Storage, Oracle ASM has these storage limits:


• 2 terabytes (TB) maximum storage for each Oracle ASM disk

• 20 petabytes (PB) maximum for the storage system


Value of COMPATIBLE.ASM must always be greater than or equal to the value of COMPATIBLE.RDBMS.  

It means Grid Home Version equal to RDBMS version or greater than RDBMS.


Saturday, 1 July 2023

Oracle ASM Useful Commands


Basic ASM Commands


ASMCMD> spget

+VOTING/prod-cluster/asmparameterfile/registry.253.756259253

ASMCMD>


$ env | grep ORA

ORACLE_SID=+ASM1

ORACLE_HOME=/u01/app/12.1.0/grid

Get into the ASMCMD utility by running the

SQL> select instance_name, db_name, status from v$asm_client;

INSTANCE_N DB_NAME    STATUS

---------- ---------- ----------

knewdb_2   knewdb     CONNECTED

+ASM2      +ASM       CONNECTED

+APX2      +APX       CONNECTED

-MGMTDB    _mgmtdb    CONNECTED


The srvctl commands include updates to manage a password file in a disk group, such as the following for updating and displaying the location of the password file:

$ srvctl modify asm -pwfile location

$ srvctl modify database -db dbname -pwfile location $ srvctl config asm

ASM home: /u01/app/12.1.0/grid Password file: +DATA/orapwASM ASM listener: LISTENER

ASM instance count: 3

Cluster ASM listener: ASMNET1LSNR_ASM


Starting and Stopping the Node Listener


•Using the lsnrctl utility:

$ lsnrctl start listener


LSNRCTL for Linux: Version 12.1.0.1.0-Production on 31-JUL-2013 16:45:35

 Copyright (c) 1991, 2013, Oracle. All rights reserved.

Starting /u01/app/12.1.0/grid/bin/tnslsnr: please wait...... 

Intermediate output removed ...

The command completed successfully $


All other commands, such as CREATE DISKGROUP, ADD/DROP/RESIZE DISK, and so on, require the SYSDBA privilege and are not allowed with the SYSOPER privilege.

To check space on ASM


column path FORMAT a40

select group_number,disk_number,path,mode_status,name from v$asm_disk order by group_number,disk_number,path;


select group_number,name,state,type,total_mb,free_mb,usable_file_mb  from v$asm_diskgroup


select name, total_mb, free_mb, state from v$asm_diskgroup;


To check space


set lines 200 pages 200

SELECT g.group_number "Group"

, g.name "Group Name"

, g.state "State"

, g.type "Type"

, g.total_mb/1024 "Total GB"

, g.free_mb/1024 "Free GB"

, 100*(max((d.total_mb-d.free_mb)/d.total_mb)-min((d.total_mb-d.free_mb)/d.total_mb))/max((d.total_mb-d.free_mb)/d.total_mb) "Imbalance"

, 100*(max(d.total_mb)-min(d.total_mb))/max(d.total_mb) "Variance"

, 100*(min(d.free_mb/d.total_mb)) "MinFree"

, 100*(max(d.free_mb/d.total_mb)) "MaxFree"

, count(*) "DiskCnt"

FROM v$asm_disk d, v$asm_diskgroup g

WHERE d.group_number = g.group_number and

d.group_number <> 0 and

d.state = 'NORMAL' and

d.mount_status = 'CACHED'

GROUP BY g.group_number, g.name, g.state, g.type, g.total_mb, g.free_mb

ORDER BY 1;


or


set lines 200 pages 200

SELECT g.group_number "Group"

, g.name "Group Name"

, g.state "State"

, g.type "Type"

, g.total_mb/1024 "Total GB"

, g.free_mb/1024 "Free GB"

, 100*(max((d.total_mb-d.free_mb)/d.total_mb)-min((d.total_mb-d.free_mb)/d.total_mb))/max((d.total_mb-d.free_mb)/d.total_mb) "Imbalance"

, 100*(max(d.total_mb)-min(d.total_mb))/max(d.total_mb) "Variance"

, 100*(min(d.free_mb/d.total_mb)) "MinFree"

, 100*(max(d.free_mb/d.total_mb)) "MaxFree"

, count(*) "DiskCnt"

FROM v$asm_disk d, v$asm_diskgroup g

WHERE d.group_number = g.group_number and

d.group_number <> 0 and

d.state = 'NORMAL' and

d.mount_status = 'CACHED'

GROUP BY g.group_number, g.name, g.state, g.type, g.total_mb, g.free_mb

ORDER BY 1;




or


set linesize 200

column path format a40

 SQL> SELECT GROUP_NUMBER,DISK_NUMBER,REDUNDANCY,FAILGROUP ,HEADER_STATUS,state,TOTAL_MB,PATH FROM V$ASM_DISK;

GROUP_NUMBER DISK_NUMBER REDUNDA FAILGROUP                      HEADER_STATU STATE      TOTAL_MB PATH

------------ ----------- ------- ------------------------------ ------------ -------- ---------- ----------------------------------------

           0           4 UNKNOWN                                CANDIDATE    NORMAL            0 /dev/rhdisk5

           0           5 UNKNOWN                                CANDIDATE    NORMAL            0 /dev/rhdisk6

           1           0 UNKNOWN DATA1_0000          MEMBER       NORMAL       512000 /dev/rhdisk1

           2           0 UNKNOWN FRA1_0000              MEMBER       NORMAL       102400 /dev/rhdisk2

           3           0 UNKNOWN VOTING_0000                    MEMBER       NORMAL          500 /dev/rhdisk3

           3           1 UNKNOWN VOTING_0001                    MEMBER       NORMAL          500 /dev/rhdisk4


SELECT HEADER_STATUS,OS_MB,TOTAL_MB,PATH FROM V$ASM_DISK WHERE HEADER_STATUS LIKE '%CANDIDA%' ORDER BY PATH;

HEADER_STATU   TOTAL_MB PATH

------------ ---------- --------------------

CANDIDATE             0 /dev/rhdisk5

CANDIDATE             0 /dev/rhdisk6


 12c ASM Commands

--------------

To Check spfile location and to change the ASM_POWER_LIMIT for all nodes.


SQL> show parameter SPFILE


NAME TYPE VALUE

-------------------- ----------- ---------------

spfile string +DATA/cluster01/ASMPARAMETERFI LE/registry.253.821741859


SQL> ALTER SYSTEM set ASM_POWER_LIMIT=3 SCOPE=BOTH SID='*';


System altered.


SQL> show parameter ASM_POWER_LIMIT



NAME TYPE VALUE

------------------------------------ ----------- ---------------

asm_power_limit integer 3

SQL>



to change the ASM_POWER_LIMIT for particular nodes.


SQL> ALTER SYSTEM set ASM_POWER_LIMIT=1 SCOPE=BOTH SID='+ASM2';


System altered.


SQL> ALTER SYSTEM set ASM_POWER_LIMIT=1 SCOPE=BOTH SID='+ASM3';


System altered.


SQL> ALTER SYSTEM set ASM_POWER_LIMIT=5 SCOPE=BOTH SID='+ASM1';


System altered.


SQL> column NAME format A16 SQL> column VALUE format A16

SQL> select inst_id, name, value from GV$PARAMETER 2 where name like 'asm_power_limit';


INST_ID NAME VALUE

---------- ---------------- ----------------

1 asm_power_limit 5

2 asm_power_limit 1 

3 asm_power_limit 1


Use srvctl to view which database instances are running on each host. 


[grid@host01 ~]$ srvctl status database -db orcl


Instance orcl_1 is running on node host02

Instance orcl_2 is running on node host03 

Instance orcl_3 is running on node host01 

[grid@host01 ~]$


Use the srvctl stop instance command to stop the ASM instance on host01


[grid@host01 ~]$ srvctl stop asm -n host01


PRCR-1014 : Failed to stop resource ora.asm 

PRCR-1065 : Failed to stop resource ora.asm

CRS-2529: Unable to act on 'ora.asm' because that would require 

stopping or relocating 'ora.DATA.dg', but the force option was not specified


Re-run the srvctlstopasm command by using the –f option. 

Use the crsctlstat res command to check the state of the ASM and the disk group resources on host01. 

Note that the database instance is up on host01.


[grid@host01 ~]$ srvctl stop asm -n host01 -f


[grid@host01 ~]$ crsctl stat res -t


Restart the ASM instance on host01.


[grid@host01 ~]$ srvctl start asm -n host01 

[grid@host01 ~]$


To check Client connection


[grid@host02 ~]$ . oraenv ORACLE_SID = [grid] ? +ASM3

The Oracle base has been set to /u01/app/grid



[grid@host02 ~]$ asmcmd lsct data



[grid@host02 ~]$ asmcmd lsct fra



[grid@host02 ~]$ sqlplus / as sysasm


SQL*Plus: Release 12.1.0.2.0 Production on Tue Mar 17 11:23:58 2015


Copyright (c) 1982, 2014, Oracle. All rights reserved.


Connected to:

Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 -64bit Production

With the Real Application Clusters and Automatic Storage Management options



SQL> alter system relocate client 'orcl_3:orcl';


System altered.


SQL> exit

Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production

With the Real Application Clusters and Automatic Storage Management options


[grid@host02 ~]$ exit 



To check Client connection with ASM


col client_instance_name format a21

col host_name format a9 col status format a9


select distinct i.host_name, i.instance_name asm_instance_name, c.instance_name client_instance_name, c.db_name, c.status

from gv$instance i, gv$asm_client c where i.inst_id=c.inst_id;



Confirm that no ASM instance is running on host02.


[grid@host01 less_03]$ crsctl status resource ora.asm -t


 restart the Flex ASM instance that you stopped earlier in this practice.


[grid@host01 less_03]$ srvctl start asm -node host02 

[grid@host01 less_03]$


14. Confirm that Flex ASM is again running on host02. 

[grid@host01 less_03]$ srvctl status asm 

ASM is running on host01,host02,host03 

[grid@host01 less_03]$


To check diskgroup 


asmcmd lsdg


Use the asmcmdlsdsk command to view the disks that belong to the DATA disk group. 

[grid@host01 ~]$ asmcmd lsdsk -k -G DATA



Add another disk (/dev/asmdisk2p2) to the DATA disk group and remove a disk (DATA_0004: /dev/asmdisk1p2), 

but the rebalance operation must wait until a quiet time, and then proceed as quickly as possible.

 As the grid user, use SQL*Plus to connect to the ASM instance on host01 and perform the following operations:

[grid@host01 ~]$ sqlplus / as sysasm

SQL*Plus: Release 12.1.0.1.0 Production on Thu Aug 29 12:19:47 2013

...

SQL> ALTER DISKGROUP DATA ADD DISK '/dev/asmdisk2p2' REBALANCE POWER 0;

Diskgroup altered.


SQL> ALTER DISKGROUP DATA DROP DISK DATA_0004 REBALANCE POWER 0;


Diskgroup altered.


SQL>


Next, start the rebalance operations on the DATA disk group. Specify a power of 6. 

Query the V$ASM_OPERATION view to monitor the rebalance.


SQL> alter diskgroup DATA rebalance power 6;


Diskgroup altered.


SQL> SELECT group_number, operation, state, power, est_minutes FROM v$asm_operation;


Change the rebalance power to 1024. Again, query the V$ASM_OPERATION view to monitor the rebalance.


SQL> alter diskgroup DATA rebalance power 1024;


Diskgroup altered.


SQL> SELECT group_number, operation, state, power, est_minutes FROM v$asm_operation;


8.Go to your grid terminal window and examine the disk I/O statistics by using the asmcmd lsdsk --statistics command. 

Not surprisingly, the disks in the DATA disk group show the most activity.

[grid@host01 ~]$ asmcmd lsdsk --statistics


Examine the disk statistics bytes and time for the DATA disk group with the asmcmd iostat–t–G DATA command.


[grid@host01 ~]$ asmcmd iostat -t -G DATA


you dropped /dev/asmdisk1p1 (DATA_0000, Step 3) and /dev/asmdisk1p2 (DATA_0004, Step 5). You also added /dev/asmdisk2p2. 

Execute asmcmd lsdsk -G data –k to confirm the dropped disks and identify the disk name assigned to /dev/asmdisk2p2.

[grid@host01 ~]$ asmcmd lsdsk -G data -k


Unmount the DATA2 disk group on all nodes.


[grid@host01 ~]$ srvctl stop diskgroup -diskgroup DATA2

[grid@host01 ~]$


Rename the DATA2 disk group to DG2 by using the renamedg command in two phases. 

Start the first phase as follows. Be sure to specify a file name for the config parameter.


[grid@host01 ~]$ renamedg phase=one dgname=DATA2 newdgname=DG2 confirm=true config=/tmp/renamedg verbose=true asm_diskstring='/dev/asmdisk*'



Use renamedg to start phase two by using the same configuration file. 

You can leave out the asm_diskstring parameter.


[grid@host01 ~]$ renamedg phase=two dgname=DATA2 newdgname=DG2 confirm=true config=/tmp/renamedg verbose=true


Check the resources with the crsctl command.


[grid@host01 ~]$ srvctl start diskgroup -diskgroup DG2


[grid@host01 ~]$ crsctl stat res -t|more


Use srvctl to remove the old DATA2 disk group resource.


[grid@host01 ~]$ srvctl remove diskgroup -diskgroup DATA2


mount disk group

[grid@host01 ~]$ asmcmd mount DG2 



Stop the TEST resource, and then drop the TEST disk group. Make sure that the disk group resource has been removed.


[grid@host01 ~]$ srvctl stop diskgroup -diskgroup TEST


[grid@host01 ~]$ asmcmd dropdg -r -f TEST


[grid@host01 ~]$ crsctl stat res ora.TEST.dg -t CRS-2613: Could not find resource 'ora.TEST.dg'.


[grid@host01 ~]$



grid@host01 ~]$ asmcmd lsdg test2

 

State Type Rebal Sector Block Free_MB Req_mir_free_MB Usable_file_MB Voting_files Name

 

AU Total_MB Offline_disks

 


 

MOUNTED NORMAL N 3741 650 N TEST2/

 

512 4096 1048576 3900 1545               0

 

[grid@host01 ~]$

The TOTAL_MB column shows the sum of the total size of the member disks (3900 MB). 

Free_MB shows how much disk space is unused (3741 MB). Because of disk group redundancy,

this is not the usable space that is available.

That number is shown under the Usable_file_MB column (1545 MB). 

The value of Req_mir_free_MB (650 MB) represents the amount of free space that is required for ASM to restore redundancy in the event of a failure.


put TEST6 disk group is in offline mode


asmcmd offline -G TEST6 -D TEST6_0000 

It appears that the COMPATIBLE.RDBMS value for the TEST6 disk group is lower than 11.1.0.0.0!


Determine the current COMPATIBLE.RDBMS value for the TEST6 disk group. Change it to 12.1.0.0.0. Confirm the change.


[grid@host01 ~]$ asmcmd lsattr -G TEST6 -l compatible.rdbms

 

Name Value

-----------------------------

compatible.rdbms  10.1.0.0.0

 

[grid@host01 ~]$ asmcmd setattr -G TEST6 compatible.rdbms 12.1.0.0.0


[grid@host01 ~]$ asmcmd lsattr -G TEST6 -l compatible.rdbms




[grid@host01 ~]$ asmcmd lsattr -G TEST6 -l disk_repair_time 


Name Value

-----------------------------

disk_repair_time 3.6h


If the TEST6_0000 disk is not brought back online in 12960 seconds (3.6 hours), 

it will be dropped. 3.6 hours is the default unless a different time is specified when the disk is taken offline.


asmcmd lsdsk -G TEST6 -k



[grid@host01 ~]$ asmcmd lsattr -G TEST6 -l failgroup_repair_time 


Name Value

failgroup_repair_time 24.0h


[grid@host01 ~]$ asmcmd setattr -G TEST6 failgroup_repair_time 2m



[grid@host01 ~]$ asmcmd offline -G TEST6 -F FGRP1 

Diskgroup altered.


[grid@host01 ~]$ vi /u01/app/grid/diag/asm/+asm/+ASM1/trace/alert_+ASM1.log



Check the current value of compatible.rdbms. 

It must be changed to 11.2 or later to complete the operation for the next step. List the files in the XYZ tablespace.


ASMCMD> lsattr -G DATA -l compatible.rdbms

 

Name     Value


compatible.rdbms 10.1.0.0.0

 



ASMCMD> setattr -G DATA compatible.rdbms 12.1.0.0.0



ASMCMD> lsattr -G DATA -l compatible.rdbms

 

Name Value


compatible.rdbms 12.1.0.0.0

 



ASMCMD> ls -l +DATA/orcl/datafile/XYZ*


Type Redund Striped Time Sys Name

 

DATAFILE MIRROR COARSE XYZ.293.876489337

DATAFILE MIRROR COARSE XYZ.294.876489413

 

Name Value

------------------------------

compatible.rdbms 12.1.0.0.0


[grid@host01 ~]$