Oracle ASM
Oracle ASM
2009
Marie-Louise Lindstrm Service Delivery Engineer Oracle Support ACS
Agenda
Concepts, Architecture New Features in Database Release 11g Tips and Techniques Examples, Demo
1.
Automatic Storage Management (ASM) instance Oracle instance that manages the diskgroup metadata ASM Disk Groups Logical grouping of disks Determines file mirroring options ASM Disks LUNs presented to ASM ASM Files Files that are stored in ASM disk groups are called ASM files, this includes database files
2.
3.
4.
ASM provides filesystem and volume manager capabilities built into the Oracle Database kernel Provides a simple storage management interface across all server and storage platforms Discover and associate (adding) disks under ASM management A disk can be partition of a physical spindle, the entire spindle, or a LUN in a RAID group set Logical volumes can be presented to ASM as disks, but not a good practice, since an extra layer is added to IO code path, and thus some level of overhead
I/O is spread evenly across all available disk drives to prevent hot spots and maximize performance ASM eliminates the need for over provisioning and maximizes storage resource utilization facilitating database consolidation Performs automatic online redistribution after the incremental addition or removal of storage capacity Maintains redundant copies of data to provide high availability, or leverage 3rd party RAID functionality Capable of leveraging 3rd party multi-pathing technologies
An Oracle Database can contain ASM and non-ASM files. Any new files can be created as ASM files whilst existing files can also be migrated to ASM RMAN commands enable non-ASM managed files to be relocated to an ASM disk group Enterprise Manager can be used to manage ASM disk and file management activities ASM reduces Oracle Database cost and complexity without compromising performance or availability
APPL1
APPL2
APPL3 DB2
APPL1
APPL2
APPL3
DB2
OPERATING SYSTEM
OPERATING SYSTEM
Automatic online rebalancing whenever storage configuration changes Only move data proportional to storage added No need for manual I/O tuning Online migration to new storage
Unstriped Disks
Striped Disks
ASM Instance
Oracle instance to manage the metadata for disk groups All metadata modifications are done by the ASM instance Database instances connect to ASM instance to create, delete, resize, open, or close files Database instances read/write directly to the disks One ASM instance is required per node The ASM instance comprises shared memory structure (SGA) and background processes The ASM instance contains no physical files other than the password file and the parameter file Instance alert log file and trace files in the admin directories (release 10g), or diag directory (release 11g)
Works with the RAC architecture to support clustered storage environments In RAC environments, there will be one ASM instance per clustered node, and the ASM instances communicate with each other on a peer-to-peer basis using the interconnect Generally named +ASM, in a RAC environment +ASM1, +ASM2, etc
ASM Diskgroups
SQL> create diskgroup <name> disk <path>,<path>; When the diskgroup has been created, metadata information, which includes creation date, diskgroup name, and redundancy type, is stored in the SGA and on each disk (in disk header) within the diskgroup A disk and a database file can only be part of one diskgroup A database can store its files in multiple diskgroups Recommended to have two diskgroups: Database area Flash recovery area
ASM Disks
Unit of persistent storage for a disk group Accessed through normal OS interfaces Oracle must have read/write access Must be accessible to all nodes of a cluster May have different names on different nodes Cannot be a local filesystem file Must not include partition table Unit that fails if using redundancy Disks in a disk group are the same percent full Best practice is to have disks of same capacity (size, throughput) belonging to the same diskgroup ASM must scale to support at least 10,000 disks in a disk group
Allocation Unit
ASM Disk
1 MB
ASM disks are divided into allocation units Unit of allocating storage to a file 1 MB in size ; release 10g 1, 2, 4, 8, 16, 32, 64 MB ; release 11g Small enough to not be hot Large enough for efficient sequential access Many allocation units in a single file
External redundancy No mirroring, no failure groups Normal redundancy ; default Files are two-way mirrored, requiring at least two failure groups High redundancy Using three-way mirroring, requiring at least three failure groups Redundancy level set at creation time A failure group, a subset of a diskgroup, is a collection of disks that can become unavailable due to a failure of one of its associated components; e.g., controllers or entire arrays Disks in two separate failure groups (for the same diskgroup) must not share a common failure component The primary extent and its mirror copy are always stored in separate failure groups
Redundancy Levels
External redundancy data exists only once Normal redundancy two-way mirroring one additional extent copy at least two failure groups High redundancy three-way mirroring two additional copies at least three failure groups
ASM Files
Datafiles, control files, spfiles, and redo log files, incl archived log files, can be stored in ASM diskgroups Rman backups ASM file names are in the format of
+diskgroup_name/database_name/file_type/tag_name.file_n umber.incarnation ASM Alias name format: instead of a file and incarnation number, a user-friendly string name is used For best practices, implement the Oracle Managed File (OMF) feature Log files, trace files and executables are stored outside ASM diskgroups
ASM allocates space in units called allocation units (AU) A database file is broken up into file extents (coarse and fine distribution) Changing the storage configuration, for example adding a disk to a diskgroup, will trigger a rebalance operation The objective of the rebalance operation is to provide an even distribution of file extents and space usage across all disks in the diskgroup Rebalancing involves physical movement of file extents If removing or adding several disks, it is best to add or remove drives all at once, this will reduce the number rebalance operations that are needed for storage changes
ASM Views
ASMCMD Utility
Command line interface providing access to the files and directories within the ASM diskgroups
$ export ORACLE_SID=+ASM $ asmcmd ASMCMD> lsdg State Type Rebal Sector Block AU Usable_file_MB Offline_disks Name MOUNTED NORMAL N 512 4096 1048576 111 0 DATA1/ MOUNTED EXTERN N 512 4096 1048576 885 0 FRA1/ ASMCMD> cd data1 ASMCMD> ls X1020/ X1110/ ASMCMD> cd X1110/ ASMCMD> ls CONTROLFILE/ DATAFILE/ ONLINELOG/ PARAMETERFILE/ TEMPFILE/ spfileX1110.ora ASMCMD> ls DATAFILE/ SYSAUX.260.687886011 SYSTEM.261.687886193 UNDOTBS1.262.685476185 USERS.264.681997355 ASMCMD>
Req_mir_free_MB 2047 0
Migrating to ASM
ASM migration using Data Guard physical standby Use this method if your requirement is to minimize downtime during the migration. It is possible to reduce total downtime to just seconds by using the best practices described in the white paper ASM migration using Rman A simpler approach, but one that can result in downtime measured in minutes to hours, depending on the method used for migration ASM migration using DBMS_FILE_TRANSFER
1. 2. 3. 4.
Across ASM diskgroups Between the OS file system and ASM diskgroups Using Rman Using the DBMS_FILE_TRANSFER package Through ftp to the XML DB Virtual Folder Using the ASMCMD command cp in release 11g
ASM Fast Mirror Resync ASM Preferred Mirror Read ASM Rolling Upgrades and Patches ASM Scalability and Performance Enhancements Multiple Allocation Unit Size SYSASM Privilege ASMCMD Extensions To use the new diskgroup features, the disk group compatibility attribute must be set to 11.1
Define DISK_REPAIR_TIME
ALTER DISKGROUP dgroupA SET ATTRIBUTE 'DISK_REPAIR_TIME'='3H'; ALTER DISKGROUP dgroupA OFFLINE DISKS IN FAILGROUP contrl2 DROP AFTER 5H;
Extended Cluster
P
ASM_PREFERRED_READ_ FAILURE_GROUPS=DATA .SITEA
S
ASM_PREFERRED_READ_ FAILURE_GROUPS=DATA .SITEB
Maximizes database availability in a cluster How it works Place cluster in Rolling Migration mode Bring down ASM on a cluster node Upgrade or patch software Restart ASM Stop Rolling Migration mode after upgrading all nodes
SYSASM Privilege
SYSASM role to manage ASM instances separates duty of DBAs and storage administrators
SQL> CONNECT / AS SYSASM SQL> CREATE USER ossysasmusername IDENTIFIED by passwd; SQL> GRANT SYSASM TO ossysasmusername; SQL> CONNECT ossysasmusername / passwd AS SYSASM; SQL> DROP USER ossysasmusername;
SYSDBA will be deprecated for ASM management: Oracle Database 11g Release 1 behaves as in 10g In future releases SYSDBA privileges restricted in ASM instances
Allocation Unit (AU) selected at disk group creation time and may be 1,2,4,8,16,32,64 MB Set Oracle MAX_IO_SIZE = AU size Default MAX_IO_SIZE is 1MB MAX_IO_SIZE determines default AU size Striping coarse stripe size = 1 AU fine stripe size = 128KB Higher performance for large sequential I/O (DW) Better leverage of Hardware RAID read-ahead
ASMCMD Extensions
User created directories Templates Disk group compatibility Disk group name Disk names and failure groups
md_backup md_restore
repair
$ asmcmd help
cp
Copy files
lsdsk
Does the FILESYSTEMIO_OPTIONS parameter need to be configured in the Database instance when using ASM? No, the Database instance does I/O directly to the raw disk devices, whereas the FILESYSTEMIO_OPTIONS parameter is only for file systems
References
Oracle Database Storage Administrator's Guide 11g Release 1 (11.1) B31107-05
http://download.oracle.com/docs/cd/B28359_01/server.111/b31 107.pdf
Best Practices for Minimal Downtime Migration to ASM Oracle 10g Release 2
http://www.oracle.com/technology/deploy/availability/pdf/ma a_wp_10gr2_asmmigrationwithdg.pdf
References (cont.)
ASM and Multipathing Generic Best Practices and Information Matrix March 2008
http://www.oracle.com/technology/products/database/asm/pdf/ asm%20and%20multipathing%20best%20practices%20info%20matrix %203-5-08.pdf
MyOracleSupport Note:265633.1 - ASM Technical Best Practices MyOracleSupport Note:751300.1 - How to perform Rolling Upgrade/Downgrade in 11g ASM