Question : 15 tb oracle 19c database migration on aws ec2 provision and calculate IOPS with precheck vpc,ebs and consideration with example and test case
Migrating a 15 TB Oracle 19c database to AWS EC2 requires precise infrastructure sizing, strict pre-migration checks, and an optimized storage layout. The most effective method is using RMAN Backup/Restore over AWS Direct Connect, combined with Oracle Data Pump or AWS DMS for incremental syncs, achieving minimal downtime.
1. Pre-Check & Considerations
Database Pre-Checks:
- Endianness: Verify if the source is little-endian (e.g., Linux/Windows) to match the AWS EC2 target.
- Character Set: Ensure the destination database has the same character set (or is a valid superset).
- Invalid Objects: Run
@utlrp.sqlon the source to compile all objects.
Considerations:
- Architecture: Deploy a Multi-AZ architecture with Oracle Data Guard for High Availability (RPO = 0).
- ASM Configuration: Utilize Oracle ASM (Automatic Storage Management) to stripe across multiple EBS volumes.
2. Storage Attachment & Provisioning (EC2)
For a 15 TB database, use Amazon EBS io2 Block Express volumes or a striped gp3 volume group to achieve the required IOPS and throughput.
- Target Instance:
r5.24xlarge(96 vCPU, 768 GB RAM, up to 100,000 IOPS and 7,500 MB/s throughput) orr6i.24xlarge. - Storage Layout:
- Redo Logs: 3 separate
io2volumes of 100 GB each (mounted across different mount points), provisioned at 10,000 IOPS each. - Datafiles: 10 x
gp3volumes of 2 TB each (striped using ASM). - Temp Tablespace: 2 x
gp3volumes of 1 TB each. - FRA (Flash Recovery Area):
st1(Throughput Optimized HDD) of 5 TB for backups
- Redo Logs: 3 separate
3. IOPS Calculation
To calculate the required IOPS, analyze your peak workload using AWR (Automatic Workload Repository) reports from your on-premises system.
Total IOPS=Read OPS +Write IOPS
Formula & Example:
- Read IOPS: 12,000 IOPS
- Write IOPS: 8,000 IOPS
- Calculated Peak IOPS: 12,000 + 8,000 = 20,000 IOPS
- Total Throughput: (12,000 Read IOPS × 8KB) + (8,000 Write IOPS × 8KB) = 160 MB/s
Buffer/Growth Margin: Add a 20-30% buffer. So, target 26,000 IOPS and 200 MB/s throughput.
- If using
gp3, you provision this baseline statically. Eachgp3volume provides a baseline of 3,000 IOPS and 125 MB/s. Using an 8-volume ASM stripe will yield 3,000 × 8 = 24,000 IOPS and 125 × 8 = 1,000 MB/s, which easily satisfies the work
EBS Volume Selection:
- Redo Logs (Fastest): Use Amazon EBS io2 Block Express for extremely low latency, providing up to 256,000 IOPS and 4,000 MB/s throughput per volume.
- Data / Temp Files: Use Amazon EBS gp3 volumes. You can provision 15,000 IOPS (the maximum baseline for
gp3) and 1,000 MB/s throughput independently of your volume size, which is highly cost-effective. - Sizing: A 15 TB database requires a striped disk array (e.g., ASM with LUNs distributed over several
gp3volumes) to hit higher IOPS and aggregate throughput.
2. AWS Pre-Checks: VPC & EBS
- VPC & Networking: Ensure your Amazon VPC has private subnets routing through a NAT Gateway for patch downloads. Establish dedicated AWS Direct Connect or a highly reliable Site-to-Site VPN for the initial seed/migration.
- Security Groups: Whitelist the Oracle listener port (
1521) and SSH port (22) tightly between your on-premises data center and the AWS VPC. - EBS Limits: Ensure your selected EC2 Instance (e.g.,
r6i.4xlarge) features dedicated EBS bandwidth. If your database needs 30,000 IOPS, the EC2 instance must support at least this rate, or the volume performance will be throttled.
3. Considerations for 15 TB Oracle DB
- Initial Seeding: Transferring 15 TB of data over the network can take days. For large migrations, use AWS Snowball Edge for the initial backup, followed by incremental syncs.
- Database Parameters: Use Oracle Automatic Storage Management (ASM) for striping across multiple EBS volumes to balance I/O loads and prevent hot spots.
- Instance Sizing: Choose Memory-Optimized EC2 instances (e.g.,
Rfamily) as they have the CPU and RAM proportions ideally suited for Oracle Database System Global Area (SGA) caching.
4. Migration Strategy and Example Test Case
To migrate a 15 TB database with minimal downtime, utilize a hybrid RMAN + AWS DMS (Change Data Capture) methodology.
Example Test Case: Cutover Verification
- Objective: Validate data consistency and measure switchover downtime.
- Initial Load: Take a full Oracle RMAN backup and restore it to AWS EC2 via Snowball or network copying.
- Setup CDC: Use AWS Database Migration Service (AWS DMS) to continuously replicate transactions in real-time from the on-premises source to the AWS target.
- Validation Test Run: Execute the following query on both on-prem and AWS to ensure the sequence numbers match and no data loss occurred during replication:sql
SELECT name, value FROM v$sysstat WHERE name IN ('redo size', 'user commits');Switchover: Stop the application, allow DMS to catch up, finalize the cutover, and point the application to the new EC2 endpoint
or
Migrating a 20 TB Oracle 19c database to AWS EC2 requires strict sizing of compute and storage resources. You must provision sufficient EBS IOPS to avoid bottlenecking, set up a secure VPC, and use Oracle RMAN or AWS DMS for large dataset transfers. [1, 2, 3, 4, 5]
1. Provisioning & Storage Attachment
For a 20 TB database, you need to use Oracle ASM (Automatic Storage Management) or Linux LVM striping across multiple EBS volumes to achieve optimal throughput and capacity.
- EC2 Instance: Use memory-optimized instances like
r5.24xlargeorr6i.32xlargeto support high EBS bandwidth (up to 19 Gbps). - Storage Attachment (EBS): Use io2 Block Express for consistent, sub-millisecond latency.
- Data Files: 4 × 2 TB
io2volumes (provisioned at 16,000 IOPS each). - Redo Logs: 3 × 100 GB
io1orio2volumes (placed in their own ASM disk group for maximum throughput). - FRA/Backups: 2 × 5 TB
gp3volumes (good baseline performance).
- Data Files: 4 × 2 TB
2. Calculating IOPS
For an Oracle OLTP database, IOPS is calculated based on the block size and transaction throughput.
- Formula: \(IOPS = \frac{\text{Throughput (MB/s)}}{\text{Block Size (MB)}}\)
- Example Calculation: If your system processes peak workloads of 5,000 MB/sec, with an Oracle block size of 8 KB, the raw IOPS requirement is calculated as:
5000 ÷ 0.008 = 625,000 IOPS. - AWS Strategy: Divide this total across multiple ASM disks. If one
io2Block Express volume caps out at 256,000 IOPS or 4,000 MB/s, you must stripe 4 volumes together in ASM to hit 1,000,000 IOPS and bypass volume limits.
3. Precheck: VPC, EBS, and Considerations
- VPC Setup: Ensure you use private subnets. Attach a NAT Gateway only if your migration servers require temporary external internet access to reach the AWS Management Console.
- Network Baseline: Ensure you have an AWS Direct Connect (1 Gbps to 10 Gbps) or VPN setup if transferring data from an on-premises data center. A 20 TB transfer takes ~18 hours over 10 Gbps. Consider AWS Snowball Edge if bandwidth is below 1 Gbps.
- Architecture Rules: Ensure EBS-optimized is enabled on the EC2 instances. Do not place OS and Database data on the same volume.
4. Migration Strategy Example & Test Cases
Migration Strategy: Use Oracle Transportable Tablespaces (TTS) or RMAN Incremental Backups (for minimal downtime).
- Initial Backup: Take a full RMAN backup of the source database.
- Transfer: Ship backup pieces to an Amazon S3 bucket, then copy them to the target EC2 EBS volumes.
- Restore: Restore the database onto the AWS EC2 instance.
- Catch-up: Run RMAN incremental backups to sync the EC2 database with the source on-premise system.
Post-Migration Test Cases:
- Test Case 1: Data Integrity Validation: Run
RMAN VALIDATE DATABASEand compare SCN on the source and destination. - Test Case 2: Performance Benchmark: Execute the Oracle
AWR(Automatic Workload Repository) report to identify wait events (e.g.,db file sequential read). - Test Case 3: Application Failover: Reroute read-only connections and test listener connectivity over the new VPC Endpoints.
5. Challenges and Post-Migration Issues
- Endinanness Issue: If migrating from on-premises AIX/Solaris to Linux (x86-64), the platforms have different Endianness. Solution: Use Oracle RMAN CONVERT or AWS DMS for schema and data translation.
- Performance Degradation due to Throttling: EBS volumes may get throttled (burst credits run out) if IOPS is not properly provisioned.
- Network Latency: Application servers migrating alongside the database might experience latency if not placed in the same Availability Zone.
6. Troubleshooting Steps
- Issue 1: Slow Data Transfer/RMAN Restores:
- Troubleshooting: Run
sar -n DEV 1 10to check network bandwidth saturation. Increase RMAN channels using the command:CONFIGURE DEVICE TYPE DISK PARALLELISM 8;.
- Troubleshooting: Run
- Issue 2: ASM Disk Group Won't Mount:
- Troubleshooting: Check permission on block devices (
ls -l /dev/sd*). Verifyasm_diskstringpoints to the correct multipath devices in yourinit.orafile.
- Troubleshooting: Check permission on block devices (
- Issue 3: High
db file scattered readwaits after migration: - Troubleshooting: Gather fresh system and table statistics using
EXEC DBMS_STATS.GATHER_DICTIONARY_STATS;andEXEC DBMS_STATS.GATHER_FIXED_OBJECTS_STATS;.
or
Migrating a 20 TB Oracle 19c database to AWS EC2 requires precise infrastructure sizing (using Provisioned IOPS io2 or gp3 volumes) and an optimal hybrid migration strategy (e.g., RMAN cross-platform backup or AWS DMS). Careful storage striping with Oracle ASM is essential to prevent latency or IO bottlenecks. [1]
1. EC2 Provisioning & Storage Attachment
For a 20 TB database, you need to provision enterprise-grade instances with high memory and network bandwidth.
- Instance Type:
r5.24xlargeorr6i.24xlarge(96 vCPU, 768 GiB RAM) for compute. - Storage Configuration: Do not use a single massive EBS volume. Use Oracle Automatic Storage Management (ASM) or Linux LVM to stripe across multiple EBS volumes to aggregate IOPS and Throughput limits.
- EBS Type: Provision io2 Block Express for latency-sensitive core systems or gp3 for high-balance cost-performance.
2. IOPS & Throughput Calculation
Oracle databases are sensitive to disk latency. You must calculate IOPS to meet your peak AWR workload.
- Example Calculation:
If your peak workload hits 15,000 Read IOPS and 5,000 Write IOPS, you need 20,000 Provisioned IOPS.
If using gp3 (which allows up to 16,000 IOPS per volume), you can attach four 4TBgp3volumes, each capped at 5,000 IOPS, and use Oracle ASM to stripe them. This easily achieves your 20,000 IOPS while maintaining a predictable budget.
3. Prechecks & Considerations
- VPC & Subnet: Ensure a dedicated VPC with private subnets for DB servers, secure Bastion hosts, and VPN or Direct Connect (min 1 Gbps to 10 Gbps) to transfer base backups.
- Source DB Check: Run the
DBMS_TTSorRMANprechecks. For OS/Platform changes (e.g., AIX to Linux), check Endian format compatibility. - Character Set: The target and source database must have identical character sets or the target must be a superset.
4. Migration Example & Test Cases
Strategy: Transportable Tablespace (TTS) + RMAN Incremental Backups (Best for near-zero downtime of 20TB).
Migration Steps:
- Provision EC2 and configure Oracle ASM.
- Take an RMAN incremental backup of the source database.
- Transfer backup pieces via AWS Direct Connect to Amazon S3 or mount directly on EC2.
- Restore the base backup on target EC2.
- Apply consecutive RMAN incremental backups to sync the databases.
- Perform final cutover by stopping source DB activity and applying the final incremental backup.
Test Cases:
- Validation Test: Verify
DB_NAME,DBID, andGLOBAL_NAMEmatch perfectly. - Performance Test: Execute AWR reports on the EC2 target against simulated read/write volumes.
- Application Cutover Test: Execute read-only tests with application teams pointing to the newly synced EC2 target.
5. Challenges
- Network Bottleneck: Transferring 20 TB over the wire takes significant time. A 1 Gbps connection fully utilized takes ~47 hours; a 10 Gbps line takes ~4.7 hours.
- Endianness: If migrating from Big-Endian (Solaris/AIX) to Little-Endian (Linux), datafiles must be converted using RMAN
CONVERT. - SGA/PGA Alignment: Sizing EC2 memory correctly to avoid memory swapping or latch contention.
6. Post-Migration Issues & Troubleshooting
- Issue: High
db file scattered readordb file sequential readwaits.- Troubleshooting: Gather fresh system and table statistics using
DBMS_STATS.GATHER_DATABASE_STATS. Check if EBS Throughput limit (e.g., 1000 MB/sec onr6i.24xlarge) is being throttled by runningiostat -x 5at the OS level.
- Troubleshooting: Gather fresh system and table statistics using
- Issue: Sequence Cache Contention (Wait Event:
sequence cache latch).- Troubleshooting: Migrations sometimes trigger sequence cache exhaustion. Check source sequence caches, increase the
CACHEvalues on high-usage sequences (e.g.,CACHE 1000).
- Troubleshooting: Migrations sometimes trigger sequence cache exhaustion. Check source sequence caches, increase the
- Issue: Invalid PL/SQL or Views after Cross-Platform Conversion.
- Troubleshooting: Run
@?/rdbms/admin/utlrp.sqlasSYSDBAto recompile all invalid objects.
- Troubleshooting: Run
No comments:
Post a Comment