Postgresql Migration - part1

AWS DMS options

This course focuses on using the AWS DMS console to perform the migration. You can begin a database migration in one of two ways:

1. You can choose the AWS DMS console, and perform each step there.

2. You can use the AWS Command Line Interface (AWS CLI). For more information about using the AWS CLI with AWS DMS, see AWS CLI for AWS DMS.

The source database can be any of the following:

• Located on your own premises outside of AWS
• Running on an Amazon Elastic Compute Cloud (Amazon EC2) instance
• An Amazon Relational Database Service (Amazon RDS) database

To start using AWS DMS, you need a replication instance that will be used to run your migration tasks.

For a database migration using the AWS DMS console, you must do the following:

• Complete the tasks outlined in Setting up for AWS Database Migration Service.

• Allocate a replication instance that performs all the processes for the migration.

• Specify a source and target database endpoint.

• Create a task, or set of tasks, to define which tables and replication processes you want to use.

AWS DMS then creates your replication instance and performs the tasks on the data being migrated.

Note : An AWS DMS replication instance performs the actual data migration between source and target. The replication instance also caches the transaction logs during the migration. The greater the central processing unit (CPU) and memory capacity of a replication instance, the less overall time is required for the migration.

Creating a Replication Instance

Your first task in migrating a database is to create a replication instance. 

An AWS DMS replication instance is a managed Amazon EC2 instance that hosts one or more replication tasks. Amazon EC2 has sufficient storage and processing power to perform the tasks you assign and migrate data from your source database to the target database.

The required size of this instance varies, depending on the amount of data you need to migrate and the tasks that you need the instance to perform.

The following figure shows an example of a replication instance running several associated replication tasks.

A single replication instance can host one or more replication tasks, depending on the characteristics of your migration and the capacity of the replication server. AWS DMS provides a variety of replication instances so you can choose the optimal configuration for your use case. For more information about the various classes of replication instances, see Choosing the right AWS DMS replication instance for your migration.

To begin, log in to AWS and navigate to the AWS DMS console for your Region. Choose Create replication instance.

Or you can navigate to Replication instances to begin:

1. In the navigation pane, choose Replication instances.

2. Choose Create replication instance.

Now you are ready to create the replication instance. On the Create replication instance page, specify your replication instance information. For more information about completing each field, select the + signs.

Name

–

Enter a name for the replication instance. The name must conform with the following constraints:

• It can contain up to 63 printable ASCII characters (excluding /,", and @).
• The first character must be a letter.
• The name cannot end with a hyphen.

The name should be unique for your account for the AWS Region you selected. You can choose to add some intelligence to the name. For example, you could include the AWS Region and task you are performing, such as west2-mysql2mysql-instance1.

Descripton

–

Enter a brief description of the replication instance. 

Instance class

–

Choose an instance class with the configuration you need for your migration. Keep in mind that the instance must have enough storage, network, and processing power to successfully complete your migration. For more information about how to determine which instance class is best for your migration, see Working with an AWS DMS replication instance.

                

Engine version

–

By default, the replication instance runs the latest version of the AWS DMS replication engine software. We recommend that you accept this default, although you can choose a previous engine version if necessary.

Allocated storage (GiB)

–

Storage is primarily consumed by log files and cached transactions. For cached transactions, storage is used only when the cached transactions need to be written to disk. If the replication instance has sufficient memory, AWS DMS can stream cached and ongoing changes without writing them to disk. Therefore, AWS DMS doesn't use a significant amount of storage. Some exceptions include the following:

• Very large tables that incur a significant transaction load – Loading a large table can take some time, so cached transactions are more likely to be written to disk during a large table load. For example, suppose that your load takes 24 hours, and you produce 2 GB of transactions each hour. In this case, you might want to ensure that you have 48 GB of space for cached transactions.

• Tasks that are configured to pause prior to loading cached transactions – In this case, all transactions are cached until the full load is complete for all tables. With this configuration, a significant amount of storage might be consumed by cached transactions.
• Tasks configured with tables being loaded into Amazon Redshift – This configuration isn't an issue when Amazon Aurora is the target.

In most cases, the default storage allocation is sufficient. However, it's always a good idea to pay attention to storage-related metrics, and scale up your storage if you find that you are consuming more than the default allocation. For more information, see the Choosing the optimum size for a replication instance section of the AWS DMS user guide.

VPC

–

Choose the virtual private cloud (VPC) that you want to use. If your source or target database is in a VPC, choose that VPC. If your source and target databases are in different VPCs, ensure that they are both in public subnets and publicly accessible. Then, choose the VPC where the replication instance is to be located.

The replication instance must be able to access the data in the source VPC. If neither your source nor your target database is in a VPC, select a VPC where the replication instance is to be located.

Multi-AZ

–

Use this optional parameter to create a standby replica of your replication instance in another Availability Zone for failover support.

If you intend to use change data capture (CDC) or ongoing replication, you should enable this option.Publicly accessible

–

Choose this option if you want the replication instance to be accessible from the internet.

Next, to set values for network and encryption settings, choose the Advanced security and network configuration tab. For more information, select the + signs.

Replication subnet group

–

Choose the replication subnet group in your selected VPC where you want the replication instance to be created. If your source database is in a VPC, choose the subnet group that contains the source database as the location for your replication instance. 

For more information about replication subnet groups, see Creating a replication subnet group.

Availability Zone

–

Choose the Availability Zone where your source database is located. 

VPC security groups

–

The replication instance is created in a VPC. If your source database is in a VPC, select the VPC security group that provides access to the DB instance.

AWS Key Management Service (AWS KMS) key

–

Choose the encryption key to encrypt replication storage and connection information. If you choose (Default) aws/dms, the default AWS KMS key associated with your account and AWS Region is used. A description and your account number are shown, along with the key's Amazon Resource Name (ARN).

For more information about using the encryption key, see Setting an encryption key and specifying AWS KMS permissions.

Now, specify the Maintenance settings. For more information, select the + signs.

Maintenance window (UTC)

–

Choose a weekly day and time range during which system maintenance can occur, in Coordinated Universal Time (UTC).

Default: A 30-minute window selected at random from an 8-hour block of time per AWS Region, occurring on a random day of the week

Next, add Tags settings to help you organize your AWS DMS resources.

Finally, choose Create.