Postgresql Migration - part2

 

Specifying Source and Target Endpoints

While your replication instance is being created, you can specify the source and target data stores. These are called endpoints. The source and target endpoints can be on an EC2 instance, an Amazon RDS DB instance, or an on-premises database. The only requirement to use AWS DMS is that one of your endpoints must be on an AWS service.

Create endpoints

Begin with the source endpoint. From the side panel, choose Endpoints. Then, choose the orange Create endpoint button.

Source endpoint

The source endpoint tells AWS DMS to read data from a database. If you are using an Amazon RDS DB instance as your source endpoint, select the Select RDS DB instance check box.

Endpoint configuration

Now you will define the connection details. To proceed through the steps, select the Start button.

Endpoint identifier

The endpoint identifier is a label for the endpoint to help you identify it. Enter the name you want to use to identify the endpoint. Include in the name the type of endpoint. Two examples might be  Oracle-source or PostgreSQL-target. The name must be unique for all replication instances.

You also can opt to add a descriptive Amazon Resource Name (ARN).

Source engine

Next, select your source engine. Identify the type of database engine that is on the source endpoint.

Access to endpoint database

This step offers two options: 

1. Use AWS Secrets Manager.
2. Provide access information manually.

Decide which you will use, and enter the appropriate information. If you choose to enter the information manually, ensure that the user name has the permissions required to allow data migration.

Extra connection attributes

Extra connection attributes define the additional endpoint-specific settings for AWS DMS. For more information about extra connection attributes, see the documentation section for your data store from the AWS DMS User Guide.

AWS KMS primary key

To keep your endpoint data secure during migration, AWS DMS encrypts passwords and other important information using AWS KMS. Encryption requires an AWS KMS primary key. AWS DMS creates a default primary key for you if you don't already have one.

Tags

You can add tags to organize your AWS DMS resources. You can use tags to manage your AWS Identity and Access Management (IAM) roles and policies, and track your AWS DMS costs. You can learn more by reviewing the AWS DMS User Guide.

Test endpoint connection

A final and optional step in setting up the source endpoint is to test it. If you test the endpoint and it fails, you have the opportunity to edit the endpoint definition and test it again before creating it. Your endpoint will always be created, regardless of whether you run a test or the connection fails.

Target endpoint 

After you have created your source endpoint, you need to create your target endpoint. This is the PostgreSQL database. You will go through the same process, but select the Select RDS DB instance check box. Fill in the applicable information.

When the information is complete, choose the Create endpoint button. You have now created the source and target endpoints and are ready to move on to the tasks.

Creating a Task

Next, you will create a task. Creating a task specifies which tables to migrate, maps data using a target schema, and creates new tables on the target database. As part of creating a task, you can choose the type of migration.

Using AWS DMS, you can specify the precise mapping of your data between the source and target database. Before you specify your mapping, make sure you review the documentation section about data type mapping for your source and target database.

You can start a task as soon as you finish specifying information for that task on the Create task page. You can also start the task from the dashboard when you finish specifying task information, as shown below.

Task configuration

Now you will configure the task to enable AWS DMS to migrate. To configure the task, follow these steps:

1. Name the task in the Task identifier field.
2. Select your replication instance.
3. Select your source database endpoint.
4. Select your target database endpoint.
5. Select your migration type.

The Migration type field provides you with three choices. You can choose to: 

• Have just the existing data migrated to the target database
• Have ongoing changes sent to the target database, in addition to the migrated data
• Replicate data changes only

Task settings

Task settings control the overall behavior of your task. These settings can affect how your task manages a variety of features that often depend on the source and target endpoints and the table mappings that you specify. For example, you can tune the performance of full loads and CDC loads, or activate logging. You can enable data validation and set stream buffer settings. You can set the before image option as tasks run. You can also specify how your task responds to control table settings.

For a full list of task settings, refer to the AWS DMS User Guide, Specifying task settings for AWS Database Migration Service tasks.

There are two options when you are editing the task settings. You can use the wizard or the JavaScript Object Notation (JSON) editor. If you opt to use JSON, you can find information on the specific settings for each task in the AWS DWS User Guide. This course will focus on using the wizard. 

The following task settings depend on the migration type value that you select. For example, when you select Migrate existing data and replicate for Migration type, the following options are shown. For more information, select each +.

The following task settings depend on the migration type value that you select. For example, when you select Migrate existing data and replicate for Migration type, the following options are shown. For more information, select each +.

To learn how to activate validation and Amazon CloudWatch logs, review each of the following tabs.

ENABLE VALIDATIONENABLE AMAZON CLOUDWATCH LOGS

Choose this setting if you want AWS DMS to compare the data at the source and the target immediately after it performs a full data load. Validation helps ensure that your data was migrated accurately, but it requires additional time to complete.

Control tables

Control tables provide information about an AWS DMS task. They also provide useful statistics that you can use to plan and manage the current migration task and future tasks. You can apply these control table task settings in the Advanced task settings section, as shown below.

You can review the AWS DMS User Guide for more information about how to control table task settings.

Premigration assessment

A premigration assessment warns you of potential migration issues before starting your database migration task. You can choose one or more premigration assessments for an assessment run that you create for a task. You can learn more about premigration assessments by reviewing the documentation in the AWS DMS User Guide. 

Migration task startup configuration

Next, choose how to start your migration task. You can choose to start it automatically when the task is created, or you can manually start it later.

Tags

Just like with endpoints, tasks can be tagged. As a reminder, tags are used to add metadata to your resources. You can use tags with IAM policies to manage access to AWS DMS resources and control which actions can be applied to the AWS DMS resources.

Create task

When you have finished with the task settings, complete the process by choosing Create task

Monitoring Your Task

Monitoring is an important part of helping maintain the reliability, availability, and performance of AWS DMS and your AWS solutions. You should collect monitoring data from all parts of your AWS solution to help debug a multi-point failure if one occurs.

You can view and monitor information for your active task by choosing that specific task from the AWS Management Console.

Monitor your task

AWS provides five tools for monitoring your AWS DMS tasks. To learn more, select the following + signs.

AWS DMS events and notifications

–

AWS DMS uses Amazon Simple Notification Service (Amazon SNS) to provide notifications when an AWS DMS event occurs, such as the creation or deletion of a replication instance. For more information, see Working with events and notifications in AWS Database Migration Service.

Task status

–

You can monitor the progress of your task by checking the task status and monitoring the task's control table. Task status indicates the condition of an AWS DMS task and its associated resources. It includes status indicators, such as whether the task is being created, starting, running, or stopped. It includes the current state of the tables that the task is migrating, such as whether a full load of a table has begun or is in progress. Task status also includes details such as the number of inserts, deletes, and updates that have occurred for the table.

For more information about monitoring tasks and task resource condition, see Task status and Table state during tasks. For more information about control tables, see Control table task settings.

Amazon CloudWatch alarms and logs

–

You can use CloudWatch to watch one or more task metrics over a time period that you specify. AWS DMS also uses CloudWatch to log task information during the migration process. You can use the AWS CLI or the AWS DMS application programming interface (API) to view information about the task logs. 

For more information about using CloudWatch with AWS DMS, see Monitoring replication tasks using Amazon CloudWatch. For more information about monitoring AWS DMS metrics, see AWS Database Migration Service metrics. For more information about using AWS DMS task logs, see Viewing and managing AWS DMS task logs. 

AWS CloudTrail logs

–

AWS DMS is integrated with AWS CloudTrail, a service that logs a record of actions taken by a user, IAM role, or an AWS service in AWS DMS. CloudTrail captures all API calls for AWS DMS as events, including calls from the AWS DMS console and from code calls to the AWS DMS API operations.

If you create a trail, you can activate continuous delivery of CloudTrail events to an Amazon Simple Storage Service (Amazon S3) bucket, including events for AWS DMS. If you don't configure a trail, you can still view the most recent events in the CloudTrail console in Event history. Using the information collected by CloudTrail, you can determine the request that was made to AWS DMS. You can also determine the IP address from which the request was made, who made the request, when it was made, and additional details. For more information, see Logging AWS DMS API calls with AWS CloudTrail.

Database logs

–

You can view, download, and monitor database logs for your task endpoints using the AWS Management Console, AWS CLI, or the API for your AWS database service. For more information, see the documentation for your database service at AWS documentation.

This course covered how AWS DMS helps you securely migrate databases to AWS. You can migrate your data to and from the most widely used commercial and open-source databases, such as Oracle, MySQL, and PostgreSQL.

Now that you have completed this course, you should be able to:

• Explain migration options using AWS DMS.
• Describe how to migrate a database using the AWS DMS console.
• Identify ways to monitor AWS DMS tasks.