This article data science blogthon.
prologue
DMS is a service that makes it easy to migrate your on-premises databases to the cloud with minimal or no downtime. You can also monitor the original database and apply it to the new database. DMS supports all databases such as MySQL, PostgreSQL Oracle, SQL Server, Microsoft SQL Server, Amazon Aurora. The advantage is that the original and new databases do not have to match. So it’s easy to change from one type of database to another. DMS takes care of all the tedious work of exporting data from the old database and importing it into the new one in minutes. Supports data migration tasks such as database development, integration, and testing. The source database remains fully functional during the migration, minimizing downtime for applications that rely on it. In this article, I will explain what DMS is, how to migrate data, the process, and why you should consider his DMS for your migration.
What is AWS DMS?
AWS Database Migration Service (AWS DMS) helps you quickly migrate your database to AWS. You can migrate multiple databases, including relational, non-relational, and other types of commercial and open source databases. Continuously monitor the source and target databases, as well as network connectivity. If there is an interruption that stops the replication task, DMS will automatically fix it. Consolidate heterogeneous databases into a multi-petabyte data warehouse by streaming data to AWS Redshift and Amazon S3.
Why Consider AWS DMS for Data Migration
High resilience:
DMS can automate backup snapshots and restore them very quickly. If the primary replication server fails, the backup replication server takes over. Therefore, there is no service interruption.
- minimumThe source database remains fully functional while the database is migrated through DMS, minimizing downtime for database-dependent applications. This allows users to make changes to the database whenever they want without having to shut it down.
- Low price :
DMS offers free migration services to migrate your database to Redshift, Aurora, DynamoDB, or DocumentDB. Depending on the amount of logs stored, migration costs to other databases will be incurred.
- Comprehensive database support:
DMS supports homogeneous migrations from Oracle to Oracle and heterogeneous migrations from Oracle to the Aurora platform using SCT tools. Supports open source and commercial databases for convenient database migration.
- Failover support:
DMS is deployed in a Multi-AZ environment to help with failover support and easy availability. In a Multi-AZ deployment, DMS automatically maintains and provisions a synchronous standby replica of your replication instance in another Availability Zone.
AWS DMS process – how it works
- Connect your endpoints.
First, connect your DMS to both the source and target endpoints. The main requirement for DMS is that one user’s endpoint must be hosted on her AWS service. To perform the migration, the endpoint credentials must successfully connect to the database with the required permissions.
- Task scheduling:
Now schedule a task to run on the server to migrate the data. DMS reads data from the source database and prepares it for the target database. A full load migration occurs when source data is transferred to a target database.
- Database synchronization:
As changes are made in the source database, they are cached on the replication server. As soon as the full load migration is complete, these cached changes are applied to the target database, keeping both databases in sync.
Supported database types
Source: https://www.logicata.com/blog/aws-database-migration-service
Use AWS DMS with other AWS services. You can use DMS with other AWS services including:
- Amazon EC2 and RDS:
Users can use Amazon EC2 or RDS DB instances as targets for data migration. EC2 is a virtual machine that represents a physical server that provides secure and scalable computing power. RDS refers to a SQL database service that allows you to access your files and databases from anywhere.
- AWS Schema Conversion Tool:
The AWS Schema Conversion Tool or SCT will automatically convert your source schema into a compatible format for your target database. SCT allows you to migrate your database schema including stored procedures, views, functions, etc. to a compatible target database.
- Amazon S3:
Amazon S3 can be used in intermediate steps for storage purposes or while migrating large amounts of data. A user can set up a database migration task that transfers specific data to her S3 bucket. - AWS cloud formation:
AWS CloudFormation can set up the deployment or management of AWS resfrastructures. CloudFormation templates allow you to create, delete, and modify entire stacks (a collection of AWS resources) as a single unit or as stacks across multiple AWS accounts and Regions without having to manage resources individually.
Components of AWS Database Migration Service
There are three main DMS components:
- replication instance
- source and target endpoints
- replication task
Replication instance:
An Amazon EC2 (Elastic Compute Cloud) managed instance hosts one or more replication jobs. Handle compute-intensive workloads, heterogeneous migrations, and replication, such as Oracle to PostgreSQL migrations. Memory-optimized replication instances such as R4/R5 are useful for medium-sized in-memory databases, real-time big data analytics, and enterprise applications.
Source and target endpoints:
Endpoints link both source and target databases and help transfer data. All endpoints require information such as server name, port number, endpoint type, engine type, and credentials. Before using the endpoint for database migration tasks, the user should test the endpoint connection.
Replication task:
This is the last step before starting a migration that transfers data from the source endpoint to the destination endpoint. It shows the data that will be transferred between the target and source endpoints and when the migration should be performed.
Use Case
Homogeneous database migration:
Source: https://www.heptabit.com/blog/database-migration-to-aws-with-no-downtime
This migration requires both the source and target database engines to match and be interoperable. Examples include Microsoft SQL Server to Amazon RDS for SQL Server, Oracle to Amazon RDS for Oracle, MySQL to Amazon RDS for MySQL, and MySQL to Amazon Aurora.
Heterogeneous database migration:
The source and target database engines do not complement each other in this migration. Schema and code conversion steps are required here to address differences in database code, data types, and schema structure between the source and target databases. Examples include Oracle to PostgreSQL, Oracle to Amazon Aurora, and Microsoft SQer to MySQL migrations.
Create a migration plan:
DMS allows users to assess and track the progress of database migrations. The DMS dashboard shows the migration status regardless of the region where the user moves the database. Detect and resolve migration issues.
Continuous data replication:
DMS helps you replicate data continuously with high availability. DMS helps you synchronize the location of your development and test environments with the location of your disaster recovery instance through continuous data replication.
Conclusion
Ultimately, we can conclude that AWS DMS is a great tool for easily migrating database workloads to AWS and also allows for continuous data replication. It’s easy to set up, highly reliable, supports a wide variety of databases, and constantly monitors the target database’s network connection.
Important points:
- AWS DMS helps you quickly migrate your databases to AWS and monitor your source and target databases.
- Deployed in a Multi-AZ environment for ease of use.
- You can use the AWS Schema Conversion Tool or AWS SCT with DMS to automatically convert your source schema to a compatible format for your target database.
- There are three main components – replication instances, source and target endpoints, and replication tasks.
- Supports both homogeneous and heterogeneous database migrations.
- Endpoints point to two data stores: source and target.
- DMS assists in the ongoing replication of data and synchronizes the locations of development and test environments.
Media shown in this article are not owned by Analytics Vidhya and are used at the author’s discretion.