Migrating from AWS

There are many tools to assist you to successfully migrate your information from AWS to IBM Cloud® Object Storage, with more secure and globally accessible results.

Before you begin

Determine your goals and process for your migration before starting your migration. You may also consider training and partnerships to be beneficial. Your planning and assessment stage will consider many possibilities, including security and technical capabilities.

Documentation for any project will help keep you keep track of your resources as well as your goals. After assessing your existing projects, you may benefit by updating them to use IBM Cloud Object Storage libraries like those for (Java, Python, Node.js). If you're interested in programmer interfaces, the REST API will provide an in-depth look at operations and configurations.

Refer to the getting started guide to familiarize yourself with key concepts such as endpoints and storage classes.

Provision and configure IBM Cloud Object Storage

If you haven't already, create an instance of IBM Cloud Object Storage from the Console.
Create any buckets that you anticipate will be needed to store your transferred data.
While Object Storage is compatible with the S3 API, it may be necessary to create new Service credentials, or bring your own keys for your projects. In this guide, we will use HMAC credentials similar to the format of AWS credentials.
Managing encryption provides insights into security. Refer to product documentation on IBM® Key Protect for IBM Cloud® and Hyper Protect Crypto Services for more information.

Determine your solution

It is true that a massively complex migration requires a complete service to plan and implement migrating your data to IBM Cloud Object Storage. But whatever the size of your data, your goals and timetable take precedence. Once you have provisioned and set your target, it is time to choose a process to achieve your goals on your time.

There are many ways to achieve the goal of migrating your AWS data. Integrated solutions provide comprehensive guides to migration, as shown in the IBM Cloud Pak for Integration. In addition to full-featured migration services, you may also want to investigate third party migration tools as part of your investigation. But don't forget that there are many CLI and GUI tools readily available for use as part of your migration.

COS CLI can be used for many operations. For example, you may wish to use the CLI to configure your IBM Cloud Object Storage instances, and to create and configure buckets.
AWS CLI can be used to list your current bucket's contents to prepare for migrating from AWS, among other operations:

aws s3 ls --recursive s3://<BUCKET_NAME> --summarize > bucket-contents-source.txt

rclone has many uses, and we'll look at it specifically, next.

Migrate your data

Based on the process and tools you've chosen, choose a strategy for migrating your data. Here is a simplified process using the command line and the Go-based rclone executable as an example.

Install rclone from either a package manager or pre-compiled binary. There are more configuration options available with explanations at the IBM Cloud Object Storage documentation.
```
curl https://rclone.org/install.sh | sudo bash
```

Configure `rclone` with your AWS credentials

Start by creating 'profiles' for your source and destination of the migration in rclone. A profile contains the configuration and credentials needed for working with your date. To migrate from AWS, those credentials are needed to continue. Also, create a profile for your destination credentials specifically for IBM Cloud Object Storage.

There are many options to configuring rclone and following the rclone config wizard is one way you can create profiles. You can create an rclone config file in ~/.rclone.conf by using the command as shown. Please use the root path of your home directory if the path shown isn't available.
```
touch ~/.config/rclone/rclone.conf
```
Create the AWS configuration settings by copying the following and pasting into rclone.conf using an appropriate editor.
```
[AWS]
type = s3
provider = AWS
env_auth = false
access_key_id =
secret_access_key =
region =
```
Paste your AWS access_key_id and secret_access_key as obtained per instructions here into the appropriate fields of your configuration as shown.

Configure `rclone` with your COS credentials

To complement the credentials of the source, we look at configuring the destination profile next.

Create the COS configuration settings by copying the following and pasting into rclone.conf using an appropriate editor.
```
[COS]
type = s3
provider = IBMCOS
env_auth = false
region =
access_key_id =
secret_access_key =
endpoint =
```
Paste your HMAC access_key_id and secret_access_key into the appropriate fields of your configuration as shown in the first step. As noted in the beginning of the guide, you will want to enter the appropriate values for your instance regarding your region and endpoint.

Verify your configurations

List the buckets from your source to verify rclone is properly configured for retrieval.
```
rclone lsd AWS:
```
List the COS bucket for your destination you created to verify rclone is properly configured for storage.
```
rclone lsd COS:
```

Use `rclone` to migrate from AWS

Do a dry run (no data copied) of rclone to sync the objects in your source bucket (for example, content-to-be-migrated) to the target COS bucket (for example, new-bucket).
```
rclone --dry-run copy AWS:content-to-be-migrated COS:new-bucket
```
Check that the files you want to migrate appear after running the command. If everything looks as you expect, remove the --dry-run flag and add a -v flag to show a verbose output while the data is being copied. Using the optional --checksum flag avoids updating any files that have the same MD5 hash and object size in both locations.
```
rclone -v copy --checksum AWS:content-to-be-migrated COS:new-bucket
```

As you perform the migration of your data using the process you've outlined, you will want to validate and verify the results.

Validating your migration from AWS

Integrated query-in-place dashboards allows you to see analytics based directly on your data. Using IBM Cloud Monitoring, you can follow up your migration using pre-built charts.

Next Steps

Get started by visiting the catalog, and creating the resources to begin your journey from AWS to IBM Cloud Object Storage with confidence and efficiency.