Migrating Elasticsearch
If you are a current user of Elasticsearch, use the Snapshot/restore Elasticsearch API and the S3 repository plug-in to migrate your data into IBM Cloud® Databases for Elasticsearch.
Take a snapshot of your existing Elasticsearch, store the snapshot in an AWS S3 or IBM Cloud® Object Storage bucket and then restore the snapshot to your Databases for Elasticsearch deployment.
To migrate while data is still being written to your existing Elasticsearch, take multiple snapshots and perform multiple incremental restores. After the Databases for Elasticsearch deployment catches up to the state of your Elasticsearch, move your application writes to Databases for Elasticsearch.
Requirements
Ensure you meet the following requirements before migrating:
- Your existing Elasticsearch must have the S3 repository plug-in.
- An Elasticsearch that is version 5.x or 6.x. Migration is possible between major versions, but the versions must have compatible indexes. Indexes that are made in Elasticsearch 2.x are not compatible with 6.x and need reindexing in 5.x before migration.
- A matching Databases for Elasticsearch deployment that has at least as many resources allocated to it your existing Elasticsearch. Also, ensure that the same plug-ins are available on Databases for Elasticsearch that you have on the existing Elasticsearch.
- Your our own S3 or IBM Cloud Object Storage repository.
Incremental restores work only if the number of shards of each index on both deployments match. Don't try to reindex and change the number of shards of any indexes once you've taken snapshots.
Example Migration
A migration is explored in detail in Elasticsearch data migration using snapshot and restore.
In this example, data is still being written while the migration occurs, so multiple snapshots and restores are used.
The example's shell script is available in the IBM Cloud GitHub repository. It is provided as a starting point for you to adapt for your use-case.