Provisioning

Provision an IBM Cloud® Databases for Elasticsearch Gen 2 deployment through the catalog, the Cloud Databases CLI plug-in, the Cloud Databases API, through Terraform, or through pre-built, open-source, and enterprise-ready Terraform IBM Modules (TIM).

Provisioning through the IBM Cloud console

Provision from the console by specifying the following parameters.

Service details

  • Service name The name can be any string and is the name that is used on the web and in the CLI to identify the new deployment.
  • Resource group If you are organizing your services into resource groups, specify the resource group in this field. Otherwise, you can leave it as the default. For more information, see Managing resource groups.
  • Location The deployment's public cloud region.

Hosting model

Databases for Elasticsearch Gen 2 uses Isolated Compute: a secure single-tenant offering for complex, highly performant enterprise workloads. Each deployment runs on dedicated virtual machines with guaranteed resources.

For more information, see Gen 2 isolated compute.

Resource allocation

Use the table to choose the machine size (CPU and RAM) for your deployment, and specify the disk size. Gen 2 offers six isolated compute sizes with guaranteed resources.

Verify that the available size selections in the UI match the current offerings. The six standard sizes are: 4x16, 8x32, 8x64, 16x64, 32x128, and 30x240 (CPU x RAM in GB).

Specify the disk size depending on your requirements. It can be increased after provisioning but cannot be decreased to prevent data loss.

Service configuration

  • Database version: Set only at deployment The deployment version of your database. To ensure optimal performance, run the preferred version. The latest minor version is used automatically. For more information, see Versioning policy.
  • Database edition: Set only at deployment Elasticsearch Enterprise with an Enterprise license is available on Gen 2. The Platinum license is not currently supported.
  • Encryption: Set only at deployment If you use Key Protect, an instance and key can be selected to encrypt the deployment's disk. If you do not use your own key, the deployment automatically creates and manages its own disk encryption key.
  • Endpoints: Set only at deployment Gen 2 deployments support private endpoints only. All endpoints are private by default and require VPC connectivity through Virtual Private Endpoints (VPE). For more information, see Private endpoints.

Gen 2 deployments are provisioned in a VPC infrastructure and require VPC connectivity. Ensure that you have configured Virtual Private Endpoints (VPE) before attempting to connect to your deployment.

After you select the appropriate settings, click Create to start the provisioning process.

Provisioning through the CLI

Create a service instance through the CLI

Before provisioning, complete the instructions provided in the documentation to install the IBM Cloud CLI tool.

  1. Log in to IBM Cloud. If you use a federated user ID, it's important that you switch to a one-time passcode (ibmcloud login --sso), or use an API key (ibmcloud --apikey key or @key_file) to authenticate. For more information about how to log in by using the CLI, see General CLI (ibmcloud) commands under ibmcloud login.

    ibmcloud login
    
  2. Provision your database with the following command:

    ibmcloud resource service-instance-create <INSTANCE_NAME> <SERVICE_NAME> <SERVICE_PLAN_NAME> <LOCATION> <RESOURCE_GROUP> -p '{"members_host_flavor": "<members_host_flavor value>"}'"
    

    For example, to provision a Databases for Elasticsearch Gen 2 instance with isolated compute, set the "members_host_flavor" parameter to the desired size. Available hosting sizes and their members_host_flavor value parameters are listed in Table 1. For example, {"members_host_flavor": "b3c.4x16.encrypted"}. Note that the host flavor selection includes CPU and RAM sizes (b3c.4x16.encrypted is 4 CPU and 16 RAM).

    ibmcloud resource service-instance-create test-database databases-for-elasticsearch enterprise-gen2 us-south -p '{"members_host_flavor": "b3c.4x16.encrypted"}'"
    

    The fields in the command are described in the table that follows.

    Basic command format fields
    Field Description Flag
    INSTANCE_NAME Required The instance name can be any string and is the name that is used on the web and in the CLI to identify the new deployment.
    SERVICE_NAME Required Name or ID of the service. For Databases for Elasticsearch, use databases-for-elasticsearch.
    SERVICE_PLAN_NAME Required enterprise or platinum. Note that platinum requires an isolated host flavor.
    LOCATION Required The location where you want to deploy. To retrieve a list of regions, use the ibmcloud regions command.
    RESOURCE_GROUP The Resource group name. The default value is default. -g
    --parameters JSON file or JSON string of parameters to create service instance -p
    members_host_flavor To provision an Isolated Compute instance, use {"members_host_flavor": "<members_host_flavor value>"}. Select desired CPU and RAM configuration. For more information, see the following table or Gen 2 isolated compute.

    In the CLI, service-endpoints is a flag, not a parameter.

The members host flavor parameter

The members_host_flavor parameter defines your Compute sizing. Gen 2 uses Isolated Compute with the following available sizes:

Gen 2 host flavor sizing parameter
Members Host flavor members_host_flavor value
4 CPU x 16 RAM b3c.4x16.encrypted
8 CPU x 32 RAM b3c.8x32.encrypted
8 CPU x 64 RAM m3c.8x64.encrypted
16 CPU x 64 RAM b3c.16x64.encrypted
32 CPU x 128 RAM b3c.32x128.encrypted
30 CPU x 240 RAM m3c.30x240.encrypted

You will see a response like the following:

```text {: codeblock}
Creating service instance INSTANCE_NAME in resource group default of account    USER...
OK
Service instance INSTANCE_NAME was created.
Name:                INSTANCE_NAME
ID:                  crn:v1:bluemix:public:databases-for-elasticsearch:us-south:a/   40ddc34a846383BGB5b60e:dd13152c-fe15-4bb6-af94-fde0af5303f4::
GUID:                dd13152c-fe15-4bb6-af94-fde0af56897
Location:            LOCATION
State:               provisioning
Type:                service_instance
Sub Type:            Public
Service Endpoints:   private
Allow Cleanup:       false
Locked:              false
Created at:          2023-06-26T19:42:07Z
Updated at:          2023-06-26T19:42:07Z
Last Operation:
                      Status    create in progress
                      Message   Started create instance operation
```
- To check provisioning status, use the following command:

  ```sh {: pre}
  ibmcloud resource service-instance <INSTANCE_NAME>
  ```
  When complete, you will see a response like the following:

  ```text {: codeblock}
  Retrieving service instance INSTANCE_NAME in resource group default under account USER's Account as USER...
  OK

  Name:                  INSTANCE_NAME
  ID:                    crn:v1:bluemix:public:databases-for-elasticsearch:us-south:a/40ddc34a953a8c02f109835656860e:dd13152c-fe15-4bb6-af94-fde0af5303f4::
  GUID:                  dd13152c-fe15-4bb6-af94-fde5654765
  Location:              <LOCATION>
  Service Name:          databases-for-elasticsearch
  Service Plan Name:     standard
  Resource Group Name:   default
  State:                 active
  Type:                  service_instance
  Sub Type:              Public
  Locked:                false
  Service Endpoints:     private
  Created at:            2023-06-26T19:42:07Z
  Created by:            USER
  Updated at:            2023-06-26T19:53:25Z
  Last Operation:
                          Status    create succeeded
                          Message   Provisioning elasticsearch with version 7.17 (100%)
  ```
- Optional: To delete a service instance, run the following command:

  ```sh {: pre}
  ibmcloud resource service-instance-delete <INSTANCE_NAME_OR_CRN>
  ```

Autoscaling is not currently available on Cloud Databases Gen 2. Monitor your resources using IBM Cloud® Monitoring integration, which provides metrics for memory, disk space, and disk I/O utilization. To add resources to your instance, manually scale your deployment.

The --parameters parameter

The service-instance-create command supports a -p parameter, which allows JSON-formatted parameters to be passed to the provisioning process. For example, you can pass Cloud Resource Names (CRNs) as parameter values, which uniquely identify a resource in the cloud. All parameter names and values are passed as strings.

For example, if a database is being provisioned from a particular backup and the new database deployment needs a total of 12 GB of memory across three members, the command to provision 4 GBs per member looks like the following:

ibmcloud resource service-instance-create databases-for-elasticsearch <INSTANCE_NAME> enterprise-gen2 us-south \
-p \ '{
  "backup_id": "crn:v1:blue:public:databases-for-elasticsearch:us-south:a/54e8ffe85dcedf470db5b5ee6ac4a8d8:1b8f53db-fc2d-4e24-8470-f82b15c71717:backup:06392e97-df90-46d8-98e8-cb67e9e0a8e6",
  "members_memory_allocation_mb": "4096"
}'

Provisioning through the Resource Controller API

Complete these steps to provision by using the Resource Controller API.

  1. Obtain an IAM token from your API token.

  2. You need to know the ID of the resource group that you would like to deploy to. This information is available through the IBM Cloud CLI.

    Use a command like:

    ibmcloud resource groups
    
  3. You need to know the region you want to deploy to.

    To list all of the regions that deployments can be provisioned into from the current region, use the Cloud Databases CLI plug-in.

    The command looks like:

    ibmcloud cdb regions --json
    
  4. A host flavor represents fixed sizes of guaranteed resource allocations. To see which host flavors are available in your region, call the host flavors capability endpoint.

    The Isolated Compute host flavors available to a Databases for Elasticsearch Gen 2 instance in the us-south region are:

    • b3c.4x16.encrypted
    • b3c.8x32.encrypted
    • m3c.8x64.encrypted
    • b3c.16x64.encrypted
    • b3c.32x128.encrypted
    • m3c.30x240.encrypted

    To provision or scale your instance to 4 CPUs and 16384 megabytes or RAM, submit the following command:

    {
      "parameters": {
        "members_host_flavor": "`b3c.4x16.encrypted`"
      }
    }
    

    To scale your instance up to 8 CPUs and 32768 megabytes of RAM, submit the following command:

    {
      "parameters": {
        "members_host_flavor": "b3c.8x32.encrypted"
      }
    }
    
  5. When you have all the information, provision a new resource instance with the IBM Cloud Resource Controller.

    curl -X POST \
      https://resource-controller.cloud.ibm.com/v2/resource_instances \
      -H "Authorization: Bearer <TOKEN>" \
      -H 'Content-Type: application/json' \
        -d '{
        "name": "<INSTANCE_NAME>",
        "target": "<targeted-region>",
        "resource_group": "RESOURCE_GROUP_ID",
        "resource_plan_id": "<SERVICE_PLAN_NAME>"
        "parameters": {
            "members_host_flavor": "<members_host_flavor_value>",
            "version":"<VERSION>"
        }
      }'
    

    Provision a Databases for Elasticsearch Gen 2 instance with the "members_host_flavor" parameter set to the desired Isolated size. Available hosting sizes and their members_host_flavor value parameters are listed in Table 1. For example, {"members_host_flavor": "b3c.4x16.encrypted"}. Note that the host flavor selection includes CPU and RAM sizes (b3c.4x16.encrypted is 4 CPU and 16 RAM).

    curl -X POST \
      https://resource-controller.cloud.ibm.com/v2/resource_instances \
      -H "Authorization: Bearer <TOKEN>" \
      -H 'Content-Type: application/json' \
        -d '{ \
        "name": "my-instance", \
        "target": "us-south", \
        "resource_group": "5g9f447903254bb58972a2f3f5a4c711", \
        "resource_plan_id": "<SERVICE_PLAN_NAME>", \
        "parameters": {
        "members_host_flavor": "b3c.4x16.encrypted",
        } \
      }' \
    

    The parameters name, target, resource_group, resource_plan_id, and service_endpoints are all required.

    The fields in the command are described in the following table.

    Basic command format fields
    Field Description Flag
    NAME Required The instance name can be any string and is the name that is used on the web and in the CLI to identify the new deployment.
    SERVICE_NAME Required Name or ID of the service. For Databases for Elasticsearch, use databases-for-elasticsearch.
    SERVICE_PLAN_NAME Required enterprise or platinum
    TARGET Required The region where you want to deploy. To retrieve a list of regions, use the ibmcloud regions command.
    RESOURCE_GROUP The Resource group name. The default value is default. -g
    --parameters JSON file or JSON string of parameters to create service instance -p
    members_host_flavor To provision an Isolated Compute instance, use {"members_host_flavor": "<members_host_flavor value>"}. Select desired CPU and RAM configuration. For more information, see Gen 2 host flavor sizing parameter, or Gen 2 isolated compute.
    service-endpoints Required Gen 2 supports private endpoints only. For more information, see Service endpoints.

The members host flavor parameter

The members_host_flavor parameter defines your Compute sizing. Gen 2 uses Isolated Compute with the following available sizes:

Gen 2 host flavor sizing parameter
Members host flavor members_host_flavor value
4 CPU x 16 RAM b3c.4x16.encrypted
8 CPU x 32 RAM b3c.8x32.encrypted
8 CPU x 64 RAM m3c.8x64.encrypted
16 CPU x 64 RAM b3c.16x64.encrypted
32 CPU x 128 RAM b3c.32x128.encrypted
30 CPU x 240 RAM m3c.30x240.encrypted

Autoscaling is not currently available on Cloud Databases Gen 2. Monitor your resources using IBM Cloud® Monitoring integration, which provides metrics for memory, disk space, and disk I/O utilization. To add resources to your instance, manually scale your deployment.

List of additional parameters

In the --parameters object you can provide additional information to create your service instance, including:

  • backup_id A CRN of a backup resource to restore from. The backup must be created by a database deployment with the same service ID. The backup is loaded after provisioning and the new deployment starts up that uses that data. A backup CRN is in the format crn:v1:<...>:backup:<uuid>. If omitted, the database is provisioned empty.

  • version The version of the database to be provisioned. If omitted, the database is created with the most recent major and minor version.

  • disk_encryption_key_crn The CRN of a KMS key (for example, Hyper Protect Crypto Services or Key Protect), which is then used for disk encryption. A KMS key CRN is in the format crn:v1:<...>:key:<id>.

  • backup_encryption_key_crn - The CRN of a KMS key (for example, Hyper Protect Crypto Services or Key Protect), which is then used for backup encryption. A KMS key CRN is in the format crn:v1:<...>:key:<id>.

    To use a key for your backups, you must first enable the service-to-service delegation.

  • members_disk_allocation_mb Total amount of disk to be allocated to the database. For example, if the value is "30720", and there are three members, the deployment gets 30 GB of disk total, giving 10 GB of disk per member. If omitted, the default value for the database type is used.

Provisioning with Terraform

Use Terraform to manage your infrastructure through the ibm_database Resource for Terraform, which supports provisioning Cloud Databases deployments. Alternatively, you can use Terraform IBM Modules to manage your infrastructure through Terraform IBM Modules for Databases for Elasticsearch.

Provisioning with Terraform

Provision a Databases for Elasticsearch Gen 2 instance with the "host_flavor" parameter set to the desired Isolated size. Available hosting sizes and their host_flavor value parameters are listed in Table 1. For example, {"host_flavor": "b3c.4x16.encrypted"}. Note that the host flavor selection includes CPU and RAM sizes (b3c.4x16.encrypted is 4 CPU and 16 RAM).

data "ibm_resource_group" "group" {
  name = "<your_group>"
}
resource "ibm_database" "<your_database>" {
  name              = "<your_database_name>"
  plan              = "enterprise-gen2"
  location          = "eu-gb"
  service           = "databases-for-elasticsearch"
  resource_group_id = data.ibm_resource_group.group.id
  tags              = ["tag1", "tag2"]
  # Pin the database version at provision time.
  # If omitted, the newest available major/minor is selected (per service behavior).
  version = "<VERSION_STRING>"                  # e.g. "8.19" for Elasticsearch preferred major
  group {
    group_id = "member"
    host_flavor {
      id = "b3c.8x32.encrypted"
    }
    disk {
      allocation_mb = 256000
    }
  }
}
output "ICD Elasticsearch database connection string" {
  value = "http://${ibm_database.test_acc.ibm_database_connection.icd_conn}"
}

The host flavor parameter

The host_flavor parameter defines your Compute sizing. Gen 2 uses Isolated Compute with the following available sizes:

Gen 2 host flavor sizing parameter
Host flavor host_flavor value
4 CPU x 16 RAM b3c.4x16.encrypted
8 CPU x 32 RAM b3c.8x32.encrypted
8 CPU x 64 RAM m3c.8x64.encrypted
16 CPU x 64 RAM b3c.16x64.encrypted
32 CPU x 128 RAM b3c.32x128.encrypted
30 CPU x 240 RAM m3c.30x240.encrypted

Autoscaling is not currently available on Cloud Databases Gen 2. Monitor your resources using IBM Cloud® Monitoring integration, which provides metrics for memory, disk space, and disk I/O utilization. To add resources to your instance, manually scale your deployment.