Scaling Enterprise plan capacity

Gen 2

Event Streams capacity

The Event Streams Enterprise plan lets you specify throughput and storage capacity when a new instance of the service is created. If after using the service instance, you find that the current capacity configuration of your service instance is not meeting the demands of your solution, capacity can be scaled up to meet demands.

The scaling of throughput is currently not supported.

Throughput capacity

The throughput capacity represents the minimum throughput that can be expected to be continually available across the service instance assuming typical workloads. It also takes into account operational procedures and failure modes, such as the loss of an availability zone. Throughput is divided equally between produce and consume traffic. At any individual point in time, a higher throughput selected may be achievable but cannot be relied on to be continually available.

The minimum throughput option available is 100MB/s, divided into 50MB/s for produce workload and 50MB/s for consume workload. Scaling throughput is currently not supported. For more information on capacity recommendations and limitations, see limits and quotas. Although throughput scaling is independent of storage, for each tier a defined minimum of storage amount is required.

Storage capacity

Storage capacity is the amount of storage that is allocated in the service instance for retention of message data. Storage figures are quoted in decimal (base 10), for example, 2 TB represents 2x10^12 Bytes. Storage capacity can be scaled up, independent of throughput capacity, when data retention is important for your architecture. Storage capacity cannot be scaled down.

Event Streams stores three replicas of your data to ensure the highest level of resilience across three availability zones. When you select 2 TB of storage with Event Streams, it is equivalent to deploying 6 TB of storage, if you are running your own Apache Kafka cluster with the same replication policy enabled.

How to scale capacity

The following steps show you how to scale up throughput and storage capacity for an Event Streams Enterprise plan service instance. If you do not have an Enterprise instance, these steps help you to create one.

Currently, scaling up an Event Streams service instance capacity requires the use of the IBM Cloud CLI.

To install this tool, see install devtools.

The IBM Cloud CLI command uses the service-instance-update command to update your Event Streams service instance resource. The user ID in the account used to issue the service-instance- command must be assigned the same access policies that are needed when you create resources. For information about access requirements, see creating resources.

During the scale-up process

The time required to scale up the Event Streams service instance is variable, but under normal circumstances it does not exceed 3 hours. Both throughput and storage require provisioning extra infrastructure.

During this time, the flow of Kafka data is not affected (the scaling operation is nondisruptive).

Example

This example shows how to first deploy a service instance configured with a base capacity unit, which is the default:

100 MB/s of throughput capacity.
2 TB of storage capacity for data retention.

The example then shows how to scale this service instance to 4 TB of storage to provide:

100 MB/s of throughput capacity.
4 TB of storage capacity for data retention.

If you don't already have one, create an Event Streams service instance.

a. Log in to the IBM Cloud console.

b. Click the Event Streams service in the Catalog.

c. Select the Enterprise plan Gen2 on the service instance page, ensuring that a supported location is selected.

d. Review capacity selections of 100 MB/s throughput and 2 TB storage.

e. Enter a name for your service instance. You can use the default value.

f. Click Create. (For information about the amount of time needed to create the service instance, see Choosing your plan).
Log in to the IBM Cloud CLI.
```
ibmcloud login
```
Get the resource name of your Event Streams service instance.
```
ibmcloud resource service-instances
```
(You can find the name of your instance in the Name column.)

View the current capacity configuration of your instance.

ibmcloud resource service-instance "Event Streams resource instance name" -o JSON

The information is available in the kafka field of the extensions.dataservices section of the output:

"extensions": {
  "dataservices": {
  ...
    "kafka": {
       "storage_gb": 2000,
        "throughput_mb_s": 100,
    },
 ```

Scale up the service instance from 2 TB storage capacity to 4 TB storage capacity.

a. Run the following command from the CLI.

ibmcloud resource service-instance-update "Event Streams resource instance name" -p '{"dataservices":{"kafka":{storage_gb":"4000"}}}'

Monitor the update of the service instance.

The scale-up process can take from several minutes to three hours to complete depending on what new resources need to be allocated to the service instance.

You can get the current service instance information by using the following command.
```
ibmcloud resource service-instance "Event Streams resource instance name" --output=json
```
{: pre} Review the Last Operation section of the output. The information is continuously updated as the update proceeds. When the scale-up process completed, the last operation information indicates update succeeded or sync succeeded.

Run the command again until success is indicated.