Connecting to Event Streams

To connect to your Event Streams instance, you need the endpoint URLs for the APIs and the credentials for authentication. Learn how to obtain these details and the connectivity options that you can use.

Overview

Services that are provisioned by using the Lite, Standard, Enterprise, or Satellite plans are grouped in the dashboard under the heading Services.

All plans use IAM for authentication. You don't need to understand IAM to get started but some knowledge is recommended if you want to secure your Event Streams service. For more information, see Managing access to your Event Streams resources. To complete the following steps and be authorized to create topics, your application or Service Key must have a Manager access role. By default, the owner of the account that contains the service instance has this role.

By default, Event Streams instances are configured to use the IBM Cloud® public network, so they are accessible over the public internet. If required, you can restrict this access by selecting an alternative networking type or restricting the location that connections are accepted from. For more information, see Restricting Network Access.

Connection information

To access a service instance, create a service key. A service key contains the information that is needed to access the instance, including the endpoint details for its APIs and a unique API key credential.

To create a service key by using the IBM Cloud console:

Locate your Event Streams service on the dashboard.
Click your service tile.
Click Service Credentials.
Click New Credential.
Complete the details for your new credential. Choose a name and a role and click Add. A new credential appears in the credentials list.
Click the new credential by using View Credentials to reveal the details in JSON format.

To create a service key by using the IBM Cloud CLI, complete the following steps.

Locate your service:
```
ibmcloud resource service-instances
```

Create a service key:

ibmcloud resource service-key-create <key_name> <key_role> --instance-name <your_service_name>

Print the service key:
```
ibmcloud resource service-key <key_name>
```
A single set of endpoint details are contained in each service key. For service instances configured to be connected to a single network type, either the IBM Cloud Public network (the default) or the IBM Cloud Private network, the service key contains the details relevant to that network type. For instances configured to support both the private and public networks, details for the public network are returned. If you want details for the private network, you must add the --service-endpoint private parameter the previous CLI command, as in the following example.
```
ibmcloud resource service-key-create <private-key-name> <role> --instance-name <instance-name> --service-endpoint private
```

For more information, see Network types.

Establishing a connection

To connect a Kafka application, complete the following tasks.

Use the <bootstrap_endpoints> field from the service key as the bootstrap.servers property of your Kafka application.
Set the security.protocol property to SASL_SSL and the sasl.mechanism property to PLAIN.
Use the <user> field from the service key as the username and the <api_key> field from the service key as the password. Ensure that your application parses the details.
For more information, see Configuring your Kafka API client.

To call an HTTP API, complete the following tasks.

Use the <kafka_admin_url> field of the service key as the base URL for HTTP requests.
Use the IBM Cloud CLI ibmcloud iam oauth-tokens command to generate an auth token. Place this token in the Authorization header of the HTTP request with the value formatted as Bearer <token>. Both API key or JWT tokens are supported.
Further documentation is provided for each API.

Network connectivity

For the Satellite plan, the network connectivity depends on the infrastructure that is used and is therefore the responsibility of the user.

By default, Lite, Standard, and Enterprise instances are configured to be accessible over the public internet. If you're using the Enterprise plan, you can restrict connectivity as follows:

Private networking: If your workload is running entirely within the IBM Cloud, and public access to the service is not required, Event Streams instances can instead be configured to be accessible only over the IBM Cloud Private network. This configuration offers increased isolation and does not incur the egress bandwidth charges associated with public traffic. Instances can also be configured to be accessible over both the IBM Cloud Public and Private networks.
Context-based restrictions: You can define access rules that limit the network locations that connections are accepted from according to certain characteristics. For example, network type, IP ranges, VPC, or other services.

For more information, see Restricting network access.

Accessing an Enterprise instance over the private network from Classic infrastructure

To access your Enterprise instance over the private network for workloads deployed on IBM Cloud classic infrastructure, the Virtual Route Forwarding (VRF) and Service Endpoints features must be enabled in your account. For more information, see Restricting network access.

Accessing an Enterprise instance over the private network from a VPC

For workloads deployed in an IBM Cloud VPC to be able to access your Enterprise instance over the private network, a Virtual Private Endpoint (VPE) must be created in the VPC:

In the IBM Cloud console, click the menu icon and select VPC infrastructure > Network > Virtual private endpoint gateways.
Create a VPE for your Event Streams instance by using the guidance in About virtual private endpoint gateways.
After you create your VPE, it might take a few minutes for the new VPE and pDNS to complete the process and begin working for your VPC. Completion is confirmed when you see an IP address set in the details view of the VPE.

Accessing an Enterprise instance over the private network from outside the IBM Cloud

Solutions such as Direct Link 2.0 can be utilized to establish a connection between an external network, such as an on-premise data center, and the IBM Cloud private network. However, when dealing with workloads operating on an external network, it is essential to consider additional factors to ensure a successful connection to Kafka. Note that these considerations do not apply to HTTP workloads.

The private endpoint details allocated to your instance (as described in the service key) must be resolvable and routable from the network that the workload is running in. It is not possible to specify alternative hostname entries in the workload's bootstrap.servers properties as a way to route traffic from the external network.

The reason for this behavior is Kafka's two-step connection process. In the initial step, the hostnames that are provided in the client's bootstrap.servers property are used to establish the first bootstrap connection. However, the server then responds to the client with the actual endpoint hostname details it uses. These hostname details are the private endpoint details originally allocated to your instance and cannot be changed. Hence, the details must be resolvable and routable directly from the external network.

For more information, see Accessing private API endpoints from an on-premises network by using IBM Cloud Direct Link.

What to do next

Now you have connection and credential information, you can choose a Kafka client. For more information, see Using the Kafka API.