IBM Cloud API Docs

Introduction

Last updated: 2022-05-16

IBM Event Streams for IBM Cloud® is a high-throughput message bus built with Apache Kafka. It is optimized for event ingestion into IBM Cloud and event stream distribution between your services and applications.

Event Streams provides a REST API to help connect your existing systems to your Event Streams Kafka cluster. Using the API, you can integrate Event Streams with any system that supports RESTful APIs.

The REST producer API is a scalable REST interface for producing messages to Event Streams over a secure HTTP endpoint. Send event data to Event Streams, utilize Kafka technology to handle data feeds, and take advantage of Event Streams features to manage your data.

Use the API to connect existing systems to Event Streams. Create produce requests from your systems into Event Streams, including specifying the message key, headers, and the topics that you want to write messages to.

The REST producer API is available as part of the Event Streams Standard and Enterprise plans only.

You can download the OpenAPI specification compliant descriptors for this REST interface. Alternatively, click the three vertical dots next to the IBM Event Streams REST Producer v2 endpoint title in the top left corner of this app, then click the Download OpenAPI definition link that appears.

For more information about Event Streams, see What is Event Streams?

Other APIs

IBM Event Streams also provides the following APIs:

Endpoint URLs

The URL for the API's endpoint is provided in the kafka_http_url property.

Authentication

The supported authentication mechanism is using a bearer token. To obtain your token using the IBM Cloud CLI, first log in to IBM Cloud, then run the following command:

ibmcloud iam oauth-tokens

Place this token in the Authorization header of the HTTP request in the form Bearer<token>. Both API key or JWT tokens are supported.

Error handling

This API uses standard HTTP response codes to indicate whether a method completed successfully. A 200 response indicates success. A 400 type response indicates a failure, and a 500 type response indicates an internal system error.

HTTP status code Description Recovery
200 OK Message successfully sent to Event Streams.
400 Bad request Not a valid request.
401 Unauthorized You are unauthorized to produce to this topic.
403 Forbidden The token that you have provided does not have sufficient permissions to produce to a topic. Check if you have provided the token with the required access role.
404 Not found The topic does not exist. If Kafka setting auto.create.topics.enable is set to true (the default), Kafka automatically creates the topic with the default replication factor and number of partitions. If Kafka setting auto.create.topics.enable is set to false, this error message is displayed when attempting to produce to a topic that does not exist.
429 Too many requests Too many consecutive requests.
415 Unsupported Media Type The provided value for Content-Type header is unsupported.
408 Request timeout The request timed out producing to Event Streams.
500 Internal server error The request failed due to an internal server error.
503 Service unavailable The request failed due to Event Streams brokers being unavailable.

Kafka SDKs

Event Streams is based on the open-source Apache Kafka, therefore all SDKs are provided for by the open-source community, covering the core runtimes listed. For the most up-to-date information about these libraries refer to the project homepages. For example:

You can find details of what is supported by Event Streams at: Support summary for all recommended clients.

For information about how to install and use the SDKs, see Using the Kafka Java client.

Event Streams samples

For information about running a sample Java application that demonstrates how to connect to Event Streams and how to send and receive messages using the Kafka Java API, see:

Methods

Produce a message to a topic in IBM Event Streams.

Produce a message to a topic in IBM Event Streams. Messages can include headers, keys and values. If required, an Avro schema can be specified for keys and values.

POST /v2/topics/{topic_name}/records

Request

Path Parameters

  • topic_name
    Required*
    string

    The name of the topic you want to produce the messages to.

    Possible values: 1 ≤ length ≤ 200, Value must match regular expression ^[a-zA-Z0-9-_.]*$

Query Parameters

  • serializer
    string

    The type of third party serializer to be used. This must be specified when using schemas. Supported value is "confluent".

    Allowable values: [confluent]

Request Body

Required*
One of

The details of the message to be produced to the topic.

  • curl -i -X POST -H 'Accept: application/json' -H 'Content-Type: application/json -H 'Authorization Bearer ${token}' --data ' {
 "headers": [
  {
   "name": "colour",
   "value": "YmxhY2s="
  }
 ],
 "key": {
   "type": "avro",
   "schema": "{\"namespace\": \"com.eventstreams.samples\",\"type\": \"record\",\"name\": \"recordKeyName\",\"fields\": [{\"name\": \"keyName\", \"type\": \"string\"}]}",
   "schema_name_strategy": "topicrecord",
   "data": "{\"keyName\": \"sampleKeyName\"}"
 },
 "value": {
   "type": "avro",
   "schema": "{\"namespace\": \"com.eventstreams.samples\",\"type\": \"record\",\"name\": \"recordValueName\",\"fields\": [{\"name\": \"valueName\", \"type\": \"string\"}]}",
   "schema_name_strategy": "topicrecord",
   "data": "{\"valueName\": \"sampleValueName\"}"
 }
}' "$kafka_http_url/v2/topics/test-topic/records?serializer=confluent

Response

Response Body

ProduceResponse

Object returned on successful request.

Status Code

  • 200

    Message successfully sent to IBM Event Streams.

  • 400

    Not a valid request.

  • 401

    Not authorized to perform the requested operation on this resource due to invalid token.

  • 403

    Failed due to insufficient permissions, check if the provided token has required access role.

  • 404

    The resource could not be found.

  • 408

    The request timed out before the operation completed.

  • 413

    The payload exceeded the maximum size allowed (64kB).

  • 415

    The request contains content type header of unsupported type.

  • 429

    Failed due to too many consecutive requests.

  • 500

    The request failed due to an internal server error.

  • 503

    The request failed due to the requested server being unavailable.

Example responses
  • {
  "partition_id": 0,
  "topic_name": "test-topic",
  "offset": 12,
  "timestamp": "2022-01-24T06:43:39.079206553Z",
  "key": {
    "type": "avro",
    "schema_id": 30,
    "schema_artifact_id": "test-topic-com.eventstreams.samples.recordKeyName-key"
  },
  "value": {
    "type": "avro",
    "schema_id": 29,
    "schema_artifact_id": "test-topic-com.eventstreams.samples.recordValueName-value"
  }
}