Getting started with subscriptions

Subscriptions for apps and app scaling

Applications can subscribe to multiple event producers, but only one app can receive events from each subscription. Note that subscriptions can affect how an application scales. For example, if you expect your app to receive many events at the same time and processing each event takes several minutes, then you might need a higher maximum scale value than if each event can be quickly processed. For more information, see Configuring application scaling.

All events that are delivered to applications are received as HTTP messages. Events contain certain HTTP headers that help you to quickly determine key bits of information about the events without looking at the body (business logic) of the event. For more information, see Example HTTP headers for an IBM Cloud Object Storage event that is sent to an application.

Subscriptions for jobs and job run limitations

Subscriptions can affect how many jobs are started. For example, if your job subscribes to delete changes on an Object Storage bucket and that bucket is deleted, a job is run for each object that was in that bucket and you can quickly reach your limitation of 100 job runs. Additionally, you need to consider the runtime for each job run that is triggered by an event. For example, if your event producer triggers 10 or more events per second and each job runs for about 20 seconds, your job run limit of 100 is reached in about 10 seconds and any subsequent job runs are lost until the previously started job runs are completed. Choose a job as an event subscriber destination only if the number of incoming events is generally low and the peak number of events expected in a specific timeframe is low enough to keep the number of running jobs below the quota limit. For more information, see Limits and quotas for Code Engine.

After 10 minutes, job runs that are created by subscriptions are deleted. For more information, see Where is my job run?.

All events that are delivered to jobs are received as environment variables. For more information, see Example environment variables for an IBM Cloud Object Storage event that is sent to a job.

Eventing metadata

Events that are managed by Code Engine when you create a subscription are modified so that they adhere to the CloudEvents specification. This specification defines a set of common attributes that can be included with each event to provide a common set of metadata. By looking at the metadata, you can quickly understand key pieces of the message without parsing and understanding the entirety of the event payload. For example, each event that is delivered to an application includes an HTTP header that is called ce-type, which indicates the semantic meaning (or "reason") for the event. An event from a database might include a ce-type value of com.example.row.deleted, indicating that the event was generated because a row was deleted in the database.

The following table lists some key common attributes. Each attribute indicates whether it is a required attribute on the incoming event or whether it is optional.

For more information about the complete list of attributes, see the CloudEvents specification.

In Code Engine, when events are delivered to applications, the CloudEvent attributes appear as HTTP headers, prefixed with ce-. When events are delivered to batch jobs, the attributes appear as environment variables, prefixed with CE_ and the entire variable name is in uppercase.

ce-id: 3fb2c04e-a660-4640-8899-b82efb8169b6
ce-source: https://cloud.ibm.com/catalog/services/cloud-object-storage/mybucket
ce-specversion: 1.0
ce-subject: object-69-144
ce-time: 2021-08-17T20:22:02.917Z
ce-type: com.ibm.cloud.cos.document.delete

CE_DATA={"bucket":"mybucket","endpoint":"","key":"Notes.rtf","notification":{"bucket_name":"mybucket","content_type":"text/rtf","event_type":"Object:Delete","format":"2.0","object_length":"4642","object_name":"Notes.rtf","request_id":"b59727ee-9c4e-446a-9261-5616f6d1283b","request_time":"2021-04-13T20:10:37.631Z"},"operation":"Object:Delete"}  
CE_ID=b59727ee-9c4e-446a-9261-5616f6d1283b  
CE_SOURCE=https://cloud.ibm.com/catalog/services/cloud-object-storage/mybucket  
CE_SPECVERSION=1.0  
CE_TIME=2021-08-17T20:22:02.917Z  
CE_TYPE=com.ibm.cloud.cos.document.delete  

What happens when I create a subscription?

By default, the subscription cron create, subscription cos create and the subscription kafka create commands first check to see whether the destination application or job exists. If the destination check fails because the application or job does not exist in your project, the commands for the subscription create returns an error. If you want to create a subscription without first creating the application, use the --force option. By using the --force option, the command bypasses the destination check. Note that the Ready field of the subscription shows false until the destination application or job is created. Then, the subscription moves to a Ready: true state automatically.

After the subscription is created, the subscription is repeatedly polled for status to verify its readiness. This polling lasts for 15 seconds by default before it times out. You can change the amount of time before the command times out by using the --wait-timeout option. You can also bypass the status polling by setting the --no-wait option to false.

You can display the status of your subscription by using the subscription cron get, subscription cos get or the subscription kafka get CLI commands.