Working with alerts

Overview

In IBM Cloud Logs, you configure alerts that define the triggering conditions on filtered data on which you want to be notified. In IBM Cloud Event Notifications, you configure the destinations where you want to be notified for an alert and the conditions that define to which destination an event is routed. You can route events to one or more destinations based on conditions that you configure. You can notify to a service destination such as a webhook or PagerDuty, or to a human destination such as email or slack.

The following image shows different destinations where an event can be routed when an alert that you configure in IBM Cloud Logs is triggered:

To send events to Event Notifications, you must connect your IBM Cloud Logs instance to Event Notifications by configuring an outbound integration. For more information, see Configuring an outbound integration to connect IBM Cloud Logs with IBM Cloud Event Notifications.

Events that are generated by an instance of the IBM Cloud Logs service can be forwarded to an Event Notifications service instance that is available in the same account.

• In an IBM Cloud Logs instance, you can only configure an outbound integration between the IBM Cloud Logs instance and one Event Notifications service instance.
• Event Notifications is available in a subset of regions where the IBM Cloud Logs service is available. For more information, see Event Notifications regions.
• You can use the same Event Notifications instance in your IBM Cloud account for all of your IBM Cloud Logs instances.
• If you have multiple Event Notifications instances in the account, consider using an Event Notifications instance that is closer to your IBM Cloud Logs instance.

A service to service authorization is used in the IBM Cloud account to authorize the IBM Cloud Logs service to send an event to the Event Notifications service.

When an event of interest takes place in your IBM Cloud Logs instance, IBM Cloud Logs communicates with a connected Event Notifications instance to forward a notification to a supported destination.

When you configure alerts in an IBM Cloud Logs instance, consider the following information:

• You can define different types of alerts such as standard alerts, flow alerts, and new value alerts. For information on the alert types that are supported, see Alerts.
• You can use the Incidents page to manage alerts that are triggered. For more information, see Managing triggered alerts.
• Alerts are triggered through the IBM Cloud Event Notifications service. You configure the notification channels and conditions that trigger the alert in the IBM Cloud Event Notifications service.

The following figure shows the high level view of an IBM Cloud Logs instance and the IBM Cloud Event Notifications service that you might configure:

Configuring alerts end to end

To configure alerting in IBM Cloud Logs, complete the following steps:

1. In IBM Cloud Logs, alerts are triggered through the IBM Cloud Event Notifications service. If you do not currently use the service and have alerts configured in your IBM Cloud Activity Tracker instances or your IBM Log Analysis instances, you must provision an instance of the IBM Cloud Activity Tracker service. For more information, see Configuring an outbound integration for IBM Cloud Logs.

2. In IAM, define a service to service authorization between the IBM Cloud Logs instance and the IBM Cloud Event Notifications instance. For more information, see Creating a S2S authorization to work with the IBM Cloud Event Notifications service.

3. Configure an outbound integration in your IBM Cloud Logs instance to connect it with the IBM Cloud Event Notifications instance. For more information, see Configuring the integration with the IBM Cloud Event Notifications service.

   This task creates a source definition in the IBM Cloud Event Notifications instance, and an integration configuration in the IBM Cloud Logs instance.

4. Define your alerts in the IBM Cloud Logs instance. Select the outbound integration through which you want to notify when the alert is triggered. For more information, see Configuring alerts in IBM Cloud Logs.

5. Configure the IBM Cloud Event Notifications instance to route event notifications when an alert is triggered in IBM Cloud Logs to your target destinations.

   • Define 1 or more topics.

     A topic defines the alert conditions that you want to group together.

     For example, if you have multiple alert definitions in your IBM Cloud Logs instance that notify through the same slack channel, you can configure these alerts within the same topic.

     Another example, if you have multiple alert definitions in your IBM Cloud Logs instance that notify through different slack channels, you must configure as many topics as slack channels you use, and include in a topic the alerts that notify through the same slack channel.

   • Define 1 or more destinations.

     A destination defines a notification channel that you can use to notify when an alert is triggered.

     For more information on destinations, see Supported destinations.

   • Define 1 or more subscriptions.

     A subscription links 1 topic with 1 destination.

     You must add subscriptions to define the alerts configured in a topic are the ones notified through the destination selected in the subcription configuration.

     A subscription correlates one topic with a notification channel.

     You can have multiple subscriptions with the same topic so you can alert through multiple destination channels.

Configuring alerts in IBM Cloud Logs

When you configure an alert in IBM Cloud Logs, you define per alert the following information:

• The alert severity
• Labels that you can use to easily filter by alert type. You define labels as key:value pairs such as env:prod.
• The alert type. For more information, see Alert types.
• Filter conditions that define the data that is analyzed at ingestion. You can define a search query, one or more applications, one or more subsystems, and one or more log severity values.
• The condition that defines when an alert is triggered.
• The outbound integration that you want to use for notifications.
• A schedule when the alert is active.
• What notificaton content you include.

For more information on configuring alerts, see Configuring an alert.

Configuring alerts in IBM Cloud Event Notifications

After you enable notifications for IBM Cloud Logs, you must create topics and subscriptions in Event Notifications so that alerts can be forwarded and delivered to your selected destinations.

When you configure an alert in IBM Cloud Event Notifications, you define the following components:

• A topic that defines the alerts that are monitored from the Event Notifications instance.
• Destinations that define the notification channels where events can be routed.
• Subscriptions that define the alerts that are notified through a destination channel.

For more information, see:

• Configuring IBM Cloud Event Notifications resources.