Changing the alert evaluation window
Lags in your data pipeline can cause delays in log and metric ingestion by IBM® Cloud Logs, potentially leading to false alerts. While alert conditions are evaluated in real-time, delayed data that arrives later can retroactively affect whether those conditions are met.
When an alert is to be triggered when a user-defined threshold is exceeded, missing data might delay the alert or lead to an incorrect resolution. Conversely, when an alert is to be triggered when less-than a user-defined threshold, incomplete data can cause a false-positive alert or delayed alert resolution.
By configuring a delay in the alert evaluation, you can mitigate timing issues by shifting an alert's evaluation time frame backward by a specific amount of time. This adjustment can ensure that alert conditions are evaluated against a complete dataset, accounting for late-arriving logs or metrics.
By delaying the alert evaluation, you can reduce the risk of false positives or negatives that are caused by real-time data fluctuations and improve the accuracy and reliability of your alerts.
You can configure an alert evalution delay for the following alert types:
Configuring an alert evaluation delay
To configure an evaluation delay for an alert:
-
Create a new alert or edit an existing alert.
-
Under Conditions click Advanced settings.
-
Select Delay alert evaluation and select the number of seconds to delay the evaluation.
The delay can be a maximum of 3 hours (10800 seconds).