Monitoring metrics for Continuous Delivery

IBM Cloud® Monitoring is a third-party cloud-native, and container-intelligence management system that you can include as part of your IBM Cloud architecture. Use it to gain operational visibility into the performance and health of your applications, services, and platforms. It offers administrators, DevOps teams and developers full stack telemetry with advanced features to monitor and troubleshoot, define alerts, and design custom dashboards.

Platform metrics overview

You can configure one instance only of the IBM Cloud Monitoring service per region to collect platform metrics.

To configure the Monitoring instance, you must set the platform metrics configuration.
If a Monitoring instance in a region is already enabled to collect platform metrics, metrics from enabled-monitoring services are collected automatically and available for monitoring through this instance.
After a Monitoring instance in an account and region is enabled to collect platform metrics, it automatically collects metrics from all Continuous Delivery service instances in the same account and region. You do not need to opt in to the Continuous Delivery service metrics collection. You also cannot opt out of the Continuous Delivery service metrics collection.

To monitor platform metrics, check that the IBM Cloud Monitoring instance is provisioned in the same region where the Continuous Delivery instance is provisioned.

Enabling platform metrics from the Continuous Delivery dashboard

Complete the following steps to configure platform metrics:

Log in to IBM Cloud.
Click Resource list.
In the Services and software section, click the Continuous Delivery instance that you plan to monitor.
From the Manage page, click Actions > Add monitoring to configure platform metrics in the region of your Continuous Delivery instance.

If the menu choices include the Monitoring option, then the account and region that contain the Continuous Delivery instance is already configured for platform metrics.
Provision an instance of the IBM Cloud Monitoring service.

After you provision the Monitoring instance, the Observability page opens. To continue working with Continuous Delivery, go back to the Continuous Delivery UI.

Metrics are not visible until you set up your Tekton pipelines, configure at least one private worker pool, and trigger some Tekton pipeline triggers to generate the queue time metric.

Viewing metrics

To monitor Continuous Delivery metrics, you must launch the Monitoring web UI instance that is enabled for platform metrics in the region where your Continuous Delivery instance is available.

There are different options to launch the Monitoring web UI and monitor metrics:

Launching the Monitoring web UI from the Continuous Delivery dashboard

Complete the following steps to launch the Monitoring web UI from the Continuous Delivery dashboard:

Log in to IBM Cloud.
Click Resource list.
In the Services and software section, click the Continuous Delivery instance that you plan to monitor.
From the Manage page, click Actions > Monitoring to view the Continuous Delivery dashboard within the context of your Continuous Delivery instance.

Launching the Monitoring web UI from the Observability page

Complete the following steps to launch the Monitoring web UI from the Observability page:

Log in to IBM Cloud.
Launch the Monitoring UI.
Click Observability and select Monitoring.
From the list of instances that are available on IBM Cloud, select the Continuous Delivery instance that you want to monitor. Then, click Open Dashboard.

To monitor a Continuous Delivery instance, change the scope or make a copy of the Default dashboard.

Monitoring Continuous Delivery

Consider the following tasks when you monitor an instance of the Continuous Delivery service:

Continuous Delivery moitoring tasks
Task	Predefined Alert	What to Look For
Monitor how long delivery pipeline runs remain queued before they begin executing on a private pipeline worker pool.	Continuous Delivery Pipeline queue time	If the queue time repeatedly exceeds more than 10 seconds, the private worker pool might not have enough capacity to handle the pipeline runs being requested of it. You might need to scale up the worker pool, create a new worker pool, or reduce the number of pipeline runs.

Monitor delivery pipeline queue times

Description

The pipeline queue time tracks the elapsed time in seconds between the Continuous Delivery service receiving and queuing a request to run a pipeline, and a private worker pool dequeuing and commencing execution of the pipeline. Ideally, pipeline requests spend only a second or two on the queue before being picked up by the private worker pool. If pipeline requests remain queued for several seconds or minutes, the private worker pool might lack the capacity to handle the incoming pipeline workload. Alerting when pipeline queue times exceed a reasonable threshold, such as 10 seconds, can help draw your attention to a private worker pool that needs to be scaled up.

User action

The Continuous Delivery dashboard in Monitoring can help to identify trends in pipeline queue time. If the queue times for pipelines serviced by a private worker pool last several seconds or are trending upwards, check the configuration of the private worker pool, including the number and processing capacity of the nodes in the pool. You might lower queue times by scaling up the private worker pool, or by reducing the frequency, variety, or complexity of pipeline jobs and tasks that are dispatched to the private worker pool.

Continuous Delivery predefined dashboards

The following table outlines the pre-defined dashboards that you can use to monitor Continuous Delivery metrics:

Pre-defined dashboards
Dashboard Name	Description
`Continuous Delivery`	Default dashboard when you launch the Monitoring web UI from your service instance UI.

You cannot change the Default dashboard. To customize the dashboard, you can create a copy of it.

Metrics available by service plan

Table 3: Metrics available by plan names
Metric Name	Lite	Professional
Pipeline queue time

Predefined alerts

The following table outlines the pre-defined alerts that are available in Monitoring:

Pre-defined alerts
Alert Name	Description
Continuous Delivery high pipeline queue time	Use this alert to monitor the queue time for pipeline requests to let you know if a private worker pool is taking too long to process requests.

Continuous Delivery metrics dictionary

Pipeline queue time

The elapsed time in seconds between the Continuous Delivery service receiving and queuing a request to run a pipeline, and a private worker pool dequeuing and commencing execution of the pipeline.

Table 5: Pipeline queue time metric metadata
Metadata	Description
`Metric Name`	`ibm_toolchain_pipeline_queue_time_seconds`
`Metric Type`	`gauge`
`Value Type`	`second`
`Segment By`	`Service instance, Service instance name, Private worker pool ID`

Attributes for segmentation

Global attributes

The following attributes are available for segmenting all of the metrics listed in this topic.

Global segmentation attributes
Attribute	Attribute Name	Attribute Description	Valid Values
`Cloud Type`	`ibm_ctype`	Type of Cloud	Valid values are `public` or `local`.
`Location`	`ibm_location`	Location of the monitored resource.	You can specify a region, a data center, or `global`.
`Resource group`	`ibm_resource_group_name`	Resource group that is associated with the service instance.	Choose a resource group from the ones that are available in your account.
`Scope`	`ibm_scope`	The extent of the data samples that are considered.	You can choose the scope to be the account, an organization, or a space GUID that is associated with this metric.
`Service name`	`ibm_service_name`	Name of the service that generates this metric.	`toolchain`

Additional attributes

The following attributes are available for segmenting one or more global attributes:

Check each individual metric for the supported segmentation options.

Additional segmentation attributes
Attribute	Attribute Name	Attribute Description
`Private worker pool`	`ibm_toolchain_pipeline_worker_pool`	The unique id of a Continuous Delivery pipeline private worker pool.