IBM Cloud Docs
Aggregating data from multiple sources into a single toolchain

Aggregating data from multiple sources into a single toolchain

By default, the data that is generated by your pipeline or other tools is aggregated to the toolchain that the tools are contained in. With IBM Cloud® DevOps Insights, you can aggregate quality data that is generated by IBM Cloud® Continuous Delivery from multiple sources into a single toolchain.

Diagram of multiple sources funneled into a single toolchain
Figure 1. Aggregated data in a central toolchain with specific policies

Before you begin

  1. Configure your Continuous Delivery pipeline to send build, test, and deployment records to DevOps Insights. For more information about build, test, and deployment records, see Publishing a build record, Publishing a deployment record, and Publishing test results.

  2. Define gate policies in the single toolchain that you want to use to aggregate gate reports.

Identifying your toolchain ID

You need your toolchain ID to integrate Jenkins, Travis CI, and other continuous integration and continuous delivery (CI/CD) tools with IBM Cloud® DevOps Insights. You can also use your toolchain ID to configure the aggregation of data of multiple sources into a single toolchain.

For more information about integrating Jenkins or other continuous integration and continuous delivery tools with DevOps Insights, see Integrating Jenkins with DevOps Insights or Integrating other continuous integration and continuous delivery tools with DevOps Insights.

Finding your toolchain ID

Your toolchain ID is contained within the URL. To find your toolchain ID, complete the following steps:

  1. From the IBM Cloud® console, click the menu icon hamburger icon, and select DevOps.
  2. On the Toolchains page, select your toolchain.
  3. From your toolchain's Overview page, on the IBM Cloud tools card, click DevOps Insights.
  4. In the url, copy the everything in the string after ?toolchainId=. This string is 36 characters in length, which is composed of alphanumeric characters and dashes.

If you want to find the toolchain ID for Jenkins, select Jenkins from the toolchain instead of DevOps Insights. You can also use the same method to find the toolchain ID in the URL.

For example, https://cloud.ibm.com/devops/insights/overview?toolchainid=99c643c3-23d9-4030-9b1f-3aadfd7f9d09. The toolchain ID is 99c643c3-23d9-4030-9b1f-3aadfd7f9d09.

The API_KEY used in the script to log in to IBM Cloud must have access to the new toolchain_id that is provided. For more information about creating and managing API keys, see Managing user API keys.

Publishing records to a centralized target toolchain

To publish data to a single toolchain, you must modify the scripts of your pipeline stages, but must you modify the stages through which you are publishing data. You modify the stages only when you want to publish the data in a different toolchain. Not when you want to publish data to the current toolchain that the pipeline is in.

To publish your records to a centralized toolchain, complete the following steps:

  1. Click the menu icon hamburger icon, and select Resource List.
  2. Select your toolchain.
  3. From your toolchain's Overview page, on the Delivery pipelines card, click the pipeline that you want to use.
  4. Click the Settings icon gear icon and Configure Stage.
  5. Make your changes in the script section. For each stage where you called commands to send build, test, and deployment records, set TOOLCHAIN_ID as the environment variable to the toolchain ID. Export the toolchain ID before the variables call the command.
  6. Call commands for gate decisions, or send build, test, deployment records to DevOps Insights. For more information, see Setting consistent parameter values by using the IBM Cloud® CLI.

You export TOOLCHAIN_ID as environment variable for stages of the pipeline where they are using the DevOps Insights CLI.

The following example shows how to send build records to DevOps Insights in another toolchain. You must overwrite the toolchain ID before you call the command to evaluate gate, publish build, test, or deployment records.

#!/bin/bash

#install the latest version of the DevOps Insights plugin
ibmcloud plugin install -f doi

# Login to IBMCloud if you are not already logged in.  Assumes that $API_KEY environment variable has been set as a secured property in environment variable
ibmcloud login --apikey $API_KEY --no-region

# Overwrite the default TOOLCHAIN_ID with the Toolchain ID of the DevOps Insights instance you would like to send data to
export TOOLCHAIN_ID="$DEVOPS_INSIGHTS_TOOLCHAIN_ID"

# $GIT_BRANCH, $GIT_URL, $GIT_COMMIT are environment variables made available by the Continuous Delivery pipeline
# Assumes you have set $MY_APP_NAME and $MY_BUILD_NUMBER as described earlier

ibmcloud doi buildrecord-publish --logicalappname="$MY_APP_NAME" --buildnumber="$MY_BUILD_NUMBER" --branch $GIT_BRANCH --repositoryurl $GIT_URL --commitid $GIT_COMMIT --status pass