IBM Cloud Docs
DevOps Insights CLI

DevOps Insights CLI

The IBM Cloud® DevOps Insights CLI provides a set of commands that you can use to integrate your build with DevOps Insights. You must use two different types of commands: CLI usage commands and CLI commands to integrate with DevOps Insights.

Before you begin

  • Install the IBM Cloud CLI. See Download IBM Cloud CLI for instructions.

  • Add the IBM Cloud CLI plug-in. Run the following command:

ibmcloud plugin install doi
  • Make sure you can access a toolchain with the DevOps Insights tool that is configured for that toolchain. For more information about toolchains, see Creating a toolchain from an app.

  • Specify the toolchain ID by using one of the following methods:

    • Specify the toolchain ID as a CLI parameter to the command.
    • Set the TOOLCHAIN_ID environment variable.
    • Your IBM Cloud® Continuous Delivery pipeline might automatically set the PIPELINE_TOOLCHAIN_ID environment variable.

    The CLI needs the value of the toolchain ID. The value of toolchain ID that is specified in the CLI parameter overrides the value of environment variable.

    The toolchain ID is found in the toolchain URL that is shown in the browser. If you're using IBM® Continuous Delivery Pipeline for IBM Cloud®, you can set the toolchain ID to send your build data to a different toolchain. For more information, see Aggregating data from multiple sources into a single toolchain.

Login

Use this command to log in to IBM Cloud. The API_KEY must have access to the toolchain.

ibmcloud login --apikey API_KEY

CLI usage commands

DevOps Insights help

The following command displays the list of DevOps Insights commands:

 ibmcloud doi --help

DevOps Insights command help

The following command displays the details of the flags that are required for a command:

 ibmcloud doi <command> --help

You can pass a --region parameter to any of the commands. By setting the value of this parameter to the ibmcloud region of the toolchain, the CLI does not need to determine which region the toolchain is in, making it more efficient and reliable. This parameter is optional for compatibility with earlier versions.

Commands to integrate with DevOps Insights

When you use the CLI for a build, you must publish a build record.

The value of the logicalappname and buildnumber parameters that are passed to the CLI must remain the same across all of the command invocations.

Publishing a build record

The following command publishes a build record to DevOps Insights:

 ibmcloud doi buildrecord-publish --branch BRANCH --repositoryurl REPOSITORYURL --commitid COMMITID --status STATUS --logicalappname LOGICALAPPNAME --buildnumber BUILDNUMBER --toolchainid TOOLCHAINID [--joburl JOBURL] [--region REGION]

The following are the command options for publishing a build record.

Table 1. Command options for publishing a build record
Command Options Required or Optional Description
-B, --branch Required The repository branch that the build is being performed.
-R, --repositoryurl Required The URL of the Git repository.
-C, --commitid Required The Git commit ID.
-S, --status Required The build status. Acceptable values: pass and fail.
-L, --logicalappname Required Name of the application.
-N, --buildnumber Required Any string that identifies the build.
-I, --toolchainid Required If the TOOLCHAIN_ID environment variable is set, this flag is optional. If both the environment variable and the flag are provided, the value of the flag overrides the value of the environment variable.
-J, --joburl Optional The URL to the job's build logs that is automatically set by the CLI in the IBM® Continuous Delivery Pipeline for IBM Cloud®.
--region Optional The ibmcloud region of the toolchain.

Example

ibmcloud doi buildrecord-publish  -B master -R "https://github.com/oic/dlms.git" -C dff7884b9168168d91cb9e5aec78e93db0fa80d9 -S pass -L testapp -N master:199 -I b531487c-9c22-4f3b-9d20-5be408d57891 --region eu-gb
or
ibmcloud doi buildrecord-publish  --branch master --repositoryurl "https://github.com/oic/dlms.git" --commitid dff7884b9168168d91cb9e5aec78e93db0fa80d9 --status pass --logicalappname testapp --buildnumber master:199 --toolchainid b531487c-9c22-4f3b-9d20-5be408d57891

Publishing a test record

The following command publishes a test record to DevOps Insights:

 ibmcloud doi testrecord-publish --filelocation FILELOCATION --type TYPE --logicalappname LOGICALAPPNAME --buildnumber BUILDNUMBER --toolchainid TOOLCHAINID [--drilldownurl DRILLDOWNURL] [--env ENV] [--sqtoken SONARQUBE_TOKEN] [--tags TAGS] [--region REGION]

The following are the command options for publishing test records.

Table 2. Command options for publishing a build record
Command Options Required or Optional Description
-F, --filelocation Required Location of the results you want to upload. It can be a single file, entire directory, or several files that match a wildcard expression.
-T, --type Required The type of test results that you want to upload.
-L, --logicalappname Required Name of the application.
-N, --buildnumber Required Any string that identifies the build.
-I, --toolchainid Required If the TOOLCHAIN_ID environment variable is set, this flag is optional. If both the environment variable and the flag are provided, the value of the flag overrides the value of the environment variable.
-U, --drilldownurl Optional A URL where more information about the test results can be found. If this URL is invalid, the option is ignored.
-E, --env Optional The environment name to associate with the test results. This option is ignored for unit tests, code coverage tests, and static security scans.
-K, --sqtoken Optional This command is a SonarQube token. Valid only if the type specified is SonarQube. Used to pull more information from the SonarQube server.
--tags Optional Specify a comma-separated list of tags to associate with this test result.
--region Optional The ibmcloud region of the toolchain.

Example

ibmcloud doi testrecord-publish -F "tests/fvt/*.json" -T fvt -L testapp -N master:199 -I b531487c-9c22-4f3b-9d20-5be408d57891 --tags "CC,app1"
or
ibmcloud doi testrecord-publish --filelocation "tests/fvt/*.json" --type fvt --logicalappname testapp --buildnumber master:199 --toolchainid b531487c-9c22-4f3b-9d20-5be408d57891 --region ca-tor

The following test types are supported:

Table 3. Test record types
Type Description
unittest Unit test results
fvt Functional verification test (FVT) results
code Code coverage results
sonarqube SonarQube scan results
vulnerabilityadvisor Vulnerability Advisor results from IBM Vulnerability Advisor on Cloud
cratf Terraform report generated by Code Risk Analyzer
crabom Bill of Materials (BOM) report generated by Code Risk Analyzer
cradeploy Deployment report generated by Code Risk Analyzer
cracve Vulnerability report generated by Code Risk Analyzer
zapscan OWASP Zed Attack Proxy (ZAP) scan reports

IBM Application Security on Cloud 1.0.0 is no longer published (staticsecurityscan and dynamicsecurityscan test types). All IBM Application Security on Cloud 1.0.0 support is provided by HCL. For more information, see the HCL AppScan documentation.

Publishing a deployment record

The following command publishes a deployment record to DevOps Insights:

 ibmcloud doi deployrecord-publish --env ENV --status STATUS --logicalappname LOGICALAPPNAME --buildnumber BUILDNUMBER --toolchainid TOOLCHAINID [--joburl JOBURL] [--appurl APPURL] [--region REGION]
Table 4. Command options for publishing a deployment record
Command Options Required or Optional Description
-E, --env Required The environment where the pipeline job deployed the app.
-S, --status Required The deployment status. This value must be either pass or fail.
-L, --logicalappname Required Name of the application.
-N, --buildnumber Required Any string that identifies the build.
-I, --toolchainid Required If the TOOLCHAIN_ID environment variable is set, this flag is optional. If both the environment variable and the flag are provided, the value of the flag overrides the value of the environment variable.
-A, --appurl Optional The URL where the deployed app is running.
-J, --joburl Optional The URL to the job's build logs automatically set by the CLI in the IBM® Continuous Delivery Pipeline for IBM Cloud®.
--region Optional The ibmcloud region of the toolchain.

Example

ibmcloud doi deployrecord-publish -E "staging" -S pass -L testapp -N master:199 -I b531487c-9c22-4f3b-9d20-5be408d57891 --region au-syd
or
ibmcloud doi deployrecord-publish --env "staging" --status pass --logicalappname testapp --buildnumber master:199 --toolchainid b531487c-9c22-4f3b-9d20-5be408d57891

Evaluating gates

The following command evaluates a DevOps Insights gate:

 ibmcloud doi gate-evaluate --policy POLICY --logicalappname LOGICALAPPNAME --buildnumber BUILDNUMBER --toolchainid TOOLCHAINID [--forcedecision] [--ruletype RULETYPE] [--region REGION]

The following are command options for evaluating gates:

Table 5. Command options for evaluating gates
Command Options Required or Optional Description
-P, --policy Required The name of the policy that the gate uses to make its decision.
-L, --logicalappname Required Name of the application.
-N, --buildnumber Required Any string that identifies the build.
-I, --toolchainid Required If the TOOLCHAIN_ID environment variable is set, this flag is optional. If both the environment variable and the flag are provided, the value of the flag overrides the value of the environment variable.
-D, --forcedecision Optional Set the value to true to exit with an error code if the policy evaluation fails. The value defaults to false if this option isn't specified.
-E, --ruletype Optional A rule type to consider. If you include this option, only rules of this type are considered in the decision-making process.
--region Optional The ibmcloud region of the toolchain.

Example

ibmcloud doi gate-evaluate -P "policyname" -D true -L testapp -N master:199 -I b531487c-9c22-4f3b-9d20-5be408d57891 --region br-sao
or
ibmcloud doi gate-evaluate --policy "policyname" --forcedecision true --logicalappname testapp --buildnumber master:199 --toolchainid b531487c-9c22-4f3b-9d20-5be408d57891

Updating custom data sets and policies

The following command creates and updates custom data sets and policies for a toolchain:

 ibmcloud doi policies-update --file FILELOCATION --toolchainid TOOLCHAINID [--dryrun] [--region REGION]

The following are command options for updating custom data sets and policies:

Table 6. Command options for updating custom data sets and policies
Command Options Required or Optional Description
-F, --file Required The location of the JSON file that contains the list of custom data sets and policies to add or update. Both absolute and relative paths are accepted.
-I, --toolchainid Required If the TOOLCHAIN_ID environment variable is set, this flag is optional. If both the environment variable and the flag are provided, the value of the flag overrides the value of the environment variable.
-D, --dryrun Optional The option to simulate only the changes, with no updates.
--region Optional The ibmcloud region of the toolchain.

Example

ibmcloud doi policies-update -F "policies/policy.json" -I b531487c-9c22-4f3b-9d20-5be408d57891 --region jp-tok
or
ibmcloud doi policies-update --file "policies/policy.json" --toolchainid b531487c-9c22-4f3b-9d20-5be408d57891

JSON file structure for the updatepolicies command

A valid JSON file structure contains two fields:

{
      "custom_datasets": [],
      "policies": []
}
  • You can specify any number of policies (and custom data sets) for the array.
  • If the specified policy (and custom data set) already exists for a toolchain, the policy is updated or created.
  • Either the custom_datasets or policies array can be empty, or both can be empty.
  • The only valid values for a type_of_test custom data set are test and code.
  • If a custom data set exists for a toolchain, it can be used in the policy rules that are defined within the JSON file. You are not always required to define the custom data set within the JSON file.
  • The sample JSON file that is provided for the policies-update command lists all of the possible rule types that you can specify in a policy. All of the fields in those rules are required.
  • Use only one rule per data set.
  • The name field within a rule is optional.

Sample JSON file for the policies-update command

This sample JSON file contains two custom data sets and two policies. The first policy, name: "Orders", contains all of the rule types that you can use within a policy.

{
  "custom_datasets": [
 .  {
      "lifecycle_stage": "integrationtest",
      "type_of_test": "test",
      "label": "Integration Test"
    },
    {
      "lifecycle_stage": "covtest",
      "type_of_test": "code",
      "label": "Coverage Test"
    }
  ],
  "policies": [
    {
      "name": "Orders",
      "description": "Composite Policy.",
      "rules": [
        {
          "name": "rule1",
 .        "description": "Unit Test Rule with regression",
          "stage": "unittest",
          "percentPass": 100,
          "criticalTests": [
            "Get Weather with incomplete zip code"
          ],
          "regressionCheck": true
        },
        {
          "name": "rule2",
          "description": "Unit Test Rule without regression",
          "stage": "integrationtest",
          "percentPass": 98,
          "criticalTests": [
            "'Get Weather with incomplete zip code'"
          ],
        },
        {
          "name": "rule3",
          "description": "Functional test Rule",
          "stage": "fvt",
          "percentPass": 98,
          "criticalTests": [
            "'Get Weather with incomplete zip code'"
          ],
        },
        {
          "name": "rule4",
          "description": "Code Coverage rule",
          "stage": "code",
          "codeCoverage": 98,
        },
        {
          "name": "rule5",
          "description": "Custom dataset rule",
          "stage": "covtest",
          "codeCoverage": 60,
        },
        {
          "name": "rule6",
          "description": "Static Security Scan rule",
          "stage": "staticsecurityscan",
          "highSeverity": 40,
          "mediumSeverity": 5,
          "lowSeverity": 9
        },
        {
          "name": "rule7",
         "description": "Dynamic Security Scan rule",
          "stage": "dynamicsecurityscan",
          "highSeverity": 40,
          "mediumSeverity": 5,
          "lowSeverity": 9
        },
        {
          "name": "rule8",
          "description": "Sonarqube rule",
          "stage": "sonarqube"
        },
        {
          "name": "rule9",
          "description": "Vulnerability rule",
          "stage": "vulnerabilityadvisor"
        }
      ]
    },
    {
      "name": "UI",
      "description": "Policy to check Unit Test.",
      "rules": [
        {
          "name": "Unit Test Rule",
          "description": "Unit Test Rule",
          "stage": "integrationtest",
          "percentPass": 100,
          "criticalTests": []
        }
      ]
    }
  ]
}

FAQs

Get answers to frequently asked questions about using the DevOps Insights CLI.

Why does the CLI fail with the "You do not have access to the toolchain" message?

The API_KEY environment variable that is used to log in to IBM Cloud must be able to access the toolchain. Also, verify that you added the DevOps Insights tool integration to your toolchain.

The CLI successfully ran, why doesn't the data show up on the dashboard?

Make sure that the value of the logicalappname and buildnumber parameters that are passed to the CLI are the same across all of the stages for the build. Also, verify that a build record is uploaded for the build. The data for test records that are uploaded for a specific build does not appear on the dashboard without a build record.

How can I determine why the CLI failed?

Before you call the DevOps Insights CLI, set the IBMCLOUD_TRACE environment variable to true to turn on the debug log.

    export IBMCLOUD_TRACE=true

Observe the API calls and the responses that are shown in the log to determine the exact reason for failure.