DevOps Insights CLI
The IBM Cloud® DevOps Insights CLI provides a set of commands that you can use to integrate your build with DevOps Insights. You must use two different types of commands: CLI usage commands and CLI commands to integrate with DevOps Insights.
Before you begin
-
Install the IBM Cloud CLI. See Download IBM Cloud CLI for instructions.
-
Add the IBM Cloud CLI plug-in. Run the following command:
ibmcloud plugin install doi
-
Make sure you can access a toolchain with the DevOps Insights tool that is configured for that toolchain. For more information about toolchains, see Creating a toolchain from an app.
-
Specify the toolchain ID by using one of the following methods:
- Specify the toolchain ID as a CLI parameter to the command.
- Set the
TOOLCHAIN_ID
environment variable. - Your IBM Cloud® Continuous Delivery pipeline might automatically set the
PIPELINE_TOOLCHAIN_ID
environment variable.
The CLI needs the value of the toolchain ID. The value of toolchain ID that is specified in the CLI parameter overrides the value of environment variable.
The toolchain ID is found in the toolchain URL that is shown in the browser. If you're using IBM® Continuous Delivery Pipeline for IBM Cloud®, you can set the toolchain ID to send your build data to a different toolchain. For more information, see Aggregating data from multiple sources into a single toolchain.
Login
Use this command to log in to IBM Cloud. The API_KEY must have access to the toolchain.
ibmcloud login --apikey API_KEY
CLI usage commands
DevOps Insights help
The following command displays the list of DevOps Insights commands:
ibmcloud doi --help
DevOps Insights command help
The following command displays the details of the flags that are required for a command:
ibmcloud doi <command> --help
You can pass a --region
parameter to any of the commands. By setting the value of this parameter to the ibmcloud
region of the toolchain, the CLI does not need to determine which region the toolchain is in, making
it more efficient and reliable. This parameter is optional for compatibility with earlier versions.
Commands to integrate with DevOps Insights
When you use the CLI for a build, you must publish a build record.
The value of the logicalappname
and buildnumber
parameters that are passed to the CLI must remain the same across all of the command invocations.
Publishing a build record
The following command publishes a build record to DevOps Insights:
ibmcloud doi buildrecord-publish --branch BRANCH --repositoryurl REPOSITORYURL --commitid COMMITID --status STATUS --logicalappname LOGICALAPPNAME --buildnumber BUILDNUMBER --toolchainid TOOLCHAINID [--joburl JOBURL] [--region REGION]
The following are the command options for publishing a build record.
Command Options | Required or Optional | Description |
---|---|---|
-B , --branch |
Required | The repository branch that the build is being performed. |
-R , --repositoryurl |
Required | The URL of the Git repository. |
-C , --commitid |
Required | The Git commit ID. |
-S , --status |
Required | The build status. Acceptable values: pass and fail . |
-L , --logicalappname |
Required | Name of the application. |
-N , --buildnumber |
Required | Any string that identifies the build. |
-I , --toolchainid |
Required | If the TOOLCHAIN_ID environment variable is set, this flag is optional. If both the environment variable and the flag are provided, the value of the flag overrides the value of the environment variable. |
-J , --joburl |
Optional | The URL to the job's build logs that is automatically set by the CLI in the IBM® Continuous Delivery Pipeline for IBM Cloud®. |
--region |
Optional | The ibmcloud region of the toolchain. |
Example
ibmcloud doi buildrecord-publish -B master -R "https://github.com/oic/dlms.git" -C dff7884b9168168d91cb9e5aec78e93db0fa80d9 -S pass -L testapp -N master:199 -I b531487c-9c22-4f3b-9d20-5be408d57891 --region eu-gb
or
ibmcloud doi buildrecord-publish --branch master --repositoryurl "https://github.com/oic/dlms.git" --commitid dff7884b9168168d91cb9e5aec78e93db0fa80d9 --status pass --logicalappname testapp --buildnumber master:199 --toolchainid b531487c-9c22-4f3b-9d20-5be408d57891
Publishing a test record
The following command publishes a test record to DevOps Insights:
ibmcloud doi testrecord-publish --filelocation FILELOCATION --type TYPE --logicalappname LOGICALAPPNAME --buildnumber BUILDNUMBER --toolchainid TOOLCHAINID [--drilldownurl DRILLDOWNURL] [--env ENV] [--sqtoken SONARQUBE_TOKEN] [--tags TAGS] [--region REGION]
The following are the command options for publishing test records.
Command Options | Required or Optional | Description |
---|---|---|
-F , --filelocation |
Required | Location of the results you want to upload. It can be a single file, entire directory, or several files that match a wildcard expression. |
-T , --type |
Required | The type of test results that you want to upload. |
-L , --logicalappname |
Required | Name of the application. |
-N , --buildnumber |
Required | Any string that identifies the build. |
-I , --toolchainid |
Required | If the TOOLCHAIN_ID environment variable is set, this flag is optional. If both the environment variable and the flag are provided, the value of the flag overrides the value of the environment variable. |
-U , --drilldownurl |
Optional | A URL where more information about the test results can be found. If this URL is invalid, the option is ignored. |
-E , --env |
Optional | The environment name to associate with the test results. This option is ignored for unit tests, code coverage tests, and static security scans. |
-K , --sqtoken |
Optional | This command is a SonarQube token. Valid only if the type specified is SonarQube. Used to pull more information from the SonarQube server. |
--tags |
Optional | Specify a comma-separated list of tags to associate with this test result. |
--region |
Optional | The ibmcloud region of the toolchain. |
Example
ibmcloud doi testrecord-publish -F "tests/fvt/*.json" -T fvt -L testapp -N master:199 -I b531487c-9c22-4f3b-9d20-5be408d57891 --tags "CC,app1"
or
ibmcloud doi testrecord-publish --filelocation "tests/fvt/*.json" --type fvt --logicalappname testapp --buildnumber master:199 --toolchainid b531487c-9c22-4f3b-9d20-5be408d57891 --region ca-tor
The following test types are supported:
Type | Description |
---|---|
unittest |
Unit test results |
fvt |
Functional verification test (FVT) results |
code |
Code coverage results |
sonarqube |
SonarQube scan results |
vulnerabilityadvisor |
Vulnerability Advisor results from IBM Vulnerability Advisor on Cloud |
cratf |
Terraform report generated by Code Risk Analyzer |
crabom |
Bill of Materials (BOM) report generated by Code Risk Analyzer |
cradeploy |
Deployment report generated by Code Risk Analyzer |
cracve |
Vulnerability report generated by Code Risk Analyzer |
zapscan |
OWASP Zed Attack Proxy (ZAP) scan reports |
IBM Application Security on Cloud 1.0.0 is no longer published (staticsecurityscan
and dynamicsecurityscan
test types). All IBM Application Security on Cloud 1.0.0 support is provided by HCL. For more information,
see the HCL AppScan documentation.
Publishing a deployment record
The following command publishes a deployment record to DevOps Insights:
ibmcloud doi deployrecord-publish --env ENV --status STATUS --logicalappname LOGICALAPPNAME --buildnumber BUILDNUMBER --toolchainid TOOLCHAINID [--joburl JOBURL] [--appurl APPURL] [--region REGION]
Command Options | Required or Optional | Description |
---|---|---|
-E , --env |
Required | The environment where the pipeline job deployed the app. |
-S , --status |
Required | The deployment status. This value must be either pass or fail. |
-L , --logicalappname |
Required | Name of the application. |
-N , --buildnumber |
Required | Any string that identifies the build. |
-I , --toolchainid |
Required | If the TOOLCHAIN_ID environment variable is set, this flag is optional. If both the environment variable and the flag are provided, the value of the flag overrides the value of the environment variable. |
-A , --appurl |
Optional | The URL where the deployed app is running. |
-J , --joburl |
Optional | The URL to the job's build logs automatically set by the CLI in the IBM® Continuous Delivery Pipeline for IBM Cloud®. |
--region |
Optional | The ibmcloud region of the toolchain. |
Example
ibmcloud doi deployrecord-publish -E "staging" -S pass -L testapp -N master:199 -I b531487c-9c22-4f3b-9d20-5be408d57891 --region au-syd
or
ibmcloud doi deployrecord-publish --env "staging" --status pass --logicalappname testapp --buildnumber master:199 --toolchainid b531487c-9c22-4f3b-9d20-5be408d57891
Evaluating gates
The following command evaluates a DevOps Insights gate:
ibmcloud doi gate-evaluate --policy POLICY --logicalappname LOGICALAPPNAME --buildnumber BUILDNUMBER --toolchainid TOOLCHAINID [--forcedecision] [--ruletype RULETYPE] [--region REGION]
The following are command options for evaluating gates:
Command Options | Required or Optional | Description |
---|---|---|
-P , --policy |
Required | The name of the policy that the gate uses to make its decision. |
-L , --logicalappname |
Required | Name of the application. |
-N , --buildnumber |
Required | Any string that identifies the build. |
-I , --toolchainid |
Required | If the TOOLCHAIN_ID environment variable is set, this flag is optional. If both the environment variable and the flag are provided, the value of the flag overrides the value of the environment variable. |
-D , --forcedecision |
Optional | Set the value to true to exit with an error code if the policy evaluation fails. The value defaults to false if this option isn't specified. |
-E , --ruletype |
Optional | A rule type to consider. If you include this option, only rules of this type are considered in the decision-making process. |
--region |
Optional | The ibmcloud region of the toolchain. |
Example
ibmcloud doi gate-evaluate -P "policyname" -D true -L testapp -N master:199 -I b531487c-9c22-4f3b-9d20-5be408d57891 --region br-sao
or
ibmcloud doi gate-evaluate --policy "policyname" --forcedecision true --logicalappname testapp --buildnumber master:199 --toolchainid b531487c-9c22-4f3b-9d20-5be408d57891
Updating custom data sets and policies
The following command creates and updates custom data sets and policies for a toolchain:
ibmcloud doi policies-update --file FILELOCATION --toolchainid TOOLCHAINID [--dryrun] [--region REGION]
The following are command options for updating custom data sets and policies:
Command Options | Required or Optional | Description |
---|---|---|
-F , --file |
Required | The location of the JSON file that contains the list of custom data sets and policies to add or update. Both absolute and relative paths are accepted. |
-I , --toolchainid |
Required | If the TOOLCHAIN_ID environment variable is set, this flag is optional. If both the environment variable and the flag are provided, the value of the flag overrides the value of the environment variable. |
-D , --dryrun |
Optional | The option to simulate only the changes, with no updates. |
--region |
Optional | The ibmcloud region of the toolchain. |
Example
ibmcloud doi policies-update -F "policies/policy.json" -I b531487c-9c22-4f3b-9d20-5be408d57891 --region jp-tok
or
ibmcloud doi policies-update --file "policies/policy.json" --toolchainid b531487c-9c22-4f3b-9d20-5be408d57891
JSON file structure for the updatepolicies
command
A valid JSON file structure contains two fields:
{
"custom_datasets": [],
"policies": []
}
- You can specify any number of policies (and custom data sets) for the array.
- If the specified policy (and custom data set) already exists for a toolchain, the policy is updated or created.
- Either the
custom_datasets
orpolicies
array can be empty, or both can be empty. - The only valid values for a
type_of_test
custom data set aretest
andcode
. - If a custom data set exists for a toolchain, it can be used in the policy rules that are defined within the JSON file. You are not always required to define the custom data set within the JSON file.
- The sample JSON file that is provided for the
policies-update
command lists all of the possible rule types that you can specify in a policy. All of the fields in those rules are required. - Use only one rule per data set.
- The name field within a rule is optional.
Sample JSON file for the policies-update
command
This sample JSON file contains two custom data sets and two policies. The first policy, name: "Orders"
, contains all of the rule types that you can use within a policy.
{
"custom_datasets": [
. {
"lifecycle_stage": "integrationtest",
"type_of_test": "test",
"label": "Integration Test"
},
{
"lifecycle_stage": "covtest",
"type_of_test": "code",
"label": "Coverage Test"
}
],
"policies": [
{
"name": "Orders",
"description": "Composite Policy.",
"rules": [
{
"name": "rule1",
. "description": "Unit Test Rule with regression",
"stage": "unittest",
"percentPass": 100,
"criticalTests": [
"Get Weather with incomplete zip code"
],
"regressionCheck": true
},
{
"name": "rule2",
"description": "Unit Test Rule without regression",
"stage": "integrationtest",
"percentPass": 98,
"criticalTests": [
"'Get Weather with incomplete zip code'"
],
},
{
"name": "rule3",
"description": "Functional test Rule",
"stage": "fvt",
"percentPass": 98,
"criticalTests": [
"'Get Weather with incomplete zip code'"
],
},
{
"name": "rule4",
"description": "Code Coverage rule",
"stage": "code",
"codeCoverage": 98,
},
{
"name": "rule5",
"description": "Custom dataset rule",
"stage": "covtest",
"codeCoverage": 60,
},
{
"name": "rule6",
"description": "Static Security Scan rule",
"stage": "staticsecurityscan",
"highSeverity": 40,
"mediumSeverity": 5,
"lowSeverity": 9
},
{
"name": "rule7",
"description": "Dynamic Security Scan rule",
"stage": "dynamicsecurityscan",
"highSeverity": 40,
"mediumSeverity": 5,
"lowSeverity": 9
},
{
"name": "rule8",
"description": "Sonarqube rule",
"stage": "sonarqube"
},
{
"name": "rule9",
"description": "Vulnerability rule",
"stage": "vulnerabilityadvisor"
}
]
},
{
"name": "UI",
"description": "Policy to check Unit Test.",
"rules": [
{
"name": "Unit Test Rule",
"description": "Unit Test Rule",
"stage": "integrationtest",
"percentPass": 100,
"criticalTests": []
}
]
}
]
}
FAQs
Get answers to frequently asked questions about using the DevOps Insights CLI.
Why does the CLI fail with the "You do not have access to the toolchain" message?
The API_KEY
environment variable that is used to log in to IBM Cloud must be able to access the toolchain. Also, verify that you added the DevOps Insights tool integration to your toolchain.
The CLI successfully ran, why doesn't the data show up on the dashboard?
Make sure that the value of the logicalappname
and buildnumber
parameters that are passed to the CLI are the same across all of the stages for the build. Also, verify that a build record is uploaded for the build.
The data for test records that are uploaded for a specific build does not appear on the dashboard without a build record.
How can I determine why the CLI failed?
Before you call the DevOps Insights CLI, set the IBMCLOUD_TRACE
environment variable to true to turn on the debug log.
export IBMCLOUD_TRACE=true
Observe the API calls and the responses that are shown in the log to determine the exact reason for failure.