Publishing test results

Test results inform DevOps Insights about the tests that are run during the build process. These tests must generate a JSON or XML-based file. The generated test file is published as a test record. Along with the test file, this record contains the application name, build number, types of test, and other fields.

Before you begin

Publish a build record.

Uploading test results

The application name and build ID in the test record must match the ones that are used in the build record for a specific build. The following example script uploads functional verification test (FVT) results to DevOps Insights:

# Run tests and generate a test results file here.

#install the DevOps Insights plugin
ibmcloud plugin install -f doi

# Login to IBMCloud if you are not already logged in.  Assumes that $API_KEY environment variable has been set as a secured property
ibmcloud login --apikey $API_KEY --no-region

ibmcloud doi testrecord-publish --logicalappname="$MY_APP_NAME" --buildnumber="$MY_BUILD_NUMBER" --filelocation fvttest.json --type fvt

In the example script, testrecord-publish specifies that the command uploads test records. The --filelocation flag indicates the location of the test results file relative to the root directory of the job. The --type flag indicates the type of test result.

The following example script runs tests and then uploads Mocha results to DevOps Insights:

# Run tests and generate a test results file here.
istanbul cover --report lcov --report json-summary _mocha -- -R xunit -O output=./test/results/mocha.xml 'test/**/*.js'

#install the DevOps Insights plugin
ibmcloud plugin install -f doi

# Login to IBMCloud if you are not already logged in.  Assumes that $API_KEY environment variable has been set as a secured property
ibmcloud login --apikey $API_KEY --no-region

#Publish test results.  Assumes that MY_APP_NAME and MY_BUILD_NUMBER environment variables are already set
ibmcloud doi testrecord-publish --logicalappname="$MY_APP_NAME" --buildnumber="$MY_BUILD_NUMBER" --filelocation /test/results/mocha.xml --type unittest
ibmcloud doi testrecord-publish --logicalappname="$MY_APP_NAME" --buildnumber="$MY_BUILD_NUMBER" --filelocation ./coverage/coverage-summary.json --type code

The command supports the following type values:

Table 1. Test record types
Type	Description
`unittest`	Unit test results
`fvt`	Functional verification test results
`code`	Code coverage results
`sonarqube`	SonarQube scan results
`staticsecurityscan`	Static security scan results from IBM Application Security on Cloud
`dynamicsecurityscan`	Dynamic security scan results from IBM Application Security on Cloud
`vulnerabilityadvisor`	Vulnerability Advisor results from IBM Vulnerability Advisor on Cloud

Test records must provide data in one of the following supported formats:

Table 2. Supported formats
Test Type	Supported Formats
Unit test	Mocha, xUnit, Karma/Mocha
Functional verification test	Mocha, xUnit, Karma/Mocha
Code coverage	Cobertura, lcov, JaCoCo
SonarQube	Scan data that is provided by SonarQube scans
Static AppScan	Static App Scans that are provided by IBM Application Security on Cloud
Dynamic AppScan	Dynamic App Scans that are provided by IBM Application Security on Cloud
Vulnerability Advisor Results	Vulnerability Advisor results from IBM Vulnerability Advisor on Cloud

Viewing test results

When your pipeline runs, it publishes the test result data to DevOps Insights. You can view the test result data on the Quality Dashboard page.

From the IBM Cloud console, click the menu icon , and select Resource List.
Select your toolchain.
From your toolchain's Overview page, on the IBM Cloud tools card, click DevOps Insights.
Click Quality Dashboard in the navigation to open the page.

For more information about the Quality Dashboard page, see DevOps data aggregation.

Next steps

Evaluate gates.