Publishing test results (deprecated)

You can publish test results by using idra (deprecated) to integrate your Delivery Pipeline with DevOps Insights. Test results inform IBM Cloud® DevOps Insights about the tests that are run during the build process. These tests must generate some kind of JSON or XML-based file. The generated test file is published as a test record. Along with the test file, this record contains the application name, build number, types of test, and other fields. The application name and build ID in the test record must match the ones that are used in the build record for a specific build. After you run a test in a pipeline, you can upload the results to DevOps Insights.

Before you begin

Before you publish test results, you must publish your build record. For more information, see publishing a build record by using idra (deprecated).

Publishing test results

Test results must provide data in one of these supported formats:

Test type and format
Test Type	Supported Formats
Functional verification test	Mocha, xUnit, Karma/Mocha
Unit test	Mocha, xUnit, Karma/Mocha
Code coverage	Cobertura, lcov, JaCoCo
SonarQube	Scan data that is provided by SonarQube scans
Static AppScan	Static App Scans that are provided by IBM Application Security on Cloud
Dynamic AppScan	Dynamic App Scans that are provided by IBM Application Security on Cloud
Vulnerability Advisor	Vulnerability Advisor results from IBM Vulnerability Advisor on Cloud

The following example script uploads FVT test results to DevOps Insights:

# Run tests and generate a test results file here.
# Add user api key to stage environment variable as a secured property
export LOGICAL_APP_NAME="SampleApp"
export BUILD_PREFIX="master"

# node 4.x or above is needed
export PATH=/opt/IBM/node-v4.2/bin:$PATH
npm install -g grunt-idra3

idra --publishtestresult --filelocation=fvttest.json --type=fvt

In the example script, the idra command with the --publishtestresult flag specifies that the script uploads results. The --filelocation flag indicates the location of the test results file relative to the root directory of the job. The --type flag indicates the type of test result.

The following example script runs tests and then uploads Mocha results to DevOps Insights:

# Add user api key to stage environment variable as a secured property
export LOGICAL_APP_NAME="SampleApp"
export BUILD_PREFIX="master"

# Run tests and generate a test results file here.
istanbul cover --report lcov --report json-summary _mocha -- -R xunit -O output=./test/results/mocha.xml 'test/**/*.js'

# node 4.x or above is needed
export PATH=/opt/IBM/node-v4.2/bin:$PATH
npm install -g grunt-idra3

idra --publishtestresult --filelocation=./test/results/mocha.xml --type=unittest
idra --publishtestresult --filelocation=./coverage/coverage-summary.json --type=code

The idra command supports the following type values:

idra command types
Type	Description
`unittest`	Unit test results
`fvt`	Functional verification test results
`code`	Code coverage results
`sonarqube`	SonarQube scan results
`staticsecurityscan`	Static security scan results from IBM Application Security on Cloud
`dynamicsecurityscan`	Dynamic security scan results from IBM Application Security on Cloud
`vulnerabilityadvisor`	Vulnerability Advisor results from IBM Vulnerability Advisor on Cloud

To learn more about the idra command, see the grunt-idra3 package's page on npm.

Viewing test results

When your pipeline runs, it publishes the test result data to DevOps Insights. To view the test result data in DevOps Insights, go to the Quality Dashboard page. For more information, see DevOps data aggregation.

Next steps

Learn how to evaluate gates by using idra (deprecated).