Publishing test results (deprecated)
You can publish test results by using idra (deprecated) to integrate your Delivery Pipeline with DevOps Insights. Test results inform IBM Cloud® DevOps Insights about the tests that are run during the build process. These tests must generate some kind of JSON or XML-based file. The generated test file is published as a test record. Along with the test file, this record contains the application name, build number, types of test, and other fields. The application name and build ID in the test record must match the ones that are used in the build record for a specific build. After you run a test in a pipeline, you can upload the results to DevOps Insights.
Before you begin
Before you publish test results, you must publish your build record. For more information, see publishing a build record by using idra (deprecated).
Publishing test results
Test results must provide data in one of these supported formats:
Test Type | Supported Formats |
---|---|
Functional verification test | Mocha, xUnit, Karma/Mocha |
Unit test | Mocha, xUnit, Karma/Mocha |
Code coverage | Cobertura, lcov, JaCoCo |
SonarQube | Scan data that is provided by SonarQube scans |
Static AppScan | Static App Scans that are provided by IBM Application Security on Cloud |
Dynamic AppScan | Dynamic App Scans that are provided by IBM Application Security on Cloud |
Vulnerability Advisor | Vulnerability Advisor results from IBM Vulnerability Advisor on Cloud |
The following example script uploads FVT test results to DevOps Insights:
# Run tests and generate a test results file here.
# Add user api key to stage environment variable as a secured property
export LOGICAL_APP_NAME="SampleApp"
export BUILD_PREFIX="master"
# node 4.x or above is needed
export PATH=/opt/IBM/node-v4.2/bin:$PATH
npm install -g grunt-idra3
idra --publishtestresult --filelocation=fvttest.json --type=fvt
In the example script, the idra
command with the --publishtestresult
flag specifies that the script uploads results. The --filelocation
flag indicates the location of the test results file relative to the
root directory of the job. The --type
flag indicates the type of test result.
The following example script runs tests and then uploads Mocha results to DevOps Insights:
# Add user api key to stage environment variable as a secured property
export LOGICAL_APP_NAME="SampleApp"
export BUILD_PREFIX="master"
# Run tests and generate a test results file here.
istanbul cover --report lcov --report json-summary _mocha -- -R xunit -O output=./test/results/mocha.xml 'test/**/*.js'
# node 4.x or above is needed
export PATH=/opt/IBM/node-v4.2/bin:$PATH
npm install -g grunt-idra3
idra --publishtestresult --filelocation=./test/results/mocha.xml --type=unittest
idra --publishtestresult --filelocation=./coverage/coverage-summary.json --type=code
The idra
command supports the following type
values:
Type | Description |
---|---|
unittest |
Unit test results |
fvt |
Functional verification test results |
code |
Code coverage results |
sonarqube |
SonarQube scan results |
staticsecurityscan |
Static security scan results from IBM Application Security on Cloud |
dynamicsecurityscan |
Dynamic security scan results from IBM Application Security on Cloud |
vulnerabilityadvisor |
Vulnerability Advisor results from IBM Vulnerability Advisor on Cloud |
To learn more about the idra
command, see the grunt-idra3 package's page on npm.
Viewing test results
When your pipeline runs, it publishes the test result data to DevOps Insights. To view the test result data in DevOps Insights, go to the Quality Dashboard page. For more information, see DevOps data aggregation.
Next steps
Learn how to evaluate gates by using idra (deprecated).