Publishing test data to DevOps Insights
Users can upload test results to IBM Cloud® DevOps Insights from any continuous integration and continuous delivery (CI/CD) tools and see all of those test results in the DevOps Insights Quality Dashboard page. A CI/CD tool refers to tools like Jenkins, Travis CI, and IBM® Continuous Delivery Pipeline for IBM Cloud® that are used to create builds, run tests, and perform deployments.
For more information about the Quality Dashboard page, see DevOps data aggregation.
Supported test data format in DevOps Insights
The following table showed a list of supported tests and formats.
| Display Label | Data Type | Supported Data Formats and File Extension |
|---|---|---|
| Unit Tests | Test Case | JUnit (JSON), xUnit (xml), Mocha (JSON), KarmaMocha (JSON) |
| Code Coverage | Code Coverage | Cobertura (xml), lcov (info), JaCoCo (xml) |
| Functional Verification Tests | Test Case | JUnit (JSON), xUnit (xml), Mocha (JSON), KarmaMocha (JSON) |
| SonarQube | SonarQube | SonarQube |
| Dynamic Security Scan | Dynamic Security Scan | IBM Cloud® Application Security (xml) |
| Static Security Scan | Static Security Scan | IBM Cloud® Application Security (xml) |
| Vulnerability Advisor | Vulnerability Advisor | Vulnerability Advisor (JSON) |
Uploading test case results
To upload unit tests, functional verification tests or custom test data, use the following command:
ibmcloud doi testrecord-publish --logicalappname "$MY_APP_NAME" --buildnumber "$MY_BUILD_NUMBER" --filelocation "./test-sample-folder/unit_test_mocha.json" --type=unittest
For a sample test result files that you can upload to DevOps Insights, see this GitHub repository with dummy data.
Uploading Code Coverage Results
To upload Code Coverage results or custom code data, use the following command:
ibmcloud doi testrecord-publish --logicalappname "$MY_APP_NAME" --buildnumber "$MY_BUILD_NUMBER" --filelocation "./test-sample-folder/code_coverage_cobertura.xml" --type=code
For a sample test result files that you can upload to DevOps Insights, see this GitHub repository with dummy data.
Uploading IBM Application Scan reports
Users can get the report from the IBM AppScan CLI tool. Scan the application from the CLI to get the results in compressed file format.
appscan.sh get_result -i $job_id -d "$APPSCAN_SCAN_NAME.zip" -t zip
Unzip the results in a folder and go to that folder, then upload a file called Report-final.xml
ibmcloud doi testrecord-publish --logicalappname "$MY_APP_NAME" --buildnumber "$MY_BUILD_NUMBER" --filelocation "./Report-final.xml" --type staticsecurityscan
Change the type to dynamicsecurityscan if the report is from a dynamic scan.
Uploading SonarQube results
To upload data from SonarQube, you must provide the SonarQube server token by using --token. Your SonarQube server is required to be accessible from your CI/CD tool.
After you run a scan by using SonarQube, you can upload SonarQube results by running this command.
ibmcloud doi testrecord-publish --logicalappname "$MY_APP_NAME" --buildnumber "$MY_BUILD_NUMBER" --filelocation ".scannerwork/report-task.txt" --type=sonarqube --token=$SONARQUBE_TOKEN
report-task.txt is a file that is generated during the SonarQube scan.
Uploading IBM Vulnerability Advisor results
To get the result of the vulnerability advisor scan from the CLI, use the following command:
ibmcloud cr va ${PIPELINE_IMAGE_URL} -o json > vulnerability_advisor.json
You can get your image's repository and tag from the Vulnerability Advisor UI or in the CLI by using ibmcloud cr image-list. DevOps Insights accepts only the JSON output. You can upload Vulnerability Advisor output file to DevOps
Insights by running this command.
ibmcloud doi testrecord-publish --logicalappname "${APP_NAME}" --buildnumber "${BUILD_NUMBER}" --filelocation "vulnerability_advisor.json" --type vulnerabilityadvisor