Part 2: Set up a Continuous Integration (CI) toolchain
This tutorial is part 2 of a 4-part tutorial series where you learn IBM Cloud® DevSecOps best practices by using a complete reference implementation that is available as a service and powered by IBM Cloud® Continuous Delivery. In part 2 of this tutorial series, you use the toolchain template for continuous integration (CI) with security and compliance-related best practices in DevSecOps. It is preconfigured for continuous deployment with inventory integration, change management with Git Repos and Issue Tracking, evidence collection, and deployment to IBM Cloud Kubernetes Service.
Before you begin
Before you begin part 2 of this tutorial series, ensure that you complete the following prerequisites:
- Complete Part 1: Set up prerequisites.
- View the Getting started with DevSecOps in IBM Cloud - Part 1 video.
- Start the IBM Cloud console. The console is the starting point of this tutorial.
Continuous Integration (CI) toolchain introduction
The CI toolchain implements the following best practices:
- Runs a static code scanner on the application repositories to detect secrets in the application source code and vulnerable packages that are used as application dependencies
- Builds a container image on every Git commit, setting a tag based on build number, timestamp, and commit ID for traceability
- Tests the Dockerfile before the image is created
- Stores the built image in a private image registry
- Automatically configures access permissions for target cluster deployment by using API tokens that can be revoked
- Scans the container image for security vulnerabilities
- Adds a Docker signature upon successful completion
- Inserts the built image tag into the deployment manifest automatically
- Uses an explicit namespace, in cluster, to insulate each deployment (and make it easy to clear, by
kubectl delete namespace) - Automatically builds, validates, and deploys to the Kubernetes cluster any code merged into the target Git repository branch.
The target cluster is configured during toolchain setup (by using an IBM Cloud API key and cluster name). You can later change these settings by altering the Delivery Pipeline configuration.
Guided setup overview for the CI toolchain
The Continuous Delivery service provides templates that guide you through the toolchain setup process in a logical order, presenting you with the recommended configuration options that are needed to create your toolchain.
A progress indicator shows the steps to complete the configuration. You can use the progress indicator to navigate to a previous step with a mouse click. The configuration options for the current step are displayed in the main area of the page.
To advance to the next step, click Continue. You can advance to the next step only when the configuration for the current step is complete and valid. You can navigate to the previous step by clicking Back.
Some steps include a Switch to advanced configuration toggle. These steps by default present you with the minimum recommended configuration needed. However, advanced users that need finer grained control can click the Switch to advanced configuration toggle to reveal all options for the underlying integration.
After all the steps are successfully completed, you create the toolchain by clicking Create on the Summary step.
You can always go back to previous steps in the guided installer. The toolchain installer retains all the configuration settings from the successive steps.
Start the CI toolchain setup
Use one of the following options to access the template for the CI toolchain:
-
Click Create toolchain.
-
From the IBM Cloud console, click Menu
and select DevOps. On the Toolchains page, click Create toolchain.
On the Create a Toolchain page, click CI - Develop a secure app with DevSecOps practices.
Set up the CI toolchain name and region
The Welcome page summarizes the purpose of the toolchain that is created, along with pointers to documentation and related materials. When you are ready to proceed, click Start to begin configuring the toolchain.
Review the default information for the toolchain settings:
- The toolchain's name identifies it in IBM Cloud. Make sure that the toolchain's name is unique within your toolchains for the same region and resource group in IBM Cloud.
- The toolchain region is the region where the toolchain itself and related resources (Git repos, pipelines, and so on) are created. Toolchains can deploy to any region, regardless of where they are created.
Set up CI tool integrations
Multiple repositories must be configured during the guided setup, as described in the following sections.
You can clone the sample repositories or you can use your own, but the toolchain supports Git Repos and Issue Tracking repositories only.
Application
The application source code repository can be hosted either on GitHub or the IBM-hosted Git Repos and Issue Tracking repository from the list of source providers. For consistency, this tutorial refers to Git Repos and Issue Tracking.
The recommended options are displayed by default, but you can click the Switch to advanced configuration toggle to see all of the configuration options available for the underlying Git integration. The default behavior of the toolchain is Use default sample that clones the sample application into an IBM-hosted Git Repos and Issue Tracking Repository.
Enter a name for the IBM-hosted Git Repos and Issue Tracking repository that is created by the toolchain as your application repository.
The region of the repository remains the same as the region of the toolchain.
If you want to link an existing Application Repository for the toolchain, select the Bring your own application option, and provide it as input to Repository URL field. As noted earlier, the toolchain currently supports linking only to existing Git Repos and Issue Tracking repositories. If you want to know more about Bring your own application, see Bringing your own app to DevSecOps.
Inventory
The inventory repository records details of artifacts that are built by the CI toolchains.
Issues
The issues repository records issues that are found while the CI pipeline is running.
Secrets
Specify the secret vault integrations to be added to your toolchain by using the provided toggles, as explained in Managing IBM Cloud secrets.
The CI toolchain supports Arbitrary secrets and IAM credentials secret types only.
This tutorial uses IBM Cloud® Secrets Manager as the vault for secrets. The Region, Resource group, and Service name fields are automatically populated based on available choices. Click the drop-down indicators to see the other choices.
Evidence Storage
The evidence repository stores all the evidence and artifacts that are generated by the DevSecOps CI pipeline.
Toggle the COS bucket slider to also store all the evidence in a Cloud Object Storage bucket that can be configured on the next page.
Cloud Object Storage bucket
To use this feature, you must have a Cloud Object Storage instance and a bucket. Click here to create a Cloud Object Storage instance, and read the recommendation for configuring a bucket that can act as a compliance evidence locker.
- Cloud Object Storage instance, Bucket name, and Cloud Object Storage endpoint fields are automatically populated.
- Enter the Service ID API key
- Preferred: An existing key can be imported from a secrets vault by clicking the key icon.
- An existing key can be copy and pasted.
The endpoint field is optional. It is recommended to select or provide the endpoint during the setup of the toolchain or during the pipeline run.
Deploy
Configure the target Kubernetes cluster where the application is deployed:
-
App name: The name of the application.
- Default: hello-compliance-app
-
IBM Cloud API Key: The API key is used to interact with the ibmcloud CLI tool in several tasks.
- Preferred: An existing key can be imported from a secrets vault by clicking the key icon.
- An existing key can be copy and pasted.
- A new key can be created from here by clicking the New +.
The newly generated API key can be immediately saved to a secrets vault.
If the API key is valid and has sufficient access, the Container registry region and namespace Cluster region, name, namespace, and resource group are automatically populated. You can change any of these fields to match your configuration if needed.
Image Signing
Any images that are built by this toolchain and recorded in the inventory must be signed before they can be deployed to production.
To enable image signing, you need to provide a GPG key. You can select an existing key from a secret store or input the value directly. Alternatively you can create a new GPG key by clicking NEW.
Ensure that the key follows the appropriate encoding as required by the chosen tool to store secrets.
DevOps Insights
IBM Cloud DevOps Insights is included in the created toolchain. View your pipeline test results for every build, from every deployment and environment.
You do not need to provide any configuration steps for DevOps Insights (DOI). Select a DOI instance for your toolchain:
- Current (to be created) CI toolchain
- Another toolchain instance: select an existing toolchain (that has a DOI instance)
SonarQube
SonarQube provides an overview of the overall health and quality of your source code and highlights issues that are found in new code. The static code analyzers detect tricky bugs, such as null-pointer dereferences, logic errors, and resource leaks for multiple programming languages.
- Default configuration
- Run a SonarQube scan in an isolated Docker-in-Docker container without persisting the SonarQube instance.
- Cluster configuration
- A new SonarQube instance is provisioned in the Kubernetes cluster (chosen at the Deploy step) during the first CI pipeline run.
- Custom configuration
- Link an existing SonarQube instance with the toolchain.
Optional tools
Slack
Configure the Slack Tool to receive notifications about your PR/CI Pipeline events.
- Slack webhook
- Preferred: An existing webhook can be imported from a secrets vault by clicking the key icon.
- An existing webhook can be copy and pasted.
- Slack channel: an existing Slack channel to post message to
- Slack team name: the phrase before .slack.com in our team URL. For example, if your team URL https://team.slack.com, the team name is team.
- Automated Slack Notifications: Customize your selection by choosing the events for which you want to receive notifications.
You can add the Slack Tool after toolchain creation.
Create the CI toolchain
On the Summary page, click Create, and wait for the toolchain to be created.
Other toolchain integrations can be configured after the pipeline is created.
Explore the CI toolchain
The DevSecOps CI toolchain looks like this:
It contains two pipelines:
- ci-pr-pipeline (PR): triggers when a new Merge Request (MR) is submitted in the app repository.
- ci-pipeline (CI): triggers when an MR is merged to master in the app repository, but can be manually triggered.
Run the ci-pr Pipeline
To start the ci-pr pipeline, you need to create a merge request in your application repository.
- From the CI toolchain page, click the application repository tile. By default it gets created with a name
compliance-app-<timestamp>. - Create a branch from master branch.
- Update some code like in app or readme file and save changes.
- Submit merge request.
- On the CI toolchain page, click the pr-pipeline tile. Verify that the ci-pr pipeline is triggered (by the creation of the Merge Request).
- Wait for the ci-pr pipeline run to complete.
The corresponding Merge Request that is in your application repository is in Pending state until all the stages of PR Pipeline finish successfully.
After the PR Pipeline run is successful, you can click it to explore numerous steps completed.
- Navigate back to the merge request.
- Merge the request so that your changes are copied to the master branch of your application repository: the CI pipeline is automatically triggered.
Run the CI pipeline
Start the CI pipeline in one of the following ways:
- Automatically: after a successful PR pipeline, by approving and merging the PR to the master branch.
- Manually: to trigger the CI pipeline manually, select the Delivery Pipeline card and click Run Pipeline and select Manual Trigger.
In this tutorial, the CI Pipeline was triggered after you merged your code changes to the master branch of your application repository.
- On the CI toolchain page, click the ci-pipeline tile.
- Observe: a pipeline-run is running. Wait for the pipeline-run to complete.
After the CI Pipeline run is successful, you can click it to explore the completed steps.
Evidence from all compliance checks in the CI pipeline is uploaded to the evidence-locker repository, but also to the COS bucket if enabled during setup. Evidence is also published to DevOps Insights. You can navigate to DOI by clicking the DevOps Insights tool card in the toolchain. You can review the collected evidence on the Quality Dashboard page.
To evaluate if you have any failures in your pipeline run, you need to check the final step of your pipeline, which has a pipeline evaluator.
Viewing the running application
After a successful CI pipeline run, the sample application is deployed on your Kubernetes cluster.
The app url can be found at the end of the log of the run stage step of deploy-dev task of the CI Pipeline run. Use that url to verify that the app is running.
Pipeline customization
Custom scripts are extension points in the pipeline where adopters, teams, and users can provide scripts to run custom tasks that are required by their continuous integration and continuous deployment strategies.
Custom scripts control the pipeline stages. You can use a configuration file (pipeline-config.yaml) to configure the behavior of stages, script content, and the base image that runs the scripts. The scripts and configuration for
pipeline stages are loaded from a Git repository (repo) that can either be the application (app) repo (similar to .travis.yml or Jenkinsfile) or a custom repo.
More detailed information on customizing the CI pipelines can be found here.
Wrapping up
By completing the CI part of this tutorial, you:
- Created a DevSecOps CI toolchain
- Modified some code in the application repository
- Ran the ci-pr pipeline before you merge your changes
- Ran the ci-pipeline to build, test, and deploy your changes to your dev environment.
Next steps
Continue to Part 3: Set up a Continuous Deployment (CD) toolchain.