Managing secrets in your toolchains
Many of the tool integrations in your CI and CD toolchains require secrets, for example passwords, API keys, certificates, or other tokens. For example, an IBM Cloud® API key performs basic pipeline tasks, such as logging in to IBM Cloud. Similarly, the service ID API key writes evidence to the bucket in Cloud Object Store instance.
For security reasons, these secrets must not belong to or be affiliated with a person's identity or account because people often have greater access permissions than the toolchain automation requires. Affiliation with a person's identity or account would also violate the security principle of "least privilege". Also, people often change roles or even companies and their credentials must be removed which might break the toolchain automation. By using an identity that is affiliated specifically for automation purposes provides a separation of duties between automation and people who use the automation.
Instead, the secrets that are used for non-IBM Cloud resources (such as GitHub Enterprise) must be affiliated with a functional ID within your enterprise with only the appropriate access that is needed by the toolchains. Likewise, secrets for IBM Cloud resources must be affiliated with an IAM service ID API key that is affiliated with an IAM service ID. The IAM service ID access permissions should be scoped to the least privilege required by the toolchains.
Managing credentials like these must be done securely and in compliance with secrets management best practices. In particular, this means vaulting the required secrets by using an approved in-boundary vault provider, such as IBM Cloud® Secrets Manager, IBM® Key Protect, or HashiCorp Vault and then linking your toolchain secrets to those resources.
The secrets management capabilities that are provided in the toolchain setup and pipeline user interfaces enable selection of vaulted secrets by using secrets integrations for IBM Cloud® Secrets Manager, IBM® Key Protect, or HashiCorp Vault. By
using the Secrets Picker dialog, a toolchain or pipeline editor can select named secrets from a bound secrets integration, that is either configured by CRN (Cloud Resource Name) or by name, that is then resolved
by reference at runtime within the toolchain and pipeline. After a secret is chosen, a CRN or canonical secret reference is injected into the corresponding toolchain or pipeline secure property where the format is either crn:v1:...secret:<secret-guid> if it's a Secrets Manager integration configured by CRN, or alternatively {vault::integration-name.secret-name} if it's a vault integration using any of the supported providers and configured by name.
The canonical by name reference format, currently does not resolve a secret that includes the period character in the secret name because this character is used to delimit each section of the canonical path.
Regardless of which type of secret reference is used, either by CRN or by name, the front-end user interface components and Secrets Picker dialog only use a secret reference. The resolved value of a by CRN or by name secret reference is never exposed to the front-end and is always dynamically resolved at runtime within a toolchain and pipeline based on a permitted authorization being available (configured using IAM Authorizations and Access Policies).
Within IBM Cloud®, the dynamic process of resolving by CRN and by name secrets references in toolchains and pipelines is performed using internal virtual private endpoints (VPE) to all IBM Cloud® Secrets Manager and IBM® Key Protect provider instances in all regions. This ensures all request and response data between toolchains, pipelines, and IBM Cloud® Secrets Manager and IBM® Key Protect provider instances is kept within the in-boundary private IBM Cloud network and does not travel over any public network channels.
In addition to manually selecting chosen secrets on a one-by-one basis from any bound secrets integrations in a toolchain, the option of using a Secret Hint is also available. This option enables a toolchain template to be predefined
with suggested secrets names (also known as Hints) that are a short form secret reference. The format of a secret hint is {vault::secret-name} whereby no secret integration name is included. This provides flexibility
to the toolchain author in that all required secret names can be prepopulated into a toolchain.yml and then these names are automatically resolved against whatever secrets integrations are configured for the toolchain.
As previously described, you can configure Secrets Manager to reference secrets by CRN. For more information, see Cloud Resource Names (CRN). This format allows for greater flexibility because you can reference secrets from an Secrets Manager instance in a different account if the correct authorization is in place. For more information see Configuring Secrets Manager.
The secrets that are used in both CI and CD are outlined as follows:
A Hint is a suggested default name that is automatically resolved against the first matching secret with the same name across any of the available by name secrets integrations that are bound to the toolchain.
DevSecOps Pipeline Secrets
| Secret | Hint | Information |
|---|---|---|
| IBM Cloud API Key | ibmcloud-api-key |
Required: CI & CD Used to authenticate with IBM public cloud and perform a wide range of operations |
| GPG Private Key | signing_key |
Required: CI only This is the certificate that is used to sign images built by the CI pipeline |
| IBM Private Worker Service API Key | private-worker-service-api-key |
Required: CI only A Service ID API Key Used to run delivery pipeline workloads on a Tekton Private Worker Service |
| GitHub Access Token | git-token |
Optional: CI & CD Used to authenticate with GitHub and provide access to the repositories |
| Artifactory API token | artifactory-token |
Required: CI & CD Used to access images used by pipeline tasks |
| Slack Web Hook | slack-webhook |
Optional: CI & CD This webhook is required if you choose to use the Slack tool integration to post toolchain status notifications |
| ServiceNow API Token | servicenow-api-token |
Required: CD only Used to access Service Now for change management operations |
| HashiCorp Vault Role ID | role-id |
Required: CI & CD Used to authenticate with the HashiCorp Vault server |
| HashiCorp Vault Secret ID | secret-id |
Required: CI & CD Used to authenticate with the HashiCorp Vault server |
| IBM Cloud Object Storage Writer API Key | cos-api-key |
Required: CI & CD Used to authenticate with the Object Storage service - This key must have writer permission |
| SonarQube password or authentication token | sonarqube-password |
Optional: CI Used to authenticate with the SonarQube source code analyzer |
If you are using a HashiCorp Vault server, ensure that the HashiCorp Vault tool integration uses the AppRole Auth Method method. When you use the AppRole
authentication method, you need role-id and secret-id to successfully integrate the HashiCorp Vault server with the toolchain. Because role-id and secret-id are secrets in themselves, it
is recommended to store them by using a IBM Key Protect tool integration so that they can be securely retrieved and applied in the toolchain workflow. All other toolchain
secrets should be stored and retrieved by using the HashiCorp Vault tool integration.
If the pipeline environment property git-token is not set, ibmcloud-api-key is used to retrieve the Git Repos and Issue Tracking Access Token by default. However, if ibmcloud-api-key does not have access
to git, git-token must be set.
Configuring the secrets stores
With IBM Cloud, you can choose from various secrets management and data protection offerings that help you protect your sensitive data and centralize your secrets. You can choose between the vault integrations depending on your requirements as explained in Managing IBM Cloud secrets. This documentation provides information about prerequisites and how to use a list of prescribed secret names that are otherwise known as hints. By using hints in a template, a toolchain can be automatically populated with preconfigured secrets without any need to manually select them from various vault integrations that are attached to the toolchain.
Use IBM Cloud® Secrets Manager to securely store and apply secrets like API keys, Image Signature, or HashiCorp Vault credentials that are part of your toolchain.
The templates also come with a HashiCorp Vault tool integration like the following example:
To use HashiCorp Vault, you must provide the following information:
- Name: A name for this tool integration. This name is displayed in the toolchain.
- Server URL: The server URL for your HashiCorp Vault Instance. For example,
https://<vault-service>.<org>.com:8200. - Integration URL: The URL that you want to navigate to when you click the HashiCorp Vault Integration tile.
- Secrets Path: The mount path where your secrets are stored in your HashiCorp Vault Instance.
- Authentication Method: The Authentication method for your HashiCorp Vault Instance. Use
AppRole. - Role ID: Identifier that selects the AppRole against which the other credentials are evaluated.
- Secret ID: Credential that is required by default for any login (with secret_id) and is intended to always be secret.
The templates also come with an IBM® Key Protect for IBM Cloud® tool integration:
If you stored the role id and secret id in Key Protect in advance, then you can select the Key Protect instance that contains those secrets in the tool card as shown in Figure 2. After that is done, then you can click
the key icons on the role id and secret id fields in the HashiCorp Vault tool card, and use the picker to apply the secrets to those fields.
Similarly, any other secrets that are used in the toolchain have a key icon that is attached to the text field. You can use the same picker control to apply the HashiCorp Vault secrets to all the remaining instances.