Accessing container registries

Images that are used by IBM Cloud® Code Engine are typically stored in a registry that can either be accessible by the public (public registry) or set up with limited access for a small group of users (private registry).

A container registry, or registry, is a service that stores container images. For example, IBM Cloud Container Registry and Docker Hub are container registries. A container registry can be public or private. A container registry that is public does not require credentials to access. In contrast, accessing a private registry does require credentials.

Code Engine requires access to container registries to complete the following actions:

To retrieve (or "pull") a container image to run an app or job
To store a newly created container image as an output of an image build
To store and retrieve local files when a build is run from local source

Code Engine handles many of the underlying details of the interactions between the system and your registry.

To pull images from a registry, Code Engine uses a special type of Kubernetes secret that is called an imagePullSecret. This image pull secret stores the credentials to access a container registry. When you add access to a container registry with Code Engine to pull images, you are creating an image pull secret. For more information about image pull secrets, see Kubernetes documentation.

Types of image registries

Images are typically stored in a registry that can either be accessible by the public (public registry) or set up with limited access for a small group of users (private registry).

Public registries, such as public Docker Hub, can be used to get started with Docker and Code Engine to create your first application or job. But when it comes to enterprise workloads, use a private registry, such as the one provided in IBM Cloud Container Registry to protect your images from being used by unauthorized users. For private registries, use registry secrets to ensure that the credentials are available to gain access to the private registry.

Public and private image registry types
Registry	Description
IBM Cloud Container Registry	With this type of registry, you can set up your own secured image repository in IBM Cloud Container Registry where you can safely store and share images between users. With IBM Cloud Container Registry, you can Manage access to images in your account. Use IBM provided images and sample apps, such as IBM Liberty, as a base image and add your own app code to it.
Any other private registry	Connect any existing private registry to Code Engine by adding access. Adding access securely saves your registry URL and credentials in a Kubernetes secret. With private registries, you can: Use existing private registries independent of their source (Docker Hub, organization-owned registries, or other private Cloud registries).
Public Docker Hub	Use this type of registry to pull existing public images from Docker Hub directly in your Code Engine applications or jobs. Important: This registry type might not meet your organization's security requirements such as access management, vulnerability scanning, or app privacy. When you pull an image from Docker Hub to use with apps or jobs in Code Engine, be aware of Docker rate limits for free plan (anonymous) users. You might experience pull limits if you receive a `429` error which indicates that you have reached your pull rate limit. To increase rate limits, you can upgrade your account to a Docker `Pro` or `Team` subscription. With public Docker Hub, you can: These images can be referred to directly when you create an app or job, no additional setup is required. Includes various open source applications.

Types of registry secrets

To access images in a registry, Code Engine uses one of the following types of registry secrets.

Code Engine managed secret - If your registry uses an IBM Cloud Container Registry namespace that is in your account, then you can let Code Engine create and manage the registry secret for you. In the console, this automatically created registry secret is called a Code Engine managed secret. In the CLI, the name of an automatically created registry secret is of the format, ce-auto-icr-private-<region>.
User managed secret - This is a secret that you create and manage. You can access images from your account with an API key or use an access token for the container registry of your choice; for example, Docker Hub. For this case, the registry secret that is listed in the console is the name of your registry secret.

If your registry is public and does not require credentials; for example, Code Engine sample images in icr.io/codeengine or Docker Hub public, then you do not need a registry secret. For this case, the registry secret that is listed in the console is None.

Setting up authorities for image registries

If your registry is public, you do not have to set up authorities to pull images. Note that pulling images from a public registry while you are getting started with Code Engine is acceptable. Use a private registry when it comes to your enterprise workloads.

What authorities do I need?

To determine the authorities that you need, consider the following cases:

When you deploy apps or run jobs, Code Engine can automatically access images that are in your own account.
If you want to access images from a shared account, other IBM Cloud accounts, or a private Docker account, you must be assigned the proper access authorities.
When you deploy apps or run jobs and your registry uses an IBM Cloud Container Registry namespace that is in your account, then you can let Code Engine automatically create and manage the registry secret for you, as long as your account has the required permissions as described in the following table.
- In the console, this registry secret is called a Code Engine managed secret. This option is available when you use the Configure image or Specify build details workflows for building an image with Code Engine.
- In the CLI, this registry secret is of the format, ce-auto-icr-private-<region>. This registry secret is automatically created when you specify the --build-source option but you do not provide the --registry-secret option with the app create, app update, job create, or job update commands.

Access authorities for image registry
Action	IAM service access	Description
Pull images	`Reader` service access	When you deploy an image as an application or job, you must pull the image from a registry. To pull images, you need read access. If the registry is public, you already have read access to the images. If the registry is private, then a registry secret is required.
Push images	`Reader` and `Writer` service access	When you build source code, you must push the image to a registry. To push images to your registry, you must have read and write access, and you must have a registry secret.
Create a namespace	`Reader`, `Writer`, and `Manager` service access	This action is only supported for IBM Cloud Container Registry.
Code Engine automatically created registry secret	`Administrator` platform access `Reader`, `Writer`, and `Manager` service access	This action is only supported for IBM Cloud Container Registry.

Can I use a service ID?

Yes, you can create a service ID and assign authorities to it. Note that service IDs are also automatically created by the Code Engine UI when you automatically create access to your IBM Cloud Container Registry. DO NOT delete this service ID as you will lose access to the images in the registry.

Can I access images in a different registry?

Yes! Here is how.

Can I restrict pull access to a certain regional registry or even a single namespace?

Yes, you can edit the existing IAM policy of the service ID that restricts the Reader service access role to that regional registry or a registry resource such as a namespace. Before you can customize registry IAM policies, you must enable IBM Cloud IAM policies for IBM Cloud Container Registry.

Accessing images from a public account

If your image is stored in a public repository, such as a public Docker Hub, then you can simply reference the image directly when you deploy your application or run your job. Note that while storing images in a public registry is fine for getting started with applications and jobs, store your enterprise images in a private registry.

Accessing images in your own account from console

If you are accessing Code Engine from an account that you own or administer, then Code Engine can automatically push and pull images to and from an IBM Cloud Container Registry namespace in your account when you create or update apps, jobs, or builds from the console. Code Engine can even create a namespace for you when you push an image. For more information, see the following topics.

Accessing images from your account with an API key

If you are accessing Code Engine with the CLI, you must first create an IAM API key and then save the IAM API key as registry access in Code Engine.

The following steps create an API key that stores the credentials of a user ID. Instead of using a user ID, you might want to create an API key for a service ID that has an IBM Cloud IAM service access policy to IBM Cloud Container Registry. If you choose to authenticate a user ID, make sure that the user is a functional ID or plan for cases where the user leaves so that Code Engine can still access the registry.

Creating an API key from the console

To create an IBM Cloud IAM API key from the console,

Launch Access (IAM) Overview.
Select API keys.
Click Create an IBM Cloud API key.
Enter a name and optional description for your API key and click Create.
Copy the API key or click download to save it.

You won’t be able to see this API key again, so be sure to record it in a safe place.

Now that you created your API key, save it as registry access.

Creating an API key with the CLI

To create an IBM Cloud IAM API key with the CLI, run the iam api-key-create command. For example, to create an API key called cliapikey with a description of My CLI API key and save it to a file called key_file, run the following command:

ibmcloud iam api-key-create cliapikey -d "My CLI API key" --file key_file

If you choose to not save your key to a file, you must record the API key that is displayed when you create it. You cannot retrieve it later.

Now that you created your API key, save it as registry access.

Accessing images in a shared account

To access images from IBM Cloud Container Registry in a shared account, you must be assigned the proper authority.

If you are planning to deploy apps and run jobs from the shared account, Code Engine can pull or push images for you when you deploy your application or create your job.

If you want to pull images from the shared account to your own account, then you must be authorized to access IBM Cloud Container Registry.

Accessing images in a different account

You can assign IBM Cloud IAM access policies to users or a service ID to restrict permissions to specific registry image namespaces or actions (such as push or pull). Then, create an API key and store these registry credentials in Code Engine.

For example, to access images in other IBM Cloud accounts, create an API key that stores the IBM Cloud Container Registry credentials of a user or service ID in that account. Then, in Code Engine, use that key to create access in your account.

Accessing images in a private Docker Hub account

To access images in a private Docker Hub account, create registry access by providing your password or an access token. By using an access token, you can more easily grant and revoke access to your Docker Hub account without requiring a password change. For more information about access tokens and Docker Hub, see Managing access tokens.

After you decide whether to use your password directly or to create an access token, create your registry access.

Add registry access to Code Engine

To set up access to an IBM Cloud Container Registry in a different IBM Cloud account, to pull images from a private Docker Hub account, or to pull or push images by using the Code Engine CLI, you can use the IBM API key or Docker Hub password or access token to create registry access through Code Engine to store your authentication key or token for you.

Adding registry access from the console

Before you begin, create a project.

After your project is in Active status, click the name of your project on the Code Engine Projects page.
From the Components page, click Secrets and configmaps.
From the Secrets and configmaps page, click Create to create your secret.
From the Create secret or configmap page, complete the following steps:
1. Select Registry secret, and click Next.
2. Provide a name; for example, mysecret-registry.
3. Specify the target registry for this secret, such as IBM Cloud Container Registry or Docker Hub.
4. Specify the location of the registry.
5. Specify a username. If this secret is for IBM Cloud Container Registry, the username is iamapikey. If this secret is for Docker Hub, it is your Docker ID.
6. Enter the credentials for the username. For IBM Cloud Container Registry, use your IAM API key. For Docker Hub, you can use your Docker Hub password or an access token. For other target registries, specify the password or API key for the username.
7. Click Create to create the secret.

Now that your secret is created from the console, go to the Secrets and configmaps page to view a list of defined secrets and configmaps. You can apply filters to customize the list to meet your needs.

You can add access to a container registry when you create an application or job, or when you build an image. Click Configure image and specify the container image to run, including the registry where the image is stored and the registry access to use to retrieve the image.

Adding registry access with the CLI

Beginning with CLI version 1.42.0, defining and working with secrets in the CLI is unified under the secret command group. See ibmcloud ce secret commands. Use the --format option to specify the category of secret, such as basic_auth, generic, ssh, tls, or registry. While you can continue to use the registry command group, take advantage of the unified secret command group. To create a secret to access a container registry, use the ibmcloud ce secret create --format registry command. To learn more about working with secrets in Code Engine, see Working with secrets.

To add IBM Cloud Container Registry or Docker Hub access with the CLI, use the secret create --format registry command. This command requires a name of the registry secret, the URL of the registry server, and the username and password information to access the registry server, and also allows other optional arguments. For a complete listing of options, see the ibmcloud ce secret create command.

For example, the following command creates registry access to an IBM Cloud Container Registry instance called myregistry that is on the us.icr.io registry server:

ibmcloud ce secret create --format registry --name myregistry --server us.icr.io --username iamapikey --password API_KEY

Example output

Creating registry secret 'myregistry'...
OK

The following table summarizes the options that are used with the secret create --format registry command in this example. For more information about the command and its options, see the ibmcloud ce secret create command.

Command description
Option	Description
`--name`	The name of the registry secret. Use a name that is unique within the project. This value is required. The name must begin and end with a lowercase alphanumeric character. The name must be 253 characters or fewer and can contain lowercase letters, numbers, periods (.), and hyphens (-).
`--server`	Enter the URL of the registry server. For Container Registry, the server name is `<region>.icr.io`. For example, `us.icr.io`. For Docker Hub, the value is `https://index.docker.io/v1/`.
`--username`	Enter the username to access the registry server. For Container Registry, it is `iamapikey`. For Docker Hub, it is your Docker ID.
`--password`	Enter the password. For Container Registry, the password is your API key. For Docker Hub, you can use your Docker Hub password or an access token.

Authorizing access to Container Registry with service ID

Before you can add access to a service ID in a different account, you must first authorize access to the service ID.

When you create a service ID, you can restrict access to a regional IBM Cloud Container Registry or even a specific namespace within that IBM Cloud Container Registry account.

Authorizing access to Container Registry with service ID from the console

To pull or push images from or to IBM Cloud Container Registry, you must create a service ID, create an access policy for the service ID, and then create an API key to store the credentials.

Step 1 Create or identify a service ID and authorize it to the IBM Cloud Container Registry service

Launch Access (IAM) Overview.
Select Service IDs.
If you have a Service ID that you want to use, select it. If not, select Create, enter a name and description, and click Create.
From the Service ID page, from the Access policies section, select Assign access.
From the Assign service ID additional access section,
1. Select Container Registry for type of access. Click Next.
2. Select the type of access: All resources or Specific resources. If you specify Specific resources, you can add attributes based on resource group, geography, region, resource type, resource ID, or resource name to further restrict access. If you select a certain resource group, make sure to select Viewer access for Resource group access. Click Next.
3. In the Roles and Actions section, select the type of access you want to grant. If you plan to use only images for your applications and jobs, select Reader. If you want to push the source code and images to Container Registry, then also select Writer. Click Review.
4. Click Add and then Assign.

Step 2 Enabling Container Registry discovery

To allow the Code Engine console to automatically discover Container registry, you must authenticate the service ID to the IAM Identity Service.

From the Service ID page, from the Access policies section, select Assign access.
From the Assign service ID additional access section,
1. Select IAM Identity Service for type of access. Click Next.
2. Select Specific resources for resource scope. Select Resource type as attribute type, keep string equals as operator and enter serviceid as value. Click Add a condition.
3. Select Resource ID as attribute type, keep string equals as operator and put the identifier of your service ID. You can find your service ID on the Details page for the service ID or in the browser URL when configuring it. Click Next.
4. In the Roles and Actions section, select Platform Operator access. Click Review
5. Click Add and then Assign.

Step 3 Creating an API key for a service ID

Create an API key for a service ID.

From the Service ID page, select API keys and then Create.
Enter a name and optional description for your API key and click Create.
Copy the API key or click download to save it.

You won’t be able to see this API key again, so be sure to record it in a safe place.

Now that you have your access policies in place for your service ID and your API key created, you can add access to Code Engine to pull images from your container registry.

Authorizing access to Container Registry with the CLI

To pull images from IBM Cloud Container Registry in a different account, you must create a service ID, create access policies for the service ID, and then create an API key to store your credentials.

Create an IBM Cloud IAM service ID for your project that is used for the IAM policies and API key credentials in the image pull secret with the iam service-id-create command. Be sure to give the service ID a description that helps you retrieve the service ID later, such as including the project name. For a complete listing of the iam service-id-create command and its options, see the ibmcloud iam service-id-create command.

For example, the following command creates a service ID called codeengine-myproject-id with the description Service ID for IBM Cloud Container Registry in Code Engine project myproject:
```
ibmcloud iam service-id-create codeengine-myproject-id --description "Service ID for IBM Cloud Container Registry in Code Engine project my proj"
```

Create a custom IBM Cloud IAM policy for your service ID that grants access to IBM Cloud Container Registry with the iam service-policy-create command. For a complete listing of the iam service-policy-create command and its options, see the ibmcloud iam service-policy-create command.

For example, the following command creates a policy for codeengine-myproject-id service ID with the role of Reader:

ibmcloud iam service-policy-create codeengine-myproject-id --roles Reader --service-name container-registry

The following table summarizes the options that are used with the iam service-policy-create command in this example. For more information about the command and its options, see the ibmcloud iam service-policy-create command.

iam service-policy-create command components
Option	Description
`<service_ID>`	Required. Replace with the `codeengine-<project_name>-id` service ID that you previously created.
`--roles <service_access_role>`	Required. Enter the service access role for IBM Cloud Container Registry that you want to scope the service ID access to. Possible values are `Reader`, `Writer`, and `Manager`. If you are pulling images, then `Reader` access is sufficient. For more information, see Setting up authorities for image registries.
`--service-name <container-registry>`	Required. Enter `container-registry` to create an IAM policy for IBM Cloud Container Registry.

Create a custom service policy to allow access to iam-identity service so that Code Engine can retrieve the API key for your service ID with the iam service-policy-create command.

For example, create a policy for codeengine-myproject-id service ID with the role of Operator:

ibmcloud iam service-policy-create codeengine-myproject-id --roles Operator --service-name iam-identity

iam service-policy-create command components
Option	Description
`<service_ID>`	Required. Replace with the `codeengine-<project_name>-id` service ID that you previously created.
`--roles <platform_access_role>`	Required. Enter the platform access role that you want to scope the service ID access to. Possible values are `Administrator`, `Editor`, `Operator`, and `Viewer`. Your service ID requires `Operator` or higher.
`--service-name <iam-identity>`	Required. Enter `iam-identity` to create an IAM policy for IAM identify services.

Create an API key for the service ID with the iam service-api-key-create command. For a complete listing of the iam service-api-key-create command and its options, see the ibmcloud iam service-api-key-create command. Name the API key similar to your service ID, and include the service ID that you previously created, codeengine-<project_name>-id. Be sure to give the API key a description that helps you retrieve the key later.

For example, the following command creates a key that is called codeengine-myproject-key for the codeengine-myproject-id service ID with a description of API key for service ID codeengine-myproject-id for Code Engine myproject:
```
ibmcloud iam service-api-key-create codeengine-myproject-key codeengine-myproject-id --description "API key for service ID codeengine-myproject-id for Code Engine myproject"
```
Example output
```
Please preserve the API key! It cannot be retrieved after it's created.

Name          codeengine-myproject-key
Description   API key for service ID codeengine-myproject-id for Code Engine myproject
Bound To      crn:v1:bluemix:public:iam-identity::a/1bb222bb2b33333ddd3d3333ee4ee444::serviceid:ServiceId-ff55555f-5fff-6666-g6g6-777777h7h7hh
Created At    2019-02-01T19:06+0000
API Key       i-8i88ii8jjjj9jjj99kkkkkkkkk_k9-llllll11mmm1
Locked        false
UUID          ApiKey-222nn2n2-o3o3-3o3o-4p44-oo444o44o4o4
```
You won’t be able to see this API key again, so be sure to record it in a safe place.

Now that you have your access policies in place for your service ID and your API key that is created, you can add access to Code Engine to pull images from your container registry.

Controlling access to Container Registry for Code Engine workloads

Suppose that you want to control access to IBM Cloud Container Registry when Code Engine pulls images. For example, you want to control access to Container Registry to specific IP addresses. Consider the following approaches.

Use a context-based restriction. By using a context-based restriction, if the IP addresses for your Code Engine project ever change, you do not need to change your access. You can restrict access to Container Registry to a network zone, where your network zone includes Code Engine and anything that requires access to the registry.
Disable the public access to IBM Cloud Container Registry and make sure Code Engine uses the private endpoints instead of the public endpoints. See Securing your connection to Container Registry.
To control access by a specific IP range, use an API endpoint to fetch the IP addresses for your particular Code Engine project. It is important to note that these IP addresses are subject to change, and you must take appropriate steps when this happens. See Code Engine public and private IP addresses and How can I add my Code Engine app to an allowlist?

Considerations for images in your registry

The name of your image that is used for your app or job must be in one of the following formats.

REGISTRY/NAMESPACEorDOCKERUSERorDOCKERORG/REPOSITORY:TAG where REGISTRY and TAG are optional. If REGISTRY is not specified, the default is docker.io. If TAG is not specified, do not include the colon (:). The default for TAG is latest.
REGISTRY/NAMESPACEorDOCKERUSERorDOCKERORG/REPOSITORY@IMAGEID where REGISTRY is optional. If REGISTRY is not specified, the default is docker.io and ibm as the Docker organization.

Rules for image name
Component	Characters allowed	Length	Additional rules
`REGISTRY`	`a-zA-Z0-9 -_. --__`	1-253	`(0-127Periods)(label:1-63,noDashOnEnd)`
`NAMESPACE`	`a-z 0-9 -_ --__`	4-30	`(start/end with letterOrNumber)`
`DOCKERUSERorDOCKERORG`	`a-z 0-9`	4-30
`REPOSITORY`	`a-z 0-9 -_. /`	2-255	`(start/end with letterOrNumber)`
`TAG`	`a-zA-Z0-9 -_. --__..`	0-128	`(NOT start with periodOrDash)`
`IMAGEID`	`a-z 0-9 :`		`(startwith sha256: noOtherColon)`

The parts of the image name must meet the following criteria.

REGISTRY must be 253 characters or fewer and can contain lowercase or uppercase letters, numbers, periods (.), hyphens (-), and underscores (_). Do not use a dash (.) as the last character. Do not use more than 127 periods (.) and the labels between them can be between 1 and 63 characters long.
NAMESPACE must be between 4 and 30 characters and must begin and end with a lowercase letter or number. NAMESPACE can contain lowercase alphanumeric characters, hyphens (-), and underscores (_).
DOCKERUSERorDOCKERORG can be used for Docker registries instead of NAMESPACE. Specify your Docker username or Docker organization. Your Docker username and organization must be between 4 and 30 characters and contains only lowercase alphanumeric characters or numbers.
REPOSITORY must be between 2 and 255 characters and must begin and end with a lowercase letter or number. REPOSITORY can contain lowercase alphanumeric characters, forward slashes (/), periods (.), hyphens (-), and underscores (_).
TAG must be between 0 and 128 characters and can contain lowercase or uppercase letters, numbers, periods (.), hyphens (-), and underscores (_). The TAG must not begin with a period or dash. If you do not include a TAG, do not include the colon either.
IMAGEID is prefixed with sha256: and can contain lowercase letters and numbers.