IBM Cloud Docs
Understanding data portability for Code Engine

Understanding data portability for Code Engine

Data portability involves a set of commands that enable customers to export the digital artifacts that are needed to implement similar workload and data processing on different service providers or on-premises software. It includes commands for exporting the service customer content, including the related configuration that is used by the service to store and process the data, on the customer's own location.

Responsibilities

IBM Cloud services provide interfaces and instructions to guide the customer to copy and store the service customer content, including the related configuration, on their own selected location.

The customer is responsible for the use of the exported data and configuration for data portability to other infrastructures, which includes:

  • The planning and execution for setting up alternative infrastructure on different cloud providers or on-premises software that provide similar capabilities to the IBM services.
  • The planning and execution for the porting of the required application code on the alternative infrastructure, including the adaptation of customer's application code, deployment automation, and so on.
  • The conversion of the exported data and configuration to the format that's required by the alternative infrastructure and adapted applications.

For more information about your responsibilities for Code Engine, see Understanding your responsibilities when using Code Engine.

Data export procedures

Code Engine provides mechanisms to build container images from source code. Source code can be stored either locally or on Github. Code Engine does not provide commands or tools to download such container images. Instead, customers need to download and store those images following the data export procedures for IBM Cloud Container Registry.

Code Engine provides mechanisms to export settings and configurations that are used to process the customer's content.

Configuration data for the following Code Engine entities can be exported using the commands specified later in this section:

  • Projects
  • Applications
  • Application revisions
  • Builds
  • Build runs
  • Configmaps
  • Secrets
  • Domain mappings
  • Event subscriptions (IBM Cloud Object Storage, cron, and Kafka)
  • Functions
  • Jobs
  • Job runs

Tools required for exporting data

For exporting Code Engine data, you need to download and install the following commandline tools:

  • The IBM Cloud CLI client
  • The Code Engine CLI plug-in
    • Visit the Code Engine CLI page which has instructions on how to install the Code Engine plug-in for the IBM Cloud CLI.
  • The Kubernetes CLI client

Exporting project configuration

A project in Code Engine is a container of related configurations and workloads. If you want to export a project, use the following ibmcloud resource commands:

To list all Code Engine projects use:

ibmcloud resource service-instances --service-name codeengine

To list all Code Engine projects in a specific region use:

ibmcloud resource service-instances --service-name codeengine --location us-south

When you have identified the Code Engine project you want to export use:

ibmcloud resource service-instance <project-name> -o json >project-name.json

Exporting a Code Engine project only exports configuration of the project entity itself. Contained configuration entities and workload entities need to be exported separately.

Exporting a KUBECONFIG to export project entities

When you have identified a project you want to export, you have to export a KUBECONFIG environment variable to scope the kubectl commands described in this document to that project.

To obtain this KUBECONFIG environment variable, run the following commands

ibmcloud code-engine project select -n <project-name>
ibmcloud code-engine project current

This selects the project and prints a statement similar to the following one:

Getting the current project context...
OK

Name:       <project-name>
ID:         123a4cbc-567d-8e90-1f2b-032d33d931ba
Subdomain:  1abc2defi34
Domain:     us-south.codeengine.appdomain.cloud
Region:     us-south

Kubernetes Config:
Context:               1abc2defi34
Environment Variable:  export KUBECONFIG="/Users/<username>/.bluemix/plugins/code-engine/<project-name>-123a4cbc-567d-8e90-1f2b-032d33d931ba.yaml"

Copy the export KUBECONFIG=xxx statement to your terminal prompt and run it. All following kubectl commands are now scoped to this project.

Exporting application configuration

To export the configuration of all applications in your project, run the following kubectl command:

kubectl get service -o json >project-name_application-configurations.json

If you want to export the configuration of a single application, use the following kubectl command:

kubectl get service <application-name> -o json >project-name-application-name_application-configuration.json

Exporting application revision configuration

To export the configuration of all application revisions in your project, run the following kubectl command:

kubectl get revision -o json >project-name_application-revisions.json

If you want to export the configuration of a single application revision, use the following kubectl command:

kubectl get revision <application-revision-name> -o json >project-name-application-revision-name_revision.json

Exporting build configuration

To export the configuration of all builds in your project, run the following kubectl command:

kubectl get build -o json >project-name_build-configurations.json

If you want to export the configuration of a single build, use the following kubectl command:

kubectl get build <build-name> -o json >project-name-build-name_build-configuration.json

Exporting build run configuration

To export the configuration of all buildruns in your project, run the following kubectl command:

kubectl get buildrun -o json >project-name_buildrun-configurations.json

If you want to export the configuration of a single buildrun, use the following kubectl command:

kubectl get buildrun <buildrun-name> -o json >project-name-buildrun-name_buildrun-configuration.json

Exporting configmap configuration

To export the content of all configmaps in your project, run the following kubectl command:

kubectl get configmap -o json >project-name_configmaps.json

If you want to export the content of a single configmap, use the following kubectl command:

kubectl get configmap <configmap-name> -o json >project-name-configmap-name_configmap.json

Exporting secret configuration

Do not export the contents of secrets to a file on a persistent storage.

To export the content of all secrets in your project, run the following kubectl command:

kubectl get secret -o json >project-name_secrets.json

If you want to export the content of a single secret, use the following kubectl command:

kubectl get secret <secret-name> -o json >project-name-secret-name_secret.json

Values stored in Code Engine secrets are usually copies of values managed in other systems, like IBM Cloud Secrets Manager. Therefore, most of the time exporting secrets is not required. If you still require to export secrets from Code Engine, you should choose exporting single secrets over exporting all secrets to avoid unintentional disclosure of sensitive information.

Exporting domain mapping configuration

To export the configuration of all domain mappings in your project, run the following kubectl command:

kubectl get domainmapping -o json >project-name_domainmappings.json

If you want to export the configuration of a single domain mapping, use the following kubectl command:

kubectl get domainmapping <domainmapping-name> -o json >project-name-domainmapping-name_domainmapping.json

Exporting IBM Cloud Object Storage event subscription configuration

To export the configuration of all IBM Cloud Object Storage event subscriptions in your project, run the following kubectl command:

kubectl get cossource -o json >project-name_cos-event-subscriptions.json

If you want to export the configuration of a single IBM Cloud Object Storage event subscription, use the following kubectl command:

kubectl get cossource <cos-event-subscription-name> -o json >project-name-cos-event-subscription-name_cos-event-subscription.json

Exporting cron event subscription configuration

To export the configuration of all cron event subscriptions in your project, run the following kubectl command:

kubectl get pingsource -o json >project-name_cron-event-subscriptions.json

If you want to export the configuration of a single cron event subscription, use the following kubectl command:

kubectl get pingsource <cron-event-subscription-name> -o json >project-name-cron-event-subscription-name_cron-event-subscription.json

Exporting Kafka event subscription configuration

To export the configuration of all Kafka event subscriptions in your project, run the following kubectl command:

kubectl get kafkasource -o json >project-name_kafka-event-subscriptions.json

If you want to export the configuration of a single Kafka event subscription, use the following kubectl command:

kubectl get kafkasource <cron-event-subscription-name> -o json >project-name-kafka-event-subscription-name_kafka-event-subscription.json

Exporting function configuration

To export the configuration of all functions in your project, run the following kubectl command:

kubectl get function -o json >project-name_functions.json

If you want to export the configuration of a single function, use the following kubectl command:

kubectl get function <function-name> -o json >project-name-function-name_function.json

Exporting job configuration

To export the configuration of all jobs in your project, run the following kubectl command:

kubectl get jobdefinition -o json >project-name_jobs.json

If you want to export the configuration of a single job, use the following kubectl command:

kubectl get jobdefinition <function-name> -o json >project-name-job-name_job.json

Exporting job run configuration

To export the configuration of all jobruns in your project, run the following kubectl command:

kubectl get jobrun -o json >project-name_jobruns.json

If you want to export the configuration of a single jobrun, use the following kubectl command:

kubectl get jobrun <function-name> -o json >project-name-jobrun-name_jobrun.json

Exported data formats

Code Engine supports the following data format and schema of the exported data, configuration, and application:

  • JSON

Example: Exporting a ConfigMap with name export-sample and format JSON results in the following JSON output:

  • Command: kubectl get configmap export-sample -o json

  • Output:

{
    "apiVersion": "v1",
    "data": {
        "APP_OWNER": "John Doe",
        "NUM_DATASETS": "24",
        "foo": "bar"
    },
    "kind": "ConfigMap",
    "metadata": {
        "creationTimestamp": "2024-10-15T09:12:47Z",
        "name": "export-sample",
        "namespace": "7yuw2furi55",
        "resourceVersion": "9162003517",
        "uid": "12962640-a294-4e1f-a006-a875f119df43"
    }
}

All commands that use the kubectl command with the -o json option, produce a JSON output that is compatible with Kubernetes. The JSON produced by other commands, like ibmcloud resource or ibmcloud code-engine cannot be assumed to be compatible with Kubernetes as is.

Code Engine doesn't support the export of the following data format and schema of the exported data, configuration, and application:

  • YAML

Data ownership

All exported data is classified as customer content. Apply the full customer ownership and licensing rights, as stated in the IBM Cloud Service Agreement.