Viewing logs

Logging can help you troubleshoot issues in IBM Cloud® Code Engine.

Viewing logs from the console

When you work with Code Engine apps, jobs, functions, or builds in the console with logging enabled, logs are forwarded to an IBM Log Analysis service where they are indexed, enabling full-text search through all generated messages and convenient querying based on specific fields.

To view logs for your app, job or build in the Code Engine console, you must create an IBM Log Analysis instance in the same region as your Code Engine project. You are not required to create this instance before you work with your Code Engine component. From your Code Engine app, job, function, or build page in the console, you can add logging capabilities.

You need to enable logging for Code Engine only one time per region, per account.

Considerations for viewing logs from the console

When you want to use logging from the console, you must first configure IBM Log Analysis platform logs to receive Code Engine logging data. To check for active Log Analysis instances, see the Observability dashboard.

Review the Log Analysis service plan information as you consider retention, search, and log analysis needs.

When you use the Log Analysis free tier and you want to view log data, configure and start logging before you start your application, or run your job or build. Also, be sure to keep your platform logs window open to receive your Code Engine logging data. Take these actions because log data is not retained when you use the Lite service plan and is lost when you close the window.

When you use a Log Analysis paid tier, you do not need to leave your logging window open. Log data is preserved for a configurable amount of time, depending on your plan. You can adjust log filters and apply searches for past log entries.

When you view log data for Code Engine applications, runs of your job, or runs of your build, delays can occur before the data is available in Log Analysis. For example, it might take around 5 minutes for your log data to show in Log Analysis, especially if you are using the Lite service plan.

When you use logging with the CLI, you do not need to configure Log Analysis platform logs, as the Code Engine CLI logging fetches its data differently.

Can I apply filters on Log Analysis data?

Yes! When you configure and start logging in the Code Engine console for a specific component, Code Engine automatically sets log filters in Log Analysis that are scoped to the context of your application, job, or build. However, you can modify and scope the preset filter to display log data at a specific level or a more granular level to a specific application revision, job run, or build run from the IBM Log Analysis page, based on your needs.

If _platform is set to 'Code Engine', then only Code Engine logs are displayed.
If label.Project:'<project_name>' is set, then only logs from a specific project are displayed.
If app:'<your_component_name>' is set, then only logs from the specified component (application, job, or build) are displayed. If your Code Engine components share the same name, the filter includes logs from these components. For example,
- The filter _platform:'Code Engine' app:myapp scopes the logs to the myapp application level.
- The filter _platform:'Code Engine' app:myapp-00002 scopes the logs to the myapp-0002 application revision level.
- The filter _platform:'Code Engine' app:myjob scopes the logs to the specific myjob job level.
- The filter _platform:'Code Engine' app:myjob-jobrun-t6m7l scopes the logs to the specific myjob-jobrun-t6m7l job run level.
- The filter _platform:'Code Engine' app:mybuild scopes the logs to the specific mybuild build level.
- The filter _platform:'Code Engine' app:mybuild-run-121212 scopes the logs to the specific mybuild-run-121212 build run level.

For more information about configuring and starting logging in the console, see Viewing logs for apps, Viewing logs for jobs, and Viewing logs for functions.

What if my log data is multi-line?

Every log line must contain log level and timestamp information to appear correctly in IBM Log Analysis. If your log lines span multiple lines, change how you format and output your logs so that they are in a single line. Use the JSONL format for your logs with IBM Log Analysis. Your logs must also conform to limits for IBM Log Analysis logs. After you make log formatting changes, you can take advantage of IBM Log Analysis search and formatting features.

Viewing app logs from the console

After you select the project that you want to work with, you can add logging capabilities from the Code Engine Overview page or one of its child pages such as the Applications page, or from the page that is specific to your application. The following steps assume that you are working from a specific Code Engine application page.

Go to an app that you created and deployed. From the Projects page on the Code Engine console, select your project and then select Applications. Select the app that you want to work with.
From your Code Engine app page, you can add logging capabilities. If you have previously created a Log Analysis instance, click Logging, and this action opens the Log Analysis service. Else, complete the following steps to configure logging capabilities.
1. From the Test application options menu, click Add logging to create the IBM Log Analysis instance. This action opens the Log Analysis service.
2. From the Log Analysis service, create your logging instance. To confirm that your logging instance is created, check the Observability dashboard.
3. From your Code Engine app page, click Add logging from the Test application options menu. This time, select a Log Analysis instance to receive platform logs. Choose the logging instance that you created in the prior step. Click Select. Code Engine requires enabled platform logs to receive Code Engine logging data. When you complete this action, Code Engine enables platform logging for you.
Now that platform logs are configured, from your Code Engine app page, click Logging from the Test application options menu to open your platform logs window. Be sure to keep this platform logs window open to receive your logging data if you are using the Log Analysis free tier. You don't need to keep this window open if you are using a Log Analysis paid tier as log data is preserved for a configurable amount of time, depending on your plan. To confirm that platform logs are set for your region, check the Observability dashboard.
(optional) Refine the filter for your search, if needed.
Test your application. Click Test application, and then click Send request in the Test application pane. To open the application in a web page, click Application URL. You can view platform logs from the test of your application in the platform logs window.

Your Log Analysis instance is now configured such that it can receive platform logging for your Code Engine app.

Alternatively, you can configure a Log Analysis instance by using the Observability dashboard to create the instance, and then by configuring platform logs. After you create your instance, click Configure platform logs. Select the Log Analysis instance to receive the platform log data by specifying a region and your log instance. You can also apply filters for your search in Log Analysis for your Code Engine log data.

Viewing job logs from the console

After you select the project that you want to work with, you can add logging capabilities from the Code Engine Overview page or one of its child pages such as the Jobs page, or from the page that is specific to your job. The following steps assume that you are working from a specific Code Engine job page.

Navigate to a job that you created and deployed. From the Projects page on the Code Engine console, select your project and then select Jobs. Select the job that you want to work with.
From your Code Engine job page, you can add logging capabilities. If you have previously created a Log Analysis instance, click Logging, and this action opens the Log Analysis service. Else, complete the following steps to configure logging capabilities.
1. From the Submit job options menu, click Add logging to create the IBM Log Analysis instance. This action opens the Log Analysis service.
2. From the Log Analysis service, create your logging instance. To confirm that your logging instance is created, check the Observability dashboard.
3. From your Code Engine job page, click Add logging from the Submit job options menu. This time, select a Log Analysis instance to receive platform logs. Choose the logging instance that you created in the prior step. Click Select. Code Engine requires enabled platform logs to receive Code Engine logging data. When you complete this action, Code Engine enables platform logging for you.
Now that platform logging is configured, from your Code Engine job page, click Logging from the Submit job options menu to open your platform logs window. Be sure to keep this platform logs window open to receive your logging data if you are using the Log Analysis free tier. Don't keep this window open if you are using a Log Analysis paid tier.
(optional) Refine the filter for your search, if needed.
Run your job. From the Job runs area, click Submit job to run your job. Provide the job run configuration values or you can take the default values. Click Submit job to run your job. You can view platform logs from the job run in the platform logs window.

Your Log Analysis instance is now configured such that it can receive platform logging for your Code Engine job.

Viewing function logs from the console

After you select the project that you want to work with, you can add logging capabilities from the Code Engine Overview page or one of its child pages such as the Functions page, or from the page that is specific to your function. The following steps assume that you are working from a specific Code Engine function page.

Go to a function that you created and deployed. From the Projects page on the Code Engine console, select your project and then select Functions. Select the function that you want to work with.
From your Code Engine function page, you can add logging capabilities. If you have previously created a Log Analysis instance, click Logging, and this action opens the Log Analysis service. Else, complete the following steps to configure logging capabilities.
1. From the Test function options menu, click Add logging to create the IBM Log Analysis instance. This action opens the Log Analysis service.
2. From the Log Analysis service, create your logging instance. To confirm that your logging instance is created, check the Observability dashboard.
3. From your Code Engine function page, click Add logging from the Test function options menu. This time, select a Log Analysis instance to receive platform logs. Choose the logging instance that you created in the prior step. Click Select. Code Engine requires enabled platform logs to receive Code Engine logging data. When you complete this action, Code Engine enables platform logging for you.
Now that platform logging is configured, from your Code Engine function page, click Logging from the Test function options menu to open your platform logs window. Be sure to keep this platform logs window open to receive your logging data if you are using the Log Analysis free tier. You don't need to keep this window open if you are using a Log Analysis paid tier as log data is preserved for a configurable amount of time, depending on your plan. To confirm that platform logs are set for your region, check the Observability dashboard.
(optional) Refine the filter for your search, if needed.
Test your function. Click Test function, and then click Send request in the Test function pane. To open the function in a web page, click Function URL. You can view platform logs from the test of your function in the platform logs window.

Your Log Analysis instance is now configured such that it can receive platform logging for your Code Engine function.

Viewing build logs from the console

You can display logs for specific build run instances from the console.

Go to the Code Engine dashboard.
Select a project (or create one).
From the project page, click Image builds.
From the Image build tab, click the name of your image build to open the build page for a defined build, or create a build.
From the build page for your defined build, click the name of the instance of your build run in the Build runs section. You might need to click Submit build to create a build run. You can view platform logs from the build run in the platform logs window. Alternatively, you can also view build log information for the build step details from the build run instance page. Expand the build steps for specific build step log data. You can optionally refine the filter for your search, if needed.

Viewing logs with the CLI

To view logging output with the CLI, you must have a running instance of your app or job. If an app is scaled to zero or a job run instance is completed, the output for the ibmcloud ce app logs and ibmcloud ce jobrun logs commands does not have log data. Alternatively, you can use the IBM Log Analysis service to view log data.

Viewing application logs with the CLI

To view app logs for a specific app with the CLI, use the application logs command. You can display logs of all the instances of an app or display logs of a specific instance of an app. The app get command displays details about your app, including the running instances of the app.

To view the logs for all instances of the myapp app, specify the name of the app with the --app option; for example,

ibmcloud ce app logs --app myapp

Example output

Getting logs for all instances of application 'myapp'...
OK

myapp-ii18y-2-deployment-7657c5f4f9-dgk5f:
Server running at http://0.0.0.0:8080/

To view the logs for a specific instance of the app, specify the name of the specific instance of the app with the --instance option; for example,

ibmcloud ce app logs --instance myapp-ii18y-2-deployment-7657c5f4f9-dgk5f

Example output

Getting logs for application instance 'myapp-a5yp2-2-deployment-65766594d4-hj6c5'...
OK

myapp-a5yp2-2-deployment-65766594d4-hj6c5:
Server running at http://0.0.0.0:8080/

Viewing job logs with the CLI

To view logs for a specific job run with the CLI, use the jobrun logs command. You can display logs of all the instances of a job run or display logs of a specific instance of a job run. The jobrun get command displays details about your job run, including the instances of the job run.

To view the logs for all instances of the testjobrun job run, specify the name of the job run with the --jobrun option; for example,

ibmcloud ce jobrun logs --jobrun testjobrun

Example output

Getting jobrun 'testjobrun'...
Getting instances of jobrun 'testjobrun'...
Getting logs for all instances of job run 'testjobrun'...
OK

testjobrun-1-0:
Hello World!

testjobrun-2-0:
Hello World!

testjobrun-3-0:
Hello World!

testjobrun-4-0:
Hello World!

testjobrun-5-0:
Hello World!

To view the logs for the testjobrun-1-0 job run instance, specify the name of a specific instance of the job run with the --instance option; for example,
```
ibmcloud ce jobrun logs --instance testjobrun-1-0 
```
Example output
```
Getting logs for job run instance 'testjobrun-1-0'...
OK

testjobrun-1-0:
Hello World!
```

Viewing build logs with the CLI

To view build logs for a specific build run with the CLI, use the buildrun logs command. You can display logs of all the instances of a build run based on the name of the build run.

To view the logs for all instances of the mybuildrun build run, specify the name of the build run with the --name option; for example,

ibmcloud ce buildrun logs --name mybuildrun

Example output

Getting build run 'mybuildrun'...
Getting instances of build run 'mybuildrun'...
Getting logs for build run 'mybuildrun'...
OK

mybuildrun-zg5rj-pod-z5gzb/step-git-source-source-r9fcf:
{"level":"info","ts":1614363665.8331757,"caller":"git/git.go:169","msg":"Successfully cloned https://github.com/IBM/CodeEngine @ 8b514ce871e50d67cfea3e344b90cade4bd26e90 (grafted, HEAD, origin/main) in path /workspace/source"}
{"level":"info","ts":1614363666.82988,"caller":"git/git.go:207","msg":"Successfully initialized and updated submodules in path /workspace/source"}

mybuildrun-zg5rj-pod-z5gzb/step-build-and-push:
INFO[0002] Retrieving image manifest node:12-alpine
INFO[0002] Retrieving image node:12-alpine
INFO[0003] Retrieving image manifest node:12-alpine
INFO[0003] Retrieving image node:12-alpine
INFO[0003] Built cross stage deps: map[]
INFO[0003] Retrieving image manifest node:12-alpine
INFO[0003] Retrieving image node:12-alpine
INFO[0004] Retrieving image manifest node:12-alpine
INFO[0004] Retrieving image node:12-alpine
INFO[0004] Executing 0 build triggers
INFO[0004] Unpacking rootfs as cmd RUN npm install requires it.
INFO[0008] RUN npm install
INFO[0008] Taking snapshot of full filesystem...
INFO[0010] cmd: /bin/sh
INFO[0010] args: [-c npm install]
INFO[0010] Running: [/bin/sh -c npm install]
npm WARN saveError ENOENT: no such file or directory, open '/package.json'
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN enoent ENOENT: no such file or directory, open '/package.json'
npm WARN !invalid#2 No description
npm WARN !invalid#2 No repository field.
npm WARN !invalid#2 No README data
npm WARN !invalid#2 No license field.

up to date in 0.267s
found 0 vulnerabilities

INFO[0011] Taking snapshot of full filesystem...
INFO[0011] COPY server.js .
INFO[0011] Taking snapshot of files...
INFO[0011] EXPOSE 8080
INFO[0011] cmd: EXPOSE
INFO[0011] Adding exposed port: 8080/tcp
INFO[0011] CMD [ "node", "server.js" ]

mybuildrun-zg5rj-pod-z5gzb/step-image-digest-exporter-ngl6j:
2021/02/26 18:21:02 warning: unsuccessful cred copy: ".docker" from "/tekton/creds" to "/tekton/home": unable to open destination: open /tekton/home/.docker/config.json: permission denied
{"severity":"INFO","timestamp":"2021-02-26T18:21:26.372494581Z","caller":"logging/config.go:116","message":"Successfully created the logger."}
{"severity":"INFO","timestamp":"2021-02-26T18:21:26.372621756Z","caller":"logging/config.go:117","message":"Logging level set to: info"}