Forwarding logs to IBM Log Analysis

Platform logs are logs that are exposed to users by various services on the IBM Cloud platform. IBM Analytics Engine supports forwarding logs from the IBM Analytics Engine service to an IBM Log Analysis instance that you have enabled to receive platform logs.

Platform logs are regional.

You can monitor logs from IBM Analytics Engine on IBM Cloud in the region where the service is available.
You can configure only one instance of IBM Log Analysis per region and per IBM Cloud account to collect platform logs in that location.

You can have multiple IBM Log Analysis instances in a location. However, only one instance in a location (region) can be configured to receive logs from IBM Cloud services such as IBM Analytics Engine in that IBM Cloud location.

Also, the logs that are generated by an IBM Analytics Engine instance belonging to an account are forwarded to an IBM Log Analysis instance under the same IBM Cloud account.

To check for the Log Analysis service that is configured to receive platform logs, see the Observability dashboard.

Prerequisite to enable platform logging

To view IBM Analytics Engine platform logs, you must use the Observability dashboard in IBM Cloud to configure platform logging. See Configuring platform logs through the Observability dashboard for the steps that you need to follow to enable logging through the Observability dashboard.

Logging APIs

After you have enabled platform logging through the Observability dashboard, proceed with the steps in the following sections to learn how to use the API endpoints to enable logging, disable logging, and view the logging configuration.

The logging API calls require the GUID of the service instance. If you didn't make a note of the GUID, see Retrieving the GUID of a serverless instance.

Now, two logging API endpoints are exposed:

log_forwarding_config
logging. This API endpoint is deprecated and will be discontinued soon. Although you may use this endpoint, you should start using the log_forwarding_config endpoint.

Enabling log forwarding from IBM Analytics Engine

To enable forwarding IBM Analytics Engine platform logs to IBM Log Analysis, invoke the following API against an existing Analytics Engine instance.

There are two methods that you can use to enable forwarding logs:

Default: In this case you only put a request for enabling log forwarding, and the logs related to the default log source or sources (spark-driver), will start getting forwarded. You can do this in one of the following ways:

By using the log_forwarding_config API endpoint:

curl -X PUT https://api.us-south.ae.cloud.ibm.com/v3/analytics_engines/<instance_guid>/log_forwarding_config -H "Authorization: Bearer $TOKEN" -H "content-type: application/json" -d '{"enabled":true}'

By using the logging API endpoint (deprecated):

curl -X PUT https://api.us-south.ae.cloud.ibm.com/v3/analytics_engines/<instance_guid>/logging -H "Authorization: Bearer $TOKEN" -H "content-type: application/json" -d '{"enable":true}

Customized: With this option, you get more control over the log forwarding configuration. For example, you can specify the sources of the logs you want to get forwarded. You can also choose to associate some tags with the logs, which can later be used in IBM Log Analysis as search keywords to narrow down the search on the applicable logs. You can do this in one of the following ways:

By using the log_forwarding_config API endpoint:

curl -X PUT https://api.us-south.ae.cloud.ibm.com/v3/analytics_engines/<instance_guid>/log_forwarding_config -H "Authorization: Bearer $TOKEN" -H "content-type: application/json" -d ' {"enabled":true,"sources": ["spark-driver", "spark-executor"],"tags": ["<tag_1>", "<tag_2>",...]} '

By using the logging API endpoint (deprecated):

curl -X PUT https://api.us-south.ae.cloud.ibm.com/v3/analytics_engines/<instance_guid>/logging -H "Authorization: Bearer $TOKEN" -H "content-type: application/json" -d ' {"enable":true,"components": ["spark-driver", "spark-executor"],"tags": ["<tag_1>", "<tag_2>",...]} '

Disabling platform logging from IBM Analytics Engine

Disable forwarding IBM Analytics Engine platform logs to IBM Log Analysis, in one of the following ways.

By using the log_forwarding_config API endpoint:

curl -X PATCH https://api.us-south.ae.cloud.ibm.com/v3/analytics_engines/<instance_guid>/log_forwarding_config -H "Authorization: Bearer $TOKEN" -H "content-type: application/json" -d '{"enabled":false}'

By using the logging API endpoint (deprecated):

curl -X PATCH https://api.us-south.ae.cloud.ibm.com/v3/analytics_engines/<instance_guid>/logging -H "Authorization: Bearer $TOKEN" -H "content-type: application/json" -d '{"enable":false}'

Viewing the log forwarding configuration of an IBM Analytics Engine instance

View the current logging configuration of an IBM Analytics Engine instance, in one of the following ways:

By using the log_forwarding_config API endpoint:

curl -X GET https://api.us-south.ae.cloud.ibm.com/v3/analytics_engines/<instance_guid>/log_forwarding_config -H "Authorization: Bearer $TOKEN"

By using the logging API endpoint (deprecated):

curl -X GET https://api.us-south.ae.cloud.ibm.com/v3/analytics_engines/<instance_guid>/logging -H "Authorization: Bearer $TOKEN"

Examples of IBM Analytics Engine log searches in the Log Analysis dashboard

You can narrow down the search for the platform logs related to IBM Analytics Engine as follows:

Click Sources on Log Analysis dashboard and choose ibmanalyticsengine to get the logs from IBM Analytics Engine.
To further narrow down your search, you can enter and search for any keywords related to your Spark application in the search window.

Examples of searches

This section lists examples of how you can search for logs. The examples all use the following sample values:

IBM Analytics Engine instance ID: 4308f7a6-2cfd-4d7e-b7a7-678e71a27957
Spark application ID: 5438585f-6413-4bb2-9251-8162d34f3dc3
Sources for which log forwarding is enabled: spark-driver and spark-executor
Associated tags: prod and us-south

The following image shows you an example of the returned search results in the Log Analysis dashboard. At the bottom of the image you can see the search entry field where you enter your search queries. In the example, the application ID 5438585f-6413-4bb2-9251-8162d34f3dc3 and the source spark-driver were entered.

Shows the search window for on the Log Analysis dashboard. — Figure 1. Log Analysis dashboard displaying search results

Examples of searches:

To retrieve the logs for all Spark applications submitted in a specific IBM Analytics Engine instance, pass the instance ID in the search window:
```
4308f7a6-2cfd-4d7e-b7a7-678e71a27957
```
To fetch the logs for a specific Spark application, you can pass:
- A combination of the IBM Analytics Engine instance ID and the Spark application ID:
```
4308f7a6-2cfd-4d7e-b7a7-678e71a27957 5438585f-6413-4bb2-9251-8162d34f3dc3
```
- Only the application ID:
```
5438585f-6413-4bb2-9251-8162d34f3dc3
```
To search the Spark master logs of an application, you can pass:
- A combination of the IBM Analytics Engine instance ID, the Spark application ID and the keyword spark-driver:
```
4308f7a6-2cfd-4d7e-b7a7-678e71a27957 5438585f-6413-4bb2-9251-8162d34f3dc3 spark-driver
```
- A combination of the application ID and the keyword spark-driver:
```
5438585f-6413-4bb2-9251-8162d34f3dc3 spark-driver
```
To search by multiple log source names, you can pass:
- A combination of the IBM Analytics Engine instance ID, the Spark application ID and the source names:
```
4308f7a6-2cfd-4d7e-b7a7-678e71a27957 5438585f-6413-4bb2-9251-8162d34f3dc3 (spark-driver OR spark-executor-2)
```
- A combination of the application ID and the log source names:
```
5438585f-6413-4bb2-9251-8162d34f3dc3 (spark-driver OR spark-executor-2)
```
To get the logs related to all the applications running in a particular instance within the us-south region and the prod environment, you can enter the following in the search field:
```
4308f7a6-2cfd-4d7e-b7a7-678e71a27957 "prod us-south"
```