IBM Cloud Docs
Configuring the IBM Cloud Logs Routing service to route platform logs to an IBM Log Analysis instance in a region

Configuring the IBM Cloud Logs Routing service to route platform logs to an IBM Log Analysis instance in a region

Use the IBM Cloud Logs Routing service to route platform logs from your IBM Cloud account to an IBM Log Analysis instance target destination. You must configure a tenant in the region and a target destination.

A tenant is the account-specific configuration of IBM Cloud Logs Routing running within a region. The tenant configuration includes a maximum of 2 target definitions. The target defines where the logs are routed.

You must create a tenant in your account in each region where you want to use IBM Cloud Logs Routing. Each region is independent and regions do not share data.

Goals

In this tutorial you will:

  • Configure IBM Cloud Logs Routing to route platform logs to an IBM Log Analysis instance in a region

  • Verify that platform logs are being routed to your IBM Log Analysis instance.

Before you begin

Complete the following steps:

  1. Review About IBM® Cloud Logs Routing to understand concepts.

  2. Install all prerequisite tools as described in the getting started.

  3. To get details on a tenant by using the API, check that you can connect to IBM Cloud Logs Routing by using the management API. For more information, see Connecting to IBM® Cloud Logs Routing.

  4. Platform logs that are routed to IBM Cloud Logs include metadata fields that you can use to manage the data and configure features in your IBM Cloud Logs instance.

    The application name is the environment that produces and sends logs to IBM Cloud Logs. Platform logs have the application name set to ibm-platform-log.

    The subsystem name is the service or application that produces and sends logs, or metrics to IBM Cloud Logs. Platform logs have the subsystem name set to CRNserviceName:instanceID. For VPC platform logs, the fields is set to is:resourceType.

  5. Set up permissions to manage targets in the account. For more information, see Setting up IAM permissions for managing tenants.

    Before you can set up the IBM Cloud Logs Routing service to route your platform logs, you need an IBM Cloud® Identity and Access Management (IAM) access token for authentication. The IBM Cloud CLI is used to obtain this information.

    1. Log in to your IBM Cloud account. Include the --sso option if you are using a federated ID.

      ibmcloud login [--sso]
      

      The output will include your username similar to:

      Targeted account Jane Doe\'s Account (111111111112a21af53a2aeed19a1542a44) <-> 11111156
      
      
       API endpoint:     https://cloud.ibm.com
       Region:           us-south
       User:             Jane.Doe@ibm.com
       Account:          Jane Doe\'s Account (111111111112a21af53a2aeed19a1542a44) <-> 11111156
       Resource group:   No resource group targeted, use 'ibmcloud target -g RESOURCE_GROUP'
      
    2. To grant access for using IBM Cloud Logs Routing to your user, collect the username from the above generated output. In our example, this would be Jane.Doe@ibm.com. Replace <username> with your username in the following command:

      ibmcloud iam user-policy-create <username> --roles Manager --service-name logs-router
      

      Instead of assigning roles directly to identities, a common strategy is to assign roles to access groups, and add identities as members to those access groups. For more information about access groups, see setting up access groups.

Creating a service to service authorization

You must use IBM Cloud® Identity and Access Management (IAM) to create an authorization that grants IBM Cloud Logs Routing access to IBM Cloud Logs.

Creating a service to service authorization through the UI

Complete the following steps:

  1. Log in to your IBM Cloud account.

  2. In the IBM Cloud console, click Manage > Access (IAM), and select Authorizations.

  3. Click Create.

  4. Configure the source account. Select This account.

  5. Select Logs Routing as the source service. Then, set the scope of the access to All resources.

  6. Select Cloud Logs as the target service. Then, set the scope of the access to All resources, which grants access to all IBM Cloud Logs instances, or a single instance by configuring Resources based on selected attributes > Service Instance.

    Other attributes are not supported for this type of authorization.

  7. In the Service Access section, select Sender to assign access to the source service that accesses the target service.

  8. Click Authorize.

Creating a service to service authorization by using the CLI

You must define a service to service (S2S) authorization between IBM Cloud Logs and IBM® Cloud Logs Routing so the IBM® Cloud Logs Routing service can send logs to your tenant.

Complete the following steps:

  1. Retrieving the IAM bearer token and export it as an environment variable. Run the following command:

    You must get an IBM Cloud® Identity and Access Management (IAM) access token to authenticate your requests to the IBM® Cloud Logs Routing service. For more information, see Retrieving an access token.

    For example, you can retrieve your IAM bearer token and export it as an environment variable by running the following CLI command:

    export IAM_TOKEN=`ibmcloud iam oauth-tokens --output json | jq -r '.iam_token'`
    

    Verify that your token has been correctly exported by displaying it in your terminal.

    echo $IAM_TOKEN
    

    The output should start with Bearer followed by a string containing letters, numbers, and symbols.

  2. Create an authorization policy.

    ibmcloud iam authorization-policy-create logs-router logs Sender
    

For more information, see Creating a S2S authorization to grant access to send logs to IBM Cloud Logs.

Retrieving your IBM Log Analysis information

To create a tenant with a target destination of tyle logdna, you must supply information about the destination where you want logs delivered. You need to supply the following information for your IBM Log Analysis instance:

  • The instance CRN
  • The ingestion key
  • The endpoint (hostname and port)

To obtain the instance CRN for the IBM Log Analysis instance, run this command:

ibmcloud resource service-instances --service-name logdna --long

This command lists all of your Log Analysis instances. You can find the CRN for each in the ID field, for example:

ID:                 crn:v1:bluemix:public:logdna:us-east:a/473958g47b35f95747:48b580c-34ad-c985-1g2g-e1g75b1a2b3c::
GUID:               48b580c-34ad-c985-1g2g-e1g75b1a2b3c
Name                Myinstance
Location            us-east

To retrieve the ingestion key for your instance, follow these steps.

To determine the correct Log Analysis endpoint information, including the hostname and port, see the list of Log Analysis endpoints.

Creating a tenant

To receive your platform logs, you need to create a tenant with the IBM Cloud Logs Routing service.

You must create your tenant in the region where you want to collect and route platform logs to the Log Analysis instance.

Creating a tenant by using the API

Run the following cURL request from the command line:

The create request creates the tenant in the region and the destination.

curl -X POST https://<MANAGEMENT-API-ENDPOINT>:<PORT>/v1/tenants \
-H "Content-Type: application/json" \
-H "Authorization: ${IAM_TOKEN}" \
-H 'IBM-API-Version: CURRENT_DATE' \
--data "{
    "name": "TENANT_NAME",
    "targets": [
        {
            "log_sink_crn": "LOG_ANALYSIS_INSTANCE_CRN",
            "name": "TARGET_NAME",
            "parameters": {
                "host": "LOG_ANALYSIS_INGESTION_ENDPOINT",
                "access_credential": "LOG_ANALYSIS_INSTANCE_INGESTION_KEY",
                "port": PORT
            }
        }
    ]
}'

Where

Parameter descriptions
Parameter Description
MANAGEMENT-API-ENDPOINT The endpoint of the IBM® Cloud Logs Routing service. For more information, see Endpoints. For example, https://management.eu-es.logs-router.cloud.ibm.com
IAM_TOKEN The IAM Token you obtained previously. If you exported it in your environment as described above, it is replaced automatically.
DATE The current date. For example, 2024-03-01
TENANT_NAME Name of the tenant. The name must be unique across tenants for this account and can be up to 35 characters long. The value can only contain these characters: a-z,0-9,-./ An example would be eu-es-tenant.
TARGET_NAME Name of the target destination. The name must be unique across all targets within a region and can be up to 35 characters long. The value can only contain these characters: a-z,0-9,-./ You can for example choose the name of your IBM Cloud Logs instance. An example would be platformlogs-eu-es.
LOG_ANALYSIS_INSTANCE_CRN The CRN of your IBM Log Analysis instance.
LOG_ANALYSIS_INGESTION_ENDPOINT The endpoint of your IBM Log Analysis instance. You can choose a public or a private ingestion endpoint. For more information, see IBM Log Analysis ingestion endpoints.
LOG_ANALYSIS_INSTANCE_INGESTION_KEY The ingestion key of the target IBM Log Analysis instance.
PORT Set to 443

The following example shows the creation of a tenant to IBM Cloud Logs Routing in the us-east region by using a VPE, and specifying target information for a IBM Log Analysis instance that is also in us-east:

curl -X POST "https://management.private.us-east.logs-router.cloud.ibm.com/v1/tenants" \
--header "Authorization: Bearer TOKEN" \
--header 'Content-Type: application/json' \
--header 'IBM-API-Version: 2024-03-01' \
--data '{
    "name": "tenant",
    "targets": [
        {
            "log_sink_crn": "crn:v1:bluemix:public:logdna:us-east:a/3516b8fa0a174a71899f5affa4f18d78:3517d2ed-9429-af34-ad52-34278391cbc8::",
            "name": "my-log-sink",
            "parameters": {
                "host": "logs.us-east.logging.cloud.ibm.com",
                "port": 443,
                "access_credential": "an-ingestion-secret"
            }
        }
    ]
}'

If the creation (onboarding) request was successful, a response that contains your tenant metadata is returned. For example,

{
    "id": "97543c-77b7-eg23-8114-999b31a2b3",
    "created_at": "2023-10-20T18:30:00.143156Z",
    "updated_at": "2023-10-20T18:30:00.143156Z",
    "crn": "crn:v1:bluemix:public:logs-router:eu-de:a/3516b8fa0a174a71899f5affa4f18d78:3517d2ed-9429-af34-ad52-34278391cbc8::",
    "name": "tenant",
    "etag": "\"822b4b5423e225206c1d75666595714a11925cd0f82b229839864443d6c3c049\"",
    "targets": [
    {
        "id": "86432b-66a6-df12-7003-888a21a2b3",
        "log_sink_crn": "crn:v1:bluemix:public:logdna:us-east:a/473958g47b35f95747:48b580c-34ad-c985-1g2g-e1g75b71a2b3::",
        "name": "my-log-sink",
        "etag": "\"c3a43545a7f2675970671ac3a57b8db067a1866b2222e1b950ee8da612e347c6\"",
        "type": "logdna",
        "created_at": "2023-10-20T18:30:00.143156Z",
        "updated_at": "2023-10-20T18:30:00.143156Z",
        "parameters": {
            "host": "logs.us-east.logging.cloud.ibm.com",
            "port": 443
        }
    }
    ]
}

Creating a tenant through the UI

When the IBM Cloud Logs Routing console is first displayed, any existing target information is displayed.

If no target is configured for a region, the region displays the Set target option. When the target is set for the first time, an IBM Cloud Logs Routing tenant is created (onboarded) and the target configured.

To set a target for a IBM Cloud Logs Routing tenant in a region, complete the following steps:

  1. Log in to your IBM Cloud account.

    After you log in with your user ID and password, the IBM Cloud dashboard opens.

  2. Click the Menu icon Menu icon > Observability.

  3. Click Logging > Routing.

    The IBM Cloud Logs Routing console is displayed with your current configurations.

  4. In the Rules section, configure a rule to create a tenant in a region. Choose the region where you want to create a tenant, then select Set target.

  5. Click Set target. Then, configure a target destination by selecting an IBM Log Analysis instance to receive logs routed by IBM Cloud Logs Routing.

    You can choose to configure a target by searching for an instance in your IBM Cloud account, or by entering the details of the instance.

    Option 1 Search by instance: You can select an IBM Log Analysis instance by selecting an instance from the list and providing the instance ingestion key or by specifying an IBM Log Analysis CRN and ingestion key. You can only choose instances that are available in the same IBM Cloud account where you are configuring the tenant.

    Option 2 Specify CRN: You can choose an instance that is available in the same IBM Cloud account where you are configuring the tenant or an instance that is available in a different account. When you choose this option, you must enter the CRN of the target instance. If you want to route to an IBM Log Analysis instance in another account, you must select the target by CRN (Cloud Resource Name) and ingestion key. The CRN of an IBM Log Analysis instance can be found by the account administrator of the IBM Log Analysis instance by clicking Menu icon > Resource list and clicking the IBM Log Analysis instance. The CRN can be copied from the Details section.

  6. Click Save.

Verifying that logs are sent to the destination target

To ensure that your platform logs are successfully flowing to IBM Log Analysis instance, do the following steps:

  1. Launch the web UI for the IBM Log Analysis instance that is configured as the target to collect platform logs in a region. This is the instance that you selected in the step where you setup a target.

  2. View logs through custom views. For more information, see Viewing logs.

All platform logs are generated in JSON. You can filter platform logs in your instance be selecting the value of ibm-platform-logs as the applicationName.