Configuring the IBM Cloud Logs Routing service to route platform logs to an IBM Cloud Logs instance in a region

Goals

In this tutorial you will:

• Configure IBM Cloud Logs Routing to route platform logs to an IBM Cloud Logs instance in a region

• Verify that platform logs are being routed to your IBM Cloud Logs instance.

Before you begin

Complete the following steps:

1. Review About IBM® Cloud Logs Routing to understand concepts.

2. Install all prerequisite tools as described in the getting started.

3. To get details on a tenant by using the API, check that you can connect to IBM Cloud Logs Routing by using the management API. For more information, see Connecting to IBM® Cloud Logs Routing.

4. Platform logs that are routed to IBM Cloud Logs include metadata fields that you can use to manage the data and configure features in your IBM Cloud Logs instance.

   The application name is the environment that produces and sends logs to IBM Cloud Logs. Platform logs have the application name set to ibm-platform-log.

   The subsystem name is the service or application that produces and sends logs, or metrics to IBM Cloud Logs. Platform logs have the subsystem name set to CRNserviceName:instanceID. For VPC platform logs, the fields is set to is:resourceType.

5. Set up permissions to manage targets in the account. For more information, see Setting up IAM permissions for managing tenants.

   Before you can set up the IBM Cloud Logs Routing service to route your platform logs, you need an IBM Cloud® Identity and Access Management (IAM) access token for authentication. The IBM Cloud CLI is used to obtain this information.

   1. Log in to your IBM Cloud account. Include the --sso option if you are using a federated ID.

      ibmcloud login [--sso]
      

      The output will include your username similar to:

      Targeted account Jane Doe\'s Account (111111111112a21af53a2aeed19a1542a44) <-> 11111156
      
      
       API endpoint:     https://cloud.ibm.com
       Region:           us-south
       User:             Jane.Doe@ibm.com
       Account:          Jane Doe\'s Account (111111111112a21af53a2aeed19a1542a44) <-> 11111156
       Resource group:   No resource group targeted, use 'ibmcloud target -g RESOURCE_GROUP'
      

   2. To grant access for using IBM Cloud Logs Routing to your user, collect the username from the above generated output. In our example, this would be Jane.Doe@ibm.com. Replace <username> with your username in the following command:

      ibmcloud iam user-policy-create <username> --roles Manager --service-name logs-router
      

      Instead of assigning roles directly to identities, a common strategy is to assign roles to access groups, and add identities as members to those access groups. For more information about access groups, see setting up access groups.

Creating a service to service authorization

You must use IBM Cloud® Identity and Access Management (IAM) to create an authorization that grants IBM Cloud Logs Routing access to IBM Cloud Logs.

Retrieving your IBM Cloud Logs information

To route data to an IBM Cloud Logs instance, you need information about the destination where you want logs delivered. You need the following information for your IBM Log Analysis instance:

• The instance CRN
• The ingestion (ingress) endpoint and port

Only public ingress endpoints are supported when configuring a target to an IBM Cloud Logs instance.

Use this command to retrieve the URL and ID:

ibmcloud resource service-instances --service-name logs --long --output JSON | jq '[.[] | {name: .name, id: .id, region: .region_id, ingestion_endpoint: .extensions.external_ingress}]'

If you have multiple instances, choose one you would like to use for receiving platform logs and save the information for later.

The endpoint is similar to:

ingestion_endpoint: 3a622101-7521-4002-bf91-8c26e17eedcf.ingress.eu-de.logs.cloud.ibm.com

The ID is similar to:

"crn:v1:bluemix:public:logs:eu-es:a/15ff25632a21af53a2aeed19a1542a44:3a622101-7521-4002-bf91-8c26e17eedcf::"

To get all the information about the instance, you can run ibmcloud resource service-instances --service-name logs --long --output JSON This command lists all of your IBM Cloud Logs instances. You need the following fields: IDthat contains the instance's CRN, extensions.external_ingress that contains the ingress endpoint, and extensions.virtual_private_endpoints.endpoints.ports.port_max that contains the port value.

Creating a tenant and target

To receive your platform logs, you need to create a tenant with the IBM Cloud Logs Routing service.

You must create your tenant in the region where you want to collect and route platform logs to the IBM Cloud Logs instance.

curl -X POST https://<MANAGEMENT-API-ENDPOINT>:443/v1/tenants \
-H "Content-Type: application/json" \
-H "Authorization: ${IAM_TOKEN}" \
-H 'IBM-API-Version: DATE' \
--data '{
    "name": "TENANT_NAME",
    "targets": [
        {
            "log_sink_crn": "CLOUD_LOGS_INSTANCE_ID",
            "name": "TARGET_NAME",
            "parameters": {
                "host": "CLOUD_LOGS_INSTANCE_INGRESS_ENDPOINT",
                "port": PORT
            }
        }
    ]
}'

{
    "id": "3517d2ed-9429-af34-ad52-34278391cbc8:",
    "created_at": "2023-10-20T18:30:00.143156Z",
    "updated_at": "2023-10-20T18:30:00.143156Z",
    "crn": "crn:v1:bluemix:public:logs-router:eu-de:a/3516b8fa0a174a71899f5affa4f18d78:3517d2ed-9429-af34-ad52-34278391cbc8::",
    "name": "tenant",
    "etag": "\"822b4b5423e225206c1d75666595714a11925cd0f82b229839864443d6c3c049\"",
    "targets": [
    {
        "id": "86432b-66a6-df12-7003-888a21a2b3",
        "log_sink_crn": "crn:v1:bluemix:public:logs:eu-gb:a/91d1d1e42f9b684df920bbd06461c222:743e3b4f-72bc-4669-9fa6-0e6621d0b232::",
        "name": "my-log-sink",
        "etag": "\"c3a43545a7f2675970671ac3a57b8db067a1866b2222e1b950ee8da612e347c6\"",
        "type": "logs",
        "created_at": "2023-10-20T18:30:00.143156Z",
        "updated_at": "2023-10-20T18:30:00.143156Z",
        "parameters": {
            "host": "743e3b4f-72bc-4669-9fa6-0e6621d0b232.ingress.eu-gb.logs.cloud.ibm.com",
            "port": 443
        }
    }
    ]
}

Verifying that logs are sent to the destination target

To ensure that your platform logs are successfully flowing to IBM Cloud Logs, do the following steps:

1. Launch the IBM Cloud Logs web UI for the IBM Cloud Logs instance that is configured as the target to collect platform logs in a region. This is the instance that you selected in the step where you setup a target.

2. View logs through custom views. For more information, see Viewing logs.

All platform logs are generated in JSON. You can filter platform logs in your instance be selecting the value of ibm-platform-logs as the applicationName.