Logging with Kubernetes clusters

Before you begin

Work in a supported region. Note: You can send data from a Kubernetes cluster that is located in the same region or in a different region.

Read about IBM Log Analysis. For more information, see About.

Use a user ID that is a member or an owner of an IBM Cloud account. To get an IBM Cloud user ID, go to: Registration.

Your IBMID must have assigned IAM policies for each of the following resources in the region that your IBM Log Analysis instance is in:

For more information about the IBM Cloud® Kubernetes Service IAM roles, see User access permissions.

Install the IBM Cloud CLI and the Kubernetes CLI plug-in. For more information, see Installing the IBM Cloud CLI.

Objectives

In this tutorial, you configure logging with logging for your IBM Cloud Kubernetes Service cluster. In particular, you will:

• Provision an IBM Log Analysis.
• Configure the logging agent in your cluster to start sending logs to LogDNA.
• Open the logging dashboard to find your logs.

Step 1. Provision an IBM Log Analysis service instance

To provision a service instance of IBM Log Analysis through the IBM Cloud console, complete the following steps:

1. Log in to the IBM Cloud account where you created your Kubernetes cluster.

2. Click Catalog. A list of IBM Cloud services opens.

3. To filter the list of services that is displayed, select the Logging and Monitoring category.

4. Click IBM Log Analysis. The Observability dashboard opens.

5. Select Create instance.

6. Enter a name for the service instance and a location.

7. Select the resource group that your cluster is in. By default, the Default resource group is set for you.

8. Choose a service plan for your service instance. By default, the Lite plan is selected for you. For more information about other service plans, see Pricing plans.

9. To provision the IBM Log Analysis service in the IBM Cloud resource group where you are logged in, click Create. The Observability dashboard opens and shows the details for your service.

To provision an instance through the CLI, see Provisioning an instance through the IBM Cloud CLI.

Step 2. Get the ingestion key

Complete the following steps to get the ingestion key:

1. Log in to your IBM Cloud account.

   After you log in, the IBM Cloud UI opens.

2. Click the Menu icon > Observability to access the Observability dashboard.

3. Select Logging. The IBM Log Analysis dashboard opens. You can see the list of logging instances that are available on IBM Cloud.

4. Identify the instance for which you want to get the ingestion key, and click View ingestion key.

5. A window opens where you can click Show to view the ingestion key.

Step3: Configure your Kubernetes cluster to send logs to your logging instance

To configure your Kubernetes cluster to send logs to your IBM Log Analysis instance, you must install a logdna-agent pod on each node of your cluster. The logging agent reads log files from the pod where it is installed, and forwards the log data to your logging instance.

To configure your Kubernetes cluster in the us-south region to forward logs to your logging instance, complete the following steps from the command line:

1. Open a terminal to log in to IBM Cloud.

   ibmcloud login -a cloud.ibm.com
   

   Select the account where you have provisioned the IBM Log Analysis instance.

2. Set the cluster where you want to configure logging as the context for this session.

   ibmcloud ks cluster-config <cluster_name_or_ID>
   

   When the download of the configuration files is finished, a command is displayed that you can use to set the path to the local Kubernetes configuration file as an environment variable. Copy and paste the command that is displayed in your terminal to set the KUBECONFIG environment variable.

   Every time you log in to the IBM Cloud Kubernetes Service CLI to work with your cluster, you must run this setup to set the path to the cluster's configuration file as a session variable. IBM Cloud Kubernetes Service uses this variable to find a local configuration file and certificates that are necessary to connect with your cluster.

3. Create a Kubernetes secret to store your logging ingestion key for your service instance. The logging ingestion key is used to open a secure web socket to the logging ingestion server and to authenticate the logging agent with the IBM Log Analysis service.

   kubectl create secret generic logdna-agent-key --from-literal=logdna-agent-key=<logDNA_ingestion_key>
   

4. Create a Kubernetes daemon set to deploy the logging agent on every worker node of your Kubernetes cluster. The logging agent collects logs with the extension *.log and extensionsless files that are stored in the /var/log directory of your pod. By default, logs are collected from all namespaces, including kube-system, and automatically forwarded to the IBM Log Analysis service.

   kubectl create -f https://assets.us-south.logging.cloud.ibm.com/clients/agent-resources.yaml
   

5. Verify that the logging agent is deployed successfully.

   kubectl get pods
   

   The deployment is successful when you see one or more logging pods. The number of logging pods equals the number of worker nodes in your cluster. All pods must be in a Running state.

Step 4: Launch the logging dashboard and view logs

To launch the logging dashboard through the IBM Cloud console, complete the following steps:

1. Log in to your IBM Cloud account.

2. From the menu , select Observability.

3. Select Logging. The list of IBM Log Analysis service instances that are available on IBM Cloud is displayed.

4. Select one instance and click View IBM Log Analysis. The logging dashboard opens. Note: With the Free service plan, you can tail your latest logs only. For more information, see Viewing logs.

Next steps

• Filter logs
• Search logs
• Define views

Note: Some of these features require a plan upgrade.