Excluding log files

Configure a logging agent to exclude logs that you do not want to monitor through the logging UI.

As of 28 March 2024 the IBM Log Analysis and IBM Cloud Activity Tracker services are deprecated and will no longer be supported as of 30 March 2025. Customers will need to migrate to IBM Cloud Logs, which replaces these two services, prior to 30 March 2025. IBM Cloud Logs will become generally available during the summer of 2024 in Frankfurt and Madrid with day-one support for EU-managed controls. The service will continue its worldwide multizone region (MZR) roll-out through 3Q2024.

You can exclude files that are located in any of the paths that are defined through the logdir parameter in a Linux system or the LOGDNA_EXCLUDE variable in a Kubernetes cluster.
You can configure multiple files. You separate multiple files by using commas.
You can use glob patterns to define what you want to exclude.
You can configure specific files.

Excluding log files for a standard Kubernetes cluster

Complete the following steps to configure the agent so that only application logs are forwarded and cluster logs are excluded:

Step 1. Set the context of the cluster

Complete the following steps:

Open a terminal to log in to IBM Cloud.
```
ibmcloud login -a cloud.ibm.com --sso
```
Select the account where you provisioned the IBM Log Analysis instance.
List the clusters to find out in which region and resource group the cluster is available.
```
ibmcloud ks clusters
```
Set the resource group and region.
```
ibmcloud target -g RESOURCE_GROUP -r REGION
```
Where

RESOURCE_GROUP is the name of the resource group where the cluster is available, for example, default.

REGION is the region where the cluster is available, for example, us-south.
Set the cluster where you want to configure logging as the context for this session.
```
ibmcloud ks cluster config --cluster <cluster_name_or_ID>
```
Where <cluster_name_or_ID> is the name or the ID of the cluster.

Step 2. Modify the logging agent YAML file

Complete the following steps:

Generate the configuration file of the agent by running the following command:

kubectl get daemonset logdna-agent -o=yaml > prod-agent-resources.yaml -n ibm-observe

Make changes. Add the section LOGDNA_EXCLUDE to the YAML file.

To exclude all cluster logs, you can add:

- name: LOGDNA_EXCLUDE
value: /var/log/containers/*_kube-system_*,/var/log/containers/*ibm-observe_*,/var/log/containerd.log,/var/log/kubelet.log,/var/log/syslog,/var/log/ntpstats/*,/var/log/alb/*

To exclude logs by namespace, for example, all of the kube-system logs, add:

- name: LOGDNA_EXCLUDE
  value: /var/log/containers/*_kube-system_*

To exclude all non-container logs, that is, logs shown in the All Apps filter view, add:
```
- name: LOGDNA_EXCLUDE
  value: /var/log/!(containers)/**
```

To exclude calico logs, add:

- name: LOGDNA_EXCLUDE
  value: /var/log/containers/calico*

To exclude all of the kube-system logs and all non-container logs, add:

- name: LOGDNA_EXCLUDE
  value: /var/log/!(containers)/**,/var/log/containers/*_kube-system_*

Step 3. Apply the changes to the logging agent

To apply the configuration changes, run the following command:

kubectl apply -f prod-agent-resources.yaml -n ibm-observe

Step 4. Verify the changes

Complete the following steps:

Get the logdna-agent pods and check that pods have restarted. Run the following command:
```
kubectl get pods -n ibm-observe
```
If pods are not restarted, delete all the logging pods.
```
kubectl delete pod PodName -n ibm-observe
```
Launch the logging UI, and verify that log entries are not showing in the logging UI.