The Logging agent

About the Logging agent

When you use the Logging agent to send logs to the IBM Cloud Logs service, logs that you send must include a value for the applicationName and subsystemName metadata fields. By default, when you configure the Logging agent, the agent sets default values for these fields. You can configure your own custom values to replace the default values.

• For more information on default values, see Metadata fields.
• For more information on how to configure the agent, see Configuring the agent to set custom values for applicationName and subsystemName metadata fields.

The severity of a log record is identified by the Logging agent as follows:

1. If the log is parsed by the Logging agent and contains a field named severity, level or logLevel, that field is used to set the severity.
2. If the log record does not contain a field named severity, level or logLevel, the message field is scanned to set the level based on a text search for debug, info, error, or warn.

If the Logging agent cannot determine the severity of a log record, it sets the severity to info.

Logging agent for orchestrated environments

You can deploy the Logging agent on a Red Hat OpenShift on IBM Cloud or IBM Cloud Kubernetes Service cluster.

You can deploy the agent on clusters that you run on-prem, in IBM Cloud, or in a different cloud.

The Logging agent is a daemon set that is designed to have one pod running on each node of a cluster. Each pod will collect relevant logs for the node its running on. The Logging agent will then forward those logs to the IBM Cloud Logs service.

By default, the Logging agent monitors and collects log data from files matching the specified path pattern in /var/log/containers/, excluding logs from files matching the exclusion pattern. The refresh interval is set to 10 seconds. You can change these values and more in the config map logger-agent-config. For more information, see Filtering logs.

You can deploy the agent in the following platforms:

• Kubernetes clusters

  For more information, see Deploying the Logging agent on a Kubernetes cluster using a Helm chart.

• OpenShift clusters

  For more information, see Deploying the Logging agent on OpenShift using a Helm chart.

The following diagram shows the high level view when the source of logs is a Kubernetes or OpenShift cluster:

Logging agent for non-orchestarted environments

You can deploy the Logging agent in Linux and Windows environments.

You must configure the log files that the Logging agent monitors and forwards its data to your IBM Cloud Logs instance.

The agent runs as a Linux daemon by using systemd. You can manage the agent by using systemctl.

The following platforms are supported:

• RHEL 8
• RHEL 9
• Ubuntu 20
• Ubuntu 22
• Debian 11
• Debian 12
• Windows Server 2016
• Windows Server 2019
• Windows Server 2022
• Windows 10 Enterprise
• Windows 11 Enterprise

For more information, see Managing the agent in Linux environments or Managing the agent in Windows environments.

Authorization methods

You can use a service ID or a trusted profile as the identity that is used by the agent to authenticate with the IBM® Cloud Logs service.

Choose a supported authorization method for the environment where you plan to deploy the agent:

You can only use Trusted Profiles to authenticate IBM Cloud resources with an IBM Cloud Logs instance when the compute resource and the instance are located in the same account.

To send logs from a Kubernetes cluster that is provisioned in a different IBM Cloud account than the IBM Cloud Logs instance, you can only use a service ID API key as the agent's authorization method.

For more information on how to generate an API key, see Generating an API Key for ingestion by using a service ID for authentication.

For more information on how to create the Trusted Profile, see Generating a Trusted Profile for ingestion.

Supported formats

The agent supports the following input formats:

• JSON
• apache
• apache2
• apache_error
• nginx
• docker (JSON with the docker-specific timestamp format)
• cri
• syslog

Supported agent versions

The following table lists the agent versions that are supported and the version of Fluent Bit the agent is based on:

For information on recommended and supported Fluent Bit plug-ins see Fluent Bit support

Agent support policy

Release agent updates are planned on a quarterly basis. Support will continue to provide assistance for two releases prior to the latest release.

For example, if agent version 1.3.x is the most currently release, then questions related to agents 1.2.x and 1.1.x will be answered.

However, new functions and security fixes will only be made available as the most current release and modification level.

For example, if a security vulnerability is found in the agent, and the current agent version is 1.3.3, the security vulnerability will be fixed and released as 1.3.4. Even though 1.2.x and 1.1.x are still supported for technical questions, patches to these releases will not be made available.