Monitoring operational logs
You can configure the Logging agent to collect and send infrastructure and application logs to an IBM Cloud Logs instance directly.
The Logging agent is based on the Fluent Bit open-source agent which is used to collect and process log data. You can deploy the Logging agent in supported environments and manage data from various sources and formats.
The following diagram shows the high level view when the destination is an IBM Cloud Logs instance:
When you can configure the Logging agent to send logs to the IBM Cloud Logs service, logs that you send must include a value for the applicationName and subsystemName metadata fields. By default, when you configure the
Logging agent, the agent sets default values for these fields. You can configure your own custom values to replace the default values.
- For more information on default values, see Metadata fields.
- For more information on how to configure the agent, see Configuring the agent to set custom values for applicationName and subsystemName metadata fields.
Logging agent for orchestrated environments
You can deploy the Logging agent on a Red Hat OpenShift on IBM Cloud or IBM Cloud Kubernetes Service cluster.
You can deploy the agent on clusters that you run on-prem, in IBM Cloud, or in a different cloud.
The Logging agent is a daemon set that is designed to have one pod running on each node of a cluster. Each pod will collect relevant logs for the node its running on. The Logging agent will then forward those logs to the IBM Cloud Logs service.
By default, the Logging agent monitors and collects log data from files matching the specified path pattern in /var/log/containers/, excluding logs from files matching the exclusion pattern. The refresh interval is set to 10 seconds.
You can change these values and more in the config map logger-agent-config.
You can deploy the agent in the following platforms:
-
Kubernetes clusters
For more information, see Deploying the Logging agent on a Kubernetes cluster using a Helm chart.
-
OpenShift clusters
For more information, see Deploying the Logging agent on OpenShift using a Helm chart.
The following diagram shows the high level view when the source of logs is a Kubernetes or OpenShift cluster:
Logging agent for non-orchestarted environments
You can deploy the Logging agent in Linux environments.
The following platforms are supported:
- RHEL 8
- RHEL 9
- Ubuntu 20
- Ubuntu 22
- Debian 11
- Debian 12
For more information, see Managing the agent Linux environments.
Supported formats
The agent supports the following input formats:
- JSON
- apache
- apache2
- apache_error
- nginx
- docker (JSON with the docker-specific timestamp format)
- cri
- syslog
Supported agent versions
The following table lists the agent versions that are supported and the version of Fluent Bit the agent is based on:
| Logging agent | Based on Fluent Bit Version | Helm chart version |
|---|---|---|
| v1.3.2 | v3.1.9 | v1.3.2 |
| v1.3.1 | v3.1.4 | v1.3.1 |
| v1.3.0 | v3.1.4 | v1.3.0 |
For information on recommended and supported Fluent Bit plug-ins see Fluent Bit support
Agent support policy
Release agent updates are planned on a quarterly basis. Support will continue to provide assistance for two releases prior to the latest release.
For example, if agent version 1.3.x is the most currently release, then questions related to agents 1.2.x and 1.1.x will be answered.
However, new functions and security fixes will only be made available as the most current release and modification level.
For example, if a security vulnerability is found in the agent, and the current agent version is 1.3.3, the security vulnerability will be fixed and released as 1.3.4. Even though 1.2.x and 1.1.x are still supported for technical questions, patches to these releases will not be made available.