Logging with bare metal servers

Use the IBM® Log Analysis service to monitor and manage logs from a bare metal server in a centralized logging system on the IBM Cloud.

As of 28 March 2024 the IBM Log Analysis and IBM Cloud Activity Tracker services are deprecated and will no longer be supported as of 30 March 2025. Customers will need to migrate to IBM Cloud Logs, which replaces these two services, prior to 30 March 2025.

These instructions are for Ubuntu Linux systems but can be used for other Linux systems.

You can collect and monitor system and application logs.

By default, the logging agent for Ubuntu monitors log files in the /var/log directory. For example, the Ubuntu system log (/var/log/syslog) is monitored by default.

On the IBM Cloud, configure a bare metal server to forward logs to an IBM Log Analysis instance by completing the following steps:

Provision a bare metal server running Ubuntu Linux.
Provision an instance of the IBM Log Analysis service.
Configure the logging agent in the bare metal server.
Optionally, add additional directories to be monitored by the agent.

Figure 1. Component overview on the IBM Cloud

In this tutorial, you will learn how to configure a bare metal server to forward logs to an IBM Log Analysis instance.

Before you begin

Read about IBM Log Analysis. For more information, see About.

Work in a supported region.

You can send data from an Ubuntu instance that is located in the same region as your logging instance, in a different region, or not in the IBM Cloud.

Use a user ID that is a member, or an owner of, an IBM Cloud account. To get an IBM Cloud IBMID, go to: Create an account.

Your IBMID must have assigned IAM policies for each of the following resources in the region that your IBM Log Analysis instance is in:

Your IBMID must have assigned IAM policies for each of the following resources:

Table 1. List of IAM policies required to complete the tutorial
Resource	Scope of the access policy	Role	Region	Information
Resource group default	Resource group	Viewer	us-south	This policy is required to allow the user to see service instances in the Default resource group.
IBM Log Analysis service	Resource group	Editor	us-south	This policy is required to allow the user to provision and administer the IBM Log Analysis service in the default resource group.

The IBM Cloud CLI must be installed. For more information, see Installing the IBM Cloud CLI.

Provision a bare metal server

If you have a bare metal server that you want to monitor, you can skip this step.

If you don't have a bare metal server, complete the following steps

Provision a bare metal server.

To complete the steps in this topic, ensure you have internet access from the bare metal server. This is needed for configuring the monitoring agent.
Configure a VPN connection between your terminal and the bare metal server

Virtual Private Networking (VPN) access enables users to manage all servers remotely and securely over the IBM Cloud® private network. A VPN connection from your location to the private network allows out-of-band management and server rescue through an encrypted VPN tunnel. VPN tunnels can be initiated to any IBM Cloud data center or PoP allowing you geographic redundancy.

Complete the following steps to configure a VPN connection between your terminal and the bare metal server:
1. Enable VPN access on each account that needs VPN access.
2. Depending on your operating system, download the latest MotionPro 32-bit or 64-bit files from the Array Networks Clients and Tools download site. Learn more.
3. Configure a standalone SSL VPN client and open a connection:
For example, if you use the MotionPro Plus client for MacOS, to add a profile, click Add.

In the Basic section, enter a Title. Enter a Gateway, for example, for a bare metal in Dallas 10, enter vpn.dal10.softlayer.com. Enter your VPN user name. Check that the Port is set to 443. Then, click OK.

To open a secure connection, click Login.
Connect to a bare metal server by using SSH

You might require a VPN to access your system depending on your security setup and ssh configuration on the bare metal host.

You must ssh to the host by using your credentials, or the root credentials that are available from the IBM Cloud Console.

You will require root permissions in order to install the monitoring agent.

For example, you can complete the following steps to get the bare metal server information that you need to ssh into the server:
1. Log in to your IBM Cloud account.
2. Click the Menu icon > Classic Infrastructure > Device List.
3. Identify the bare metal server that you want to monitor. Copy the Public IP.
4. Click the bare metal server device name.
5. Select Passwords. Copy the password for the root user.
  
  Then, from a terminal, run the following command:
```
ssh <USER_ID>@<IP_ADDRESS>
```
  Where:
  
  <USER_ID> is the user ID that you use to log in to the bare metal server. For example, use root.
  
  <IP_ADDRESS> is the public IP address of the bare metal server.
  
  For example: ssh root@45.123.122.12

Provision an IBM Log Analysis instance

To provision an instance of IBM Log Analysis through the IBM Cloud UI, complete the following steps:

Log in to your IBM Cloud account.

Click Log in to IBM Cloud to sign in to the IBM Cloud.

After you log in with your user ID and password, the IBM Cloud console opens.
Click Catalog. The list of the services that are available in IBM Cloud opens.
To filter the list of services that is displayed, click Services and select the Logging and Monitoring category.
Click the IBM Log Analysis tile.
Select a region for the service instance.
Select the Lite service plan.

By default, the Lite plan is set.

For more information about other service plans, see Pricing plans.
Specify a Service name for your IBM Log Analysis service instance.
Select the Default resource group.

By default, the Default resource group is set.
To provision the IBM Log Analysis service in the IBM Cloud selected resource group, click Create.

After you provision an instance, the IBM Log Analysis dashboard opens.

To provision an instance of logging through the CLI, see Provisioning logging through the IBM Cloud CLI.

Configure a bare metal server to send logs to your logging instance

To configure your bare metal server to send logs to your IBM Log Analysis instance, you must install a logging-agent. The logging agent reads log files from /var/log or other directories you specify, and forwards the log data to your logging instance.

To configure your bare metal server to forward logs to your logging instance, complete the following steps from an Ubuntu terminal:

Install the logging agent. Run the following commands:

echo "deb https://assets.logdna.com stable main" | sudo tee /etc/apt/sources.list.d/logdna.list

wget -O- https://assets.logdna.com/logdna.gpg | sudo apt-key add -

sudo apt-get update

sudo apt-get install logdna-agent < "/dev/null"

Set the ingestion key that the logging agent will use to forward logs to the IBM Log Analysis instance.
```
sudo logdna-agent -k <INGESTION_KEY>
```
where <INGESTION_KEY> contains the ingestion key for the IBM Log Analysis instance where the logs will be forwarded.

You can retrieve the ingestion key using the IBM Cloud console, or by the IBM Cloud CLI.
Set the authentication endpoint. The logging agent uses this host to authenticate and get the token to forward logs.
```
sudo logdna-agent -s LOGDNA_APIHOST=api.us-south.logging.cloud.ibm.com
```

Set the ingestion endpoint.

sudo logdna-agent -s LOGDNA_LOGHOST=logs.us-south.logging.cloud.ibm.com

(Optional) Define any additional log paths to be monitored. Run the following command:
```
sudo logdna-agent -d <PATH_TO_LOG_FOLDERS>
```
Where <PATH_TO_LOG_FOLDERS> is the path where logs are saved on your system. By default, /var/log is monitored.
(Optional) Configure the logging agent to tag your hosts. Run the following command:
```
sudo logdna-agent -t TAG1,TAG2
```
Tags must be separated by commas and without any spaces between the comma and tag name.
Update the logging agent with your changes. Run the following command:
```
sudo update-rc.d logdna-agent defaults
```
Start the logging agent. Run the following command:
```
sudo /etc/init.d/logdna-agent start
```

Troubleshooting

You can use the /var/log/logdna-agent.log log to determine if there are any issues with your logdna-agent installation.

Launch the logging Web UI

To launch the IBM Log Analysis dashboard from the IBM Cloud UI, complete the following steps:

Log in to your IBM Cloud account.

Click IBM Cloud dashboard to launch the IBM Cloud dashboard.

After you log in with your user ID and password, the IBM Cloud Dashboard opens.
In the navigation menu, select Observability.
Click Logging.

The list of IBM Log Analysis instances that are available on IBM Cloud is displayed.
Select one instance. Then, click Open dashboard.

The logging Web UI opens and displays your cluster logs.

View your logs

From the logging Web UI, you can view your logs as they pass through the system. You view logs by using log tailing.

With the Free service plan, you can only tail your latest logs.

For more information, see Viewing logs.

Next steps

The following additional features are available:

To use any of these features, you must upgrade the IBM Log Analysis plan to a paid plan.