Deploying the Logging agent for Red Hat OpenShift on IBM Cloud clusters using a Helm chart

You can use a Helm chart to deploy the Logging agent to collect and route infrastructure and application logs from an Red Hat OpenShift on IBM Cloud (OpenShift) cluster to an IBM Cloud Logs instance.

Complete the following steps to deploy an agent on an OpenShift cluster:

Before you begin

Make sure you have access to an Red Hat OpenShift on IBM Cloud (OpenShift) cluster with permissions to create namespaces and deploy the agent.
Install the following CLIs:
- The IBM Cloud CLI to log in to the IBM Cloud and manage IBM Cloud services such as creating an API key.
- The Openshift CLI to manage the cluster from the command line. Learn more.
- The latest release of the version 3 Helm CLI
Read about the Logging agent.
Check the agent versions that are available. For more information, see Checking the available agent versions. Note that the version of the Helm chart will match the version of the agent - for example, if you are using version 1.3.0 of the agent there's a Helm chart with version 1.3.0 that accompanies that version.

Step 1. Define the authentication method for the agent

Choose the type of identity and the authentication method for the agent. Then, create an API key if needed.

Complete the following steps:

Choose the type of identity: service ID or trusted profile.

You can use a service ID or a trusted profile as the identity that is used by the agent to authenticate with the IBM® Cloud Logs service.
Grant permissions for ingestion to the identity that you have chosen.

The role that is required for sending logs to IBM Cloud Logs is Sender.

For more information, see Setting up IAM permissions for ingestion.
Generate an API Key for service ID authentication.

For authentication with trusted profiles, this step is not required.

For more information, see Generating an API Key for ingestion.

Step 2. Configuring the Helm chart values file for the Logging agent

Complete the following steps:

Create a file named logs-values.yaml with the following content:

This file contains the configurations that are specific to your deployment.

metadata:
  name: "logs-agent"
image:
  version: "1.3.0"  # required

clusterName: ""     # Enter the name of your cluster. This information is used to improve the metadata and help with your filtering.

env:
  # ingestionHost is a required field. For example:
  # ingestionHost: "<logs instance>.ingress.us-east.logs.cloud.ibm.com"
  ingestionHost: "" # required

  # If you are using private CSE proxy, then use port number "3443"
  # If you are using private VPE Gateway, then use port number "443"
  # If you are using the public endpoint, then use port number "443"
  ingestionPort: "" # required

  iamMode: "TrustedProfile"
  # trustedProfileID - trusted profile id - required for iam trusted profile mode
  trustedProfileID: "" # required if iamMode is set to TrustedProfile

scc:
  # true here enables creation of Security Context Constraints in Openshift
  create: true

Update the fields in the yaml file with values specific to your environment.

Helm chart required parameters
Field Name	Description
`image.version`	the version of the agent to be deployed see Step 1
`clusterName`	the name of the cluster - this will introduce the tag `kubernetes.cluster_name` into all log lines
`env.ingestionHost`	the public or private ingress endpoint for the IBM Cloud Logs instance to receive the logs
`env.ingestionPort`	the ingress endpoint port Public ingress endpoint = `443` Private ingress endpoint(VPE) = `443` Private ingress endpoint(CSE) = `3443`
`iamMode`	`TrustedProfile` or `IAMAPIKey` based on the authentication method chosen in Step 1
`trustedProfileID`	If `iamMode` is `TrustedProfile` then provide the Trusted Profile ID, otherwise it's not required (for example: `Profile-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx` )
`scc.create`	set to `true` in order to create the Security Constraints in Openshift

Step 3. Install the Helm chart

If you are using the iamMode as IAMAPIKey then the apikey needs to be present in a Kubernetes secret named logs-agent with the key name IAM_API_KEY. The secret can be created using the Helm chart by including the --set secret.iamAPIKey=<your iamAPIKey> option when running the helm install. If the secret has been created manually or if you are using iamMode=TrustedProfile then do not include this option.

Complete the following steps:

Log in to the cluster.

Red Hat OpenShift on IBM Cloud is integrated with IBM Cloud Identity and Access Management (IAM). With IAM, you can authenticate users and services by using their IAM identities and authorize actions with access roles and policies. When you authenticate as a user through the Red Hat OpenShift console, your IAM identity is used to generate a Red Hat OpenShift login token that you can use to log in to the command line. You can automate logging in to your cluster by creating an IAM API key or service ID to use for the oc login command. For more information, see Accessing Red Hat OpenShift clusters.

For example, complete the steps in Using a service ID to log in to clusters to log in to your cluster.
Log in to the Helm registry. Choose one of the following options:

Option 1: Login to the Helm registry by running the helm registry login command:
```
helm registry login -u iambearer -p $(ibmcloud iam oauth-tokens --output json | jq -r .iam_token | cut -d " " -f2) icr.io
```
For more information, see Using Helm charts in Container Registry: Pulling charts from another registry or Helm repository

Option 2: Log in to the Helm registry in Container Registry by running the ibmcloud cr login command.

You can use the ibmcloud cr login command before you perform a Helm dry run or install. For more information, see Accessing Container Registry and ibmcloud cr login.

Run the following commands:
```
ibmcloud region-set global
```
```
ibmcloud cr login [--client CLIENT]
```
Perform a Helm dry run to see the resources that will be created by the Helm chart.

If you are using the iamMode=TrustedProfile then the complete command is:
```
helm install <install-name> --dry-run oci://icr.io/ibm/observe/logs-agent-helm --version <chart-version> --values <PATH>/logs-values.yaml -n ibm-observe --create-namespace
```
where:
- <install-name> is the name of the Helm installation (ie. logging-agent)
- <chart-version> is the version of the helm chart. The Helm chart version should match the agent image version. For more information, see Helm chart versions.
- <PATH> is the directory path where the logs-values.yaml file is located.
If you are using the iamMode=IAMAPIKey then the complete command is:
```
helm install <install-name> --dry-run oci://icr.io/ibm/observe/logs-agent-helm --version <chart-version> --values <PATH>/logs-values.yaml -n ibm-observe --create-namespace --set secret.iamAPIKey=<APIKey-value> --hide-secret
```
where:
- <APIKey-value> is the IAM apikey associated with the ServiceID setup in Step 1
- Add --hide-secret to hide the API key from showing in the output data after the command runs.
If you would like to inspect the helm chart contents locally, you can download the helm chart to your computer using the command: helm pull oci://icr.io/ibm/observe/logs-agent-helm --version <chart-version>. The downloaded tgz file contains the chart contents.

For example, you can run the following command from the directory where the logs-values.yaml file is available:
```
helm install logging-agent --dry-run oci://icr.io/ibm/observe/logs-agent-helm --version 1.3.0 --values ./logs-values.yaml -n ibm-observe --create-namespace --set secret.iamAPIKey=<secret> --hide-secret
```
Once the resources to be created are verified, then run the Helm install without the --dry-run option

If you are using the iamMode=TrustedProfile then the complete command is:
```
helm install <install-name>  oci://icr.io/ibm/observe/logs-agent-helm --version <chart-version> --values <PATH>/logs-values.yaml -n ibm-observe --create-namespace
```
where:
- <install-name> is the name of the Helm installation (ie. logging-agent)
- <chart-version> is the version of the helm chart. The Helm chart version should match the agent image version. For more information, see Helm chart versions.
- <PATH> is the directory path where the logs-values.yaml file is located.
If you are using the iamMode=IAMAPIKey then the complete command is:
```
helm install <install-name> oci://icr.io/ibm/observe/logs-agent-helm --version <chart-version> --values <PATH>/logs-values.yaml -n ibm-observe --create-namespace --set secret.iamAPIKey=<APIKey-value>
```
Where:
- <APIKey-value> is the IAM apikey associated with the ServiceID setup in Step 1

Step 4. Verify the agent is successfully deployed

When the agent is deployed, check the following resources are created:

The ibm-observe namespace.

To list the namespaces in the cluster, run the following command:
```
oc get namespace
```
You can also run the following command to search for the ibm-observe namespace:
```
oc get namespace | grep ibm-observe
```
A config map logs-agent in the namespace ibm-observe.

Run the following command to view the agent config details.
```
oc get configmap logs-agent -n ibm-observe
```
You can also use the following command:
```
oc describe configmaps logs-agent -n ibm-observe
```
A daemonset logs-agent in the namespace ibm-observe.

Run the following command to view the daemonset:
```
oc get ds -n ibm-observe
```

Retrieve the list of agent pods by using the following command:

oc get pods -n ibm-observe -o wide

NAME                  READY   STATUS    RESTARTS   AGE    IP              NODE           NOMINATED NODE   READINESS GATES
logs-agent-4lwvt      1/1     Running   0          2d5h   172.17.61.181   192.168.16.4   <none>           <none>
logs-agent-g7z87      1/1     Running   0          2d5h   172.17.0.48     192.168.32.4   <none>           <none>
logs-agent-nw56s      1/1     Running   0          2d5h   172.17.32.232   192.168.0.10   <none>           <none>

The READY column shows 1/1 for all pods, with a STATUS of Running. Verify that an agent pod is ready for each node in your cluster.

To check how many workers are available in your cluster, you can run the following command:

oc get nodes

NAME           STATUS   ROLES           AGE   VERSION
192.168.0.10   Ready    master,worker   8d    v1.20.0+558d959
192.168.32.4   Ready    master,worker   8d    v1.20.0+558d959
192.168.16.4   Ready    master,worker   8d    v1.20.0+558d959

The number of items in each of these two lists need to be the same, and you can match the IP addresses in the node names with the values in the NODE column of the pod listing.

If your nodes are not named by their IP, you can append the -o wide option and compare the values in the INTERNAL-IP column instead.

To view the logs of a pod, run oc logs <POD_NAME>> -n ibm-observe

Step 5. Verify logs are being delivered to your target destination

Complete the following steps:

Go to the web UI for your IBM Cloud Logs instance..
When your agent is correctly configured, you can see logs through the default dashboard view. The Logging agent tags log records with a kubernetes object that includes the cluster name.
```
kubernetes.cluster_name:<CLUSTER_NAME>
```
You can run the query kubernetes.cluster_name:<YOUR_CLUSTER_NAME> in your IBM Cloud Logs instance to search for logs that are generated by your cluster.