Migrating IBM Log Analysis instances

IBM Log Analysis architecture

You can use IBM Log Analysis instances to monitor application logs, operational logs, and platform logs in the IBM Cloud and outside the IBM Cloud.

• Scenario 1: IBM Log Analysis instances that collect platform logs only:

  To monitor platform logs, you might have configured an IBM Log Analysis instance for each region where you operate and where logs are generated by enabled services and the IBM Cloud platform.

  To migrate these instances, see Migrating instances that collect platform logs.

• Scenario 2: IBM Log Analysis instances that collect application and operational logs:

  You can send logs to the IBM Log Analysis instance through different methods depending on the source such as configuring the LogDNA agent or using the ingestion REST API and libraries to send application logs.

  To migrate these instances, see Migrating instances that collect application and operational logs.

• Scenario 3: IBM Log Analysis instances that collect all types of logs such as service, application and operational logs, and platform logs.

Validating the new architecture

While you migrate and validate the new architecture, you must send logs to your IBM Log Analysis instances, same as you currently do now. You must also send logs in parallel to your IBM Cloud Logs instances to replicate your current architecture in the account.

Until you have validated the new architecture and you can remove the IBM Log Analysis instances from the account, you must continue managing data through the deprecated services.

Checklist to plan for migration

Consider these items when you are planning your migration of IBM Log Analysis instances in an account:

• Identify the regions where you operate and where IBM Log Analysis instances are provisioned.

• Identify the IBM Log Analysis instances in regions where your platform logs are configured to be collected.

• Check your requirements for long-term storage. Do you have archiving configured for each IBM Log Analysis instance? How long do you keep the data?

  In IBM Cloud Logs, you can query all data that is stored in your configured IBM Cloud Object Storage bucket. You own and maintain the bucket and the data stored in it.

• Are you streaming any data to other 3rd-party tools?

  The data format of logs does not change as part of the deprecation of the IBM Log Analysis service.

• Are you excluding any data at ingestion to reduce costs? Would it be useful to keep these logs for search or even for compliance? The IBM Cloud Logs service offers a TCO feature to help you optimize costs based on data criticality and operational requirements. Take time to identify useful logs that you might be currently dropping due to cost constraints.

• Are you controlling data usage by configuring index rate alerts?

• Do you control access by using access groups? Do you use trusted profiles or service IDs? Identify the policies in the account that grant permissions to operate the IBM Log Analysis instances in the account.

• What notification channels do you use for alerting? Email, slack, PagerDuty, webhook, IBM Cloud Monitoring (Sysdig).

• Make an inventory of the different types of sources of infrastructure and application logs that you are monitoring. What are your sources of the data? Do you collect logs from Kubernetes clusters, Red Hat OpenShift clusters, bare servers and so on?