Migrating IBM Log Analysis or IBM Cloud Activity Tracker resources to IBM Cloud Logs resources

Understanding how IBM Cloud Logs compares to existing services

In IBM Log Analysis or IBM Cloud Activity Tracker, you can:

• Configure archiving for long-term storage by configuring an IBM Cloud Object Storage bucket.

• Define views and alerts to help you monitor your account, applications and infrastructure, and troubleshoot and do problem determination.

• Define dashboards and screens with customizable widgets to visualize your data.

• Define exclusion rules to drop data at ingestion and control log volumes and costs.

• Define parsing rules to extract new fields that you can index and use for querying and faster searching.

• Configure index rate alerts to control the rate of indexing and ingestion.

IBM Cloud Logs is a service that offers you all of the functions of IBM® Log Analysis and IBM Cloud Activity Tracker and more. For more information, see About IBM Cloud Logs.

Migrating categories

You can migrate categories into folders. Foder names are unique within the instance.

In IBM Log Analysis or IBM Cloud Activity Tracker, you can use categories to group related resources. You have categories for views, for dashboards and for screens.

In IBM Cloud Logs, you can use folders to group related resources. You have folders for views, and folders for dashboards.

When you migrate an instance, migration of categories is included in the migration.

In IBM Cloud Logs, the name of a folder for views must be unique within the instance and the name of a folder for dashboards must be unique within the instance.

Migrating views and alerts

You can migrate a view into a public view. You can migrate a view with an alert into a public view and an alert. View names are unique within an instance.

In IBM Cloud Logs, views and alerts are decoupled. You do not need a view to configure an alert. You can configure private views and public views. You can configure alerts.

• Alerts are a type of resource. They are not dependent on views.

• You can define different types of alerts such as standard alerts, flow alerts, and new value alerts. For information on the alert types that are supported, see Alerts.

• You can use the Incidents page to manage alerts that are triggered.

• Alerts are triggered through the IBM Cloud Event Notifications service. You configure the notification channels and conditions that trigger the alert in the IBM Cloud Event Notifications service.

• For more information on alerts, see Migrating alerts.

You can use the Migration tool to migrate views and alerts configured in IBM Log Analysis or IBM Cloud Activity Tracker. When you migrate an instance, migration of views and alerts is included.

• A view is migrated to an IBM Cloud Logs public view.
• A view with an alert is migrated to an IBM Cloud Logs public view and an alert definition.

Queries for views and alerts are migrated. However, since mapping is applied in a generic form across all environments, you might need to modify the proposed mapping created by the migration tool to meet your requirements.

In IBM Cloud Logs, the name of a view must be unique within the instance.

Migrating dashboards and screens

You can migrate dashboards and screens into IBM Cloud Logs dashboards.

Dashboards and screens are migrated into dashboards in IBM Cloud Logs.

• Tables are mapped into tables widgets.

• Gauges and counters are migrated into gauge widgets.

• Histograms are migrated into line chart widgets.

• The time-shifted graph is available by default through the Logs page, which is where you work with views in IBM Cloud Logs.

• Pie charts are migrated into pie chart widgets.

You can also configure other widgets such as a vertical line chart, a horizontal line char, DataPrime creator widgets or even create widgets by using Markdown coding.

When you migrate an instance, migration of Dashboards and screens are included in the migration. For dashboards, only plots are migrated and streamlined, so you will need to manually customaize the dashboard layout.

In IBM Cloud Logs, the name of a dashboard must be unique within the instance.

Migrating exclusion rules

You can migrate an exclusion rule by configuring a block parsing rule or configuring TCO policies.

In IBM Log Analysis or IBM Cloud Activity Tracker, you can define alerts on data that is excluded at ingestion by configuring an exclusion rule. However, in IBM Cloud Logs, you cannot define alerts on data excluded by using a block parsing rule. If you need to configure alerts on data that is excluded, in IBM Cloud Logs, you should configure the TCO optimizer and distribute logs through the Priority insights pipeline and the Analyze and alert pipeline. You can define alerts on data managed through either data pipeline. For more information, see Configuring the TCO Optimizer.

In IBM Log Analysis or IBM Cloud Activity Tracker, you can configure the exclusion rule so that you can view in livetail the data (shown as not retained). In IBM Cloud Logs, you can also configure a block parsing rule so that you can view through the Livetail feature the data and store in the Store and search data pipleine your excluded data. You can then query the excluded data directly from the archive.

You can use the Migration tool to migrate exclusion rules. When you migrate an instance, resources such as exclusion rules are included in the migration.

Queries for views and alerts are migrated. However, since mapping is applied in a generic form across all environments, you might need to modify the proposed mapping created by the migration tool to meet your requirements.

Migrating parsing rules

In IBM Log Analysis or IBM Cloud Activity Tracker, you can define rules to extract new fields from data in a log record.

In IBM Cloud Logs, you can configure different types of parsing rules such as extract, Timestamp Extract, and remove fields For more information, see Parsing rules. Even more, you can enrich your logs by adding fields to your JSON logs based on specific matches in your log data. For mroe information, see Enriching data.

To migrate parsing rules, you must configure an extract parsing rule in IBM Cloud Logs. You can use RegEx expressions to define how you want to extract the data. IBM Cloud Logs uses a Golang RegEx syntax.

Similar to IBM Log Analysis or IBM Cloud Activity Trackers, you have a limit on the number of indexed fields per day that you can use for fast searching. In IBM Cloud Logs, you can monitor the number of indexed fields and mapping exceptions that are generated as data is ingested. This limit applies to Priority insights searches only, that is, fast searches. It does not apply to searches on data that is stored in IBM Cloud Object Storage where you can run unlimited searches or alerting.

You can use the Migration tool to find out the indexed fields that you have configured in your instance. Migration of indexed fields is done manually.

Migrating data usage

In IBM Log Analysis or IBM Cloud Activity Tracker, you can monitor your daily and monthly usage quotas, and configure index rate alerts to be notified when an unexpected spike of data exceeds a threshold that you set based on your needs.

In IBM Cloud Logs, you can enable the Data Usage feature to collect predefined metrics that you can use to monitor the GB that are ingested and your daily quota. You can then use these metrics in custom dashboards and alerts. In addition, you can get a report that details the amount of data that is ingested into your account by application, subsystem, and severity. The report also includes the amount of data that was excluded from ingestion based on your custom exclusion rules over the selected period.

Migrating the archiving configuration

To migrate archiving, you must use new IBM Cloud Object Storage buckets: one bucket to store the logs, and a second bucket to store the metrics.

In IBM Cloud Logs, you can configure a data bucket where the ingested data is stored. You own and manage the bucket and the data. A key differentiator from IBM Log Analysis or IBM Cloud Activity Tracker is that with IBM Cloud Logs you can run unlimited searches on your data from the UI.

You can also configure a metrics bucket to store any metrics that are generated from logs to optimize storage without sacrificing important data.

You can use the Migration tool to migrate your archiving configuration. For more information, see Migrating instances with archiving configured.