Release notes for the IBM Cloud Logs migration

Updates include:

• A fix for an error in bucket lifecycle type conversion.

• Enhancements for email notifications.

  • Support for both custom and default email notifications.

  • By default the Terraform is created with a default subscription. If the --ce flag is specified, notifications will be created with custom subscriptions.

• When archiving is not enabled in the migrated IBM Log Analysis or IBM Cloud Activity Tracker instance, Terraform files to configure IBM Cloud Logs IBM Cloud Object Storage buckets are added to the manual-tf-folder,

• The required region parameter is now added to the logs-router resource Terraform file.

Support for a new command to export the IBM Cloud Logs views configuration from one IBM Cloud Logs instance and import it into another IBM Cloud Logs instance. Only views with Lucene queries are supported.

When running the ibmcloud logging migrate create-resource command with the --platform option, if the IBM Cloud Activity Tracker instance name includes logdna in its name, the IBM Cloud Activity Tracker Event Routing targets will not be created and the following Terraform error is returned: undeclared resource on at-routes.tf.

Updates include:

• A fix for an IBM Cloud Object Storage warning for life cycle policies when running Terraform.

• Support for configuring IBM Cloud Event Notifications email notifications.

• Added the viewer role for IBM Cloud Event Notifications policies.

• Removed a warning message when duplicate view names are found.

• Add support for filtering views allowing selection of multiple subsystems, applications, and severities.

Updates include:

• IBM Plugin provider is updated to 0.1.74.

• Due to an IBM Cloud Logs requirement, buckets with retention policies cannot be attached to an IBM Cloud Logs instance. If you have an archiving configuration in your IBM Cloud Activity Tracker or IBM Log Analysis instance, the migration tool has been modified to create and attach buckets without any policies.

• Duplicate view names are not supported in IBM Cloud Logs. The migration tool is enhanced to allow you to migrate duplicate views without the current manual steps. Duplicate views are created following the naming convention of viewname-copy1, viewname-copy2, and so on.

• Fix for error for empty groups data.

Updates include:

• Fix for namespace query mapping for views and alerts.

Updates include:

• Fix to resolve an issue when duplicate access rule names are found during migration.

• Improved checking that all required parameters are specified before running migration commands.

• Fix for progress bar not advancing.

Updates include:

• Migrating IBM Cloud Activity Tracker or IBM Log Analysis log groups into IBM Cloud Logs data access rules.

  IBM Cloud Logs uses DPXL as its query language. The migration copies the current IBM Cloud Activity Tracker or IBM Log Analysis log group configuration (1 or more queries) to a data rule in the IBM Cloud Logs instance. Each query must be manually migrated to DPXL individually. For more information about DPXL, see the DataPrime Expression Language (DPXL) reference.

• Fixes related to configuring IBM Cloud Logs and IBM Cloud Object Storage buckets, including Windows support.

• Fixes for errors when migrating IAM policies for users when an inactive user is found.

• Fixes for errors when threshold was found to be of different types, for example, string in some cases and float64 in others.

Updates include:

• Fix duplicate topic name creation to allow creation of views that have the same name in an instance. A counter has been used to differentiate views with duplicate names.

• Add mandatory check before command execution to bring awareness of prerequisites before running the tool.

• Corrected the help for the update-resources command.

Updates include:

• Fix for an error when migrating exclusion rules where filters were not being applied.

• Addition of a new update-resources command with an --iam option. This command option can be used to migrate IAM policies for users, service IDs, trusted profiles, and access groups associated with an IBM Cloud Activity Tracker or IBM Log Analysis instances to IBM Cloud Logs IAM policies.

• Fix for terraform to add resource name validation.

Updates include:

• Enhanced query mapping to add mappings for Kubernetes fields: node, pod, namespace, and cluster_name to kubernetes.node_name, kubernetes.pod_name, kubernetes.namespace_name, kubernetes.cluster_name.

• Add support to automatically migrate IAM resources, users, serviceIds, trusted profiles and access groups per instance.

• Fixes a bug where alerts were failing to create properly if an alert encountered a problem.

Updates include:

• Fixes for problems when configuring buckets with activity tracking events or metrics enabled:

  • Documentation updates to the permissions that are required to create buckets. IBM Cloud Object Storage buckets with IBM Cloud Activity Tracker or IBM Cloud Monitoring enabled must have the manager role (storage.bucket.put_activity_tracking, cloud-object-storage.bucket.put_metrics_monitoring) to create the new buckets with the correct configuration.

  • Configuring the buckets so that the IBM Cloud Object Storage service identifies where to send events and metrics.

  • A fix for a bug when IBM Cloud Object Storage buckets are created using the API.

• Adding labels to alerts that correlate the alert in IBM Cloud Logs with the IBM Cloud Event Notifications topic where data from the alert is processed.

Updates include:

• The period when an IBM Cloud Event Notifications alert is re-sent is set to 10 minutes.

• If a service to service policy for IBM Cloud Event Notifications and IBM Cloud Logs is not required, one is no longer created.

• A fix for an error during migration when you have archiving configured in your IBM Cloud Activity Tracker or IBM Log Analysis instance, and the collection of metrics for the bucket is not enabled

• Parsing rules are now maintained in the correct default order. Rules created during migration within an instance are created after the default parsing rule that is included when you provision an instance of IBM Cloud Logs.

Updates include:

• Support to configure all supported IBM Cloud Logs retention periods.

• Collection of IAM access report information.

• Limit IBM Cloud Logs Routing to public ingress endpoints.

• Improved validation of the usage of the -t and --api options.

• Updates to the CLI help.

• Added plug-in version to log output.

• Fix for issue when creating webhooks for alerts via the API. A service to service authorization and integration with IBM Cloud Event Notifications are only created when you confirm that you want the integration created.

• Fix for Terraform description validation for invalid characters. Support was added for commas.

• Fix for IBM Cloud Object Storage bucket creation error when the IBM Cloud Activity Tracker or IBM Log Analysis have archiving configured to an IBM Cloud Object Storage bucket and the endpoint used is a public endpint.

• Fix for IBM Cloud Logs Routing name error. The migration tool now enforces the 35 character naming limitation.

In addition to bug fixes, updates include:

• The ability, with a single command, to continue sending logs to your IBM Log Analysis instance while also sending logs to the newly created IBM Cloud Logs instance. This dual destination configuration will help you verify your migration.

• The ability to specify the instance name and resource group ID for the created IBM Cloud Logs instance when running the migration command.

• The ability to set the retention plan for the created IBM Cloud Logs instance.

• Additional validation of the instance CRN.

Updates include:

• The ability to create an IBM Cloud Logs instance in a custom resourceGroupId.

• When migrating 1 IBM Cloud Activity Tracker or 1 IBM Log Analysis instance, any associated IBM Cloud Activity Tracker Event Routing or IBM Cloud Logs Routing configuration can also be migrated.

• A fix for a dashboard migration bug.

• The addition of streaming configuration files collected from IBM Cloud Activity Tracker and IBM Log Analysis instances.

• A fix for an error when doing query mapping where there was a difference in the conversion of error text. With this fix, error text will remain as-is.

Updates include:

• A fix to address a bug creating IBM Cloud Object Storage buckets when the bucket configuration includes lifecycle policies and retention policies.

• A fix to address a panic error when accessing an empty archive configuration.

Updates include:

• Support for running the migration tool in Windows systems.

• Enhancements migrating PagerDuty alerts. Terraform scripts are provided.

• Enhancd migration of views and alerts.

  Migration of apps and subsystems into filters in IBM Cloud Logs views and alerts.

  The App quick search field values in a Mezmo view are mapped to application filters in IBM Cloud Logs views and alerts.

Updates include:

• Support for the migration of lifecycle policies and retention policies for the data bucket only.
• A new command has been added to map a Mezmo view query to a Lucene query.
• A fix to set the incident_settings retriggering_period_seconds for alerts.
• A fix to remove empty struct in the platform logs report.
• Support for the latest IBM Cloud Terraform provider.

Updates include:

• Configure an outbound integration with the IBM Cloud Event Notifications service.
• Create one or more topics and subscriptions in the IBM Cloud Event Notifications service.

Updates include:

• Update IBM Cloud Event Notifications notification based on an updated response.

• Automated Terraform to run with a single flag.

• Help fixes.

• Fix for IBM Key Protect and archiving errors.

• Support in Terraform for views with an empty query.

Updates include:

• Consolidating the IBM Cloud Activity Tracker Event Routing configuration into 1 IBM Cloud Logs instance using either Terraform or the API.

• Data usage support.

• Fixes related to:

  • Dashboard
  • IBM Cloud Logs Routing
  • IBM Cloud Object Storage bucket errors

Updates include:

• IBM Cloud Activity Tracker Event Routing configuration when IBM Cloud Activity Tracker Event Routing is not configured in the account

• Fix error setting the severity when alerts are created.

• Vulnerability fixes.