Detecting drift in workspaces

IBM Cloud® Schematics enables users to deploy, manage, and manipulate infrastructure resources with Terraform based workspaces using declarative Infrastructure as Code (IaC) concepts. However, when a Terraform config is deployed and resources are created, it does not mean that the resources stay as declared by the config. Any change in the infrastructure state is called drift. It occurs when your deployed infrastructure differs from the wanted (declarative) state defined by your template configuration.

Drift can occur for many reasons. The most frequent cause is changes made to your infrastructure manually outside of Terraform automation. The Terraform state file of your deployed workspace is then no longer synchronized with your deployed infrastructure resources, and the workspace is said to be in drift.

Drift can happen for many reasons within the context of your configuration:

Adding, or removing resources from the Template configuration without applying the changes.
Changing template resource definitions.
External to your template configuration, drift occurs when changes are made manually. For example, from a command line operation on a cloud resource, or change through the cloud console.
Can occur through other automation tools.

Example drift scenario

A VSI instance is provisioned by using IBM Cloud Schematics and a configuration template. A DevOps cloud user can modify the provisioned VSI configuration by logging in to the Cloud console and modifying the boot volume of an instance or adding an Ethernet interface. These changes result in your infrastructure deployment in drift.

IBM Cloud Schematics enables, to predictably manage the resource lifecycle of your infrastructure by using Terraform. Drift occurs when the real-world state of your infrastructure differs from the state that is defined in your Terraform template configuration.

Terraform cannot detect drift in resources and attributes that are not managed or configured by using Terraform. For example, Terraform cannot not detect changes in a virtual machine that results from installing applications locally or by using configuration management tools like Chef or Ansible.

Drift detection in IBM Cloud

Drift detection for your Terraform automation workspaces is possible in IBM Cloud Schematics. You can use following three methods to check the drift detection.

Drift detection using the UI

You can initiate drift detection for workspaces from the Schematics workspace job page. It initiates a job to detect drift for the workspace and its deployed resources. During execution, the drift detection job is in progress, on completion it has a failure or success. To review the details of the drift job, you need to check the drift job log for the drift status.

Viewing detect drift logs using the UI

Use the following steps to view the drift job log.

From the workspace dashboard, select the workspace that you want check for drift.
Select and open your workspace.
Select the Actions drop down list.
Select Detect drift option to initiate the detect drift job.
During execution, the status shows in progress moving to a success or a job failure status on completion.

The drift status can be determined by reviewing the output of the job logon when the job is in a success state. A sample success job execution with detected drift is shown in the screen capture.

Review the success job log to identify the drift details.

2022/04/19 10:10:44 -----  Terraform DRIFT  -----
2022/04/19 10:10:44 Starting command: terraform-drift-cli drift
2022/04/19 10:10:44 Terraform Drift | configuration drift identfied
2022/04/19 10:10:44 Terraform Drift | resource         operation   attribute   drift value
2022/04/19 10:10:44 Terraform Drift | ibm_is_vpc.vpc   +           tags        schematics:us-east.workspace.myworkspace-drift-demo.bfdd0e2d
2022/04/19 10:10:44 Terraform Drift | ibm_is_vpc.vpc   +           tags        tag:new1
2022/04/19 10:10:44 Terraform Drift | ibm_is_vpc.vpc   +           tags        tag:new2
2022/04/19 10:10:44 Terraform Drift |
2022/04/19 10:10:44 Command finished successfully.
2022/04/19 10:10:52 Done with the workspace action

If the job fails, review the cause of the failure in the log, and correct the error condition by rerunning the job.

A failure example job log identifies the cause of the drift job failure.

2022/04/13 13:05:46 -----  Terraform Commands  -----
2022/04/13 13:05:46 Could not execute job: Error : Drift cannot be executed since state file doesn't exist. Please run terraform apply to generate state file.

Drift detection using the CLI

You can initiate detecting drift from the create workspaces command. The drift detection initiates a job to detect drift for the workspace and its specific resources. The drift detection job is in progress or completed with the appropriate status such as failure or success. Instead, to know the details of the drift job, you need to check the drift job log for the drift status. Use the following commands to view the detect drift.

Creating and viewing the detect drift logs using the CLI

You can follow these steps to detect the drift in Schematics workspaces using the CLI.

Create the Schematics workspaces.
Get your workspace ID.
Run the ibmcloud schematics plan.
Fetch the ibmcloud schematics job logs.
Run the ibmcloud schematics apply.

Run the ibmcloud schematics job run to create a job in Schematics workspace.

ibmcloud schematics job run --command-object workspace --command-object-id <workspace_id> --command-name drift

Run the ibmcloud schematics workspace action to retrieve all activities of your workspace.
```
ibmcloud schematics workspace action --id <workspace_id>
```
Run the ibmcloud schematics job logs to retrieve the detailed logs of a job to view the drift details.
```
ibmcloud schematics logs --id <workspace_id> --act-id <Job_id>
```

Creating and viewing the detect drift through API

Review the CURL commands to create and view the drift through API.

Retrieve your IAM access steps to authenticate with the IBM Cloud Schematics.

Create the workspace. As part of the payload you need to add the following drift configuration.

{
    commands: [
        {
        command: 'drift',
        command_name: 'drift command',
        command_desc: 'command to detect drift in workspace resources',
        },
    ],
    operation_name: 'drift',
    description: 'command to detect drift in workspace resources',
    }

Get the workspace details.
Run the Schematics job plan.
Fetch the ibmcloud schematics job logs.
Run the ibmcloud schematics apply.
Run the ibmcloud schematics job run to create a job for your workspace.
Run the ibmcloud schematics workspace action to retrieve all activities of your workspace.
Fetch the ibmcloud schematics job logs.