Build, deploy, test and monitor a predictive machine learning model

Create a project

1. From the catalog, create Watson Studio
   1. Select a region
   2. Select a Lite pricing plan
   3. Change the Service name to watson-studio-tutorial
   4. Select a resource group and click Create
2. Click on the Launch in twisty and select IBM watsonx.
3. Create a project by clicking + Create a new project in the Projects section.
4. Provide iris_project as the project name.
5. In the Define storage, Add a new instance of a Object Storage service.
6. Click Create. Your new project opens and you can start adding resources to it.

Import data

As mentioned earlier, you will be using the Iris data set. The Iris dataset was used in R.A. Fisher's classic 1936 paper, The Use of Multiple Measurements in Taxonomic Problems, and can also be found on the UCI Machine Learning Repository. This small dataset is often used for testing out machine learning algorithms and visualizations. The aim is to classify Iris flowers among three species (Setosa, Versicolor or Virginica) from measurements of length and width of sepals and petals. The iris data set contains 3 classes of 50 instances each, where each class refers to a type of iris plant.

A sepal is a part of the flower of angiosperms (flowering plants). Usually green, sepals typically function as protection for the flower in bud, and often as support for the petals when in bloom. Petals are modified leaves that surround the reproductive parts of flowers. They are often brightly colored or unusually shaped to attract pollinators. https://en.wikipedia.org/wiki/Iris_flower_data_set

Download iris_initial.csv which consists of 40 instances of each species. Make sure the downloaded file is named iris_initial.csv.

1. Select the Assets tab if not already selected.
2. Under Data in this project, click Drop data files here or browse for files to upload.
3. Upload the downloaded iris_initial.csv.
4. Once added, you should see iris_initial.csv under the All assets section of the project.

Test the deployed model

1. Under Test tab of your deployment, click on JSON input icon next to Enter input data and provide the JSONbelow as input.

      {
      "input_data": [{
        "fields": ["sepal_length", "sepal_width", "petal_length", "petal_width"],
        "values": [
          [5.1,3.5,1.4,0.2], [3.2,1.2,5.2,1.7]
        ]
      }]
    }
   

2. Click Predict and you should see the Prediction results in table and JSON view.
3. You can change the input data and continue testing your model.

Provision IBM Watson OpenScale service

In this section, you will create a Watson OpenScale service to monitor the health, performance, accuracy and quality metrics of your deployed machine learning model.

1. Create a IBM Watson OpenScale service (watsonx.governance)
   1. Select a region preferably in the same region where you created the Machine Learning service.
   2. Choose Lite plan.
   3. Set the service name to watson-openscale-tutorial.
   4. Select a resource group.
   5. Click Create.
2. Once the service is provisioned, Click Manage on the left pane and click Launch Watson OpenScale.
3. Click on Manual setup to manually setup the monitors.

System setup

In this section, as part of preparing your model for monitoring you will set up and enable monitors for each deployment that you are tracking with IBM Watson OpenScale.

1. Click on Database. This is to store your model transactions and model evaluation results. (it may already be selected)
   1. Click the Edit icon on the Database tile
   2. Choose Free lite plan database as your Database type
   3. Click Save.
2. Click on Machine learning providers
   1. Click on Add machine learning provider and click the edit icon on the Connection tile.
   2. Select Watson Machine Learning(V2) as your service provider type.
   3. In the Deployment space dropdown, select the deployment space iris_deployment_space you created above.
   4. Leave the Environment type to Pre-production.
   5. Click Save.
3. On the far left pane:
   1. Click the icon for Insights dashboard(first icon) to add a deployment
   2. Click on Add to dashboard to start the wizard on the Select model location page.
      1. On the Deployment spaces tab click on the iris_deployment_space radio button
      2. Click Next
   3. On the Select deployed model page:
      1. Click iris_deployment
      2. Click Next
   4. On the Provide model information page:
      1. Data type: Numerical/categorical
      2. Algorithm type: Multi-class classification
      3. Click View summary
   5. Click Finish

The iris_deployment pre production dashboard is now displayed.

Click Actions > Configure monitors

1. Click the pencil icon on the Training data tile to start the wizard.
   1. In the Select configuration method page
      1. Click Use manual setup
      2. Click Next
   2. In the Specify training data method page
      1. For Training data option choose Database or cloud storage
      2. For Location choose Cloud Object Storage
      3. For Resource instance ID and API key, run the below command in the Cloud Shell. Make sure to change the value after --instance-name to match the name of the Object Storage instance you have been using for this tutorial.

         ibmcloud resource service-key $(ibmcloud resource service-keys --instance-name "cloud-object-storage-tutorial" | awk '/WDP-Project-Management/ {print $1}')
         

      4. Copy and paste the Credentials resource_instance_id. It will begin with crn and end with two colons ::.
      5. Copy and paste the Credentials api key without any trailing spaces.
      6. Click Connect.
      7. Select the Bucket that starts with irisproject-donotdelete-.
      8. Select iris_initial.csv from the Data set dropdown.
      9. Click Next
   3. In the Select the feature columns and label column method page
      1. The defaults should be correct. Species as the Label/Target and the rest as Features.
      2. Click Next
   4. In the Select model output method page
      1. The defaults should be correct, prediction for Prediction and probability for Probability.
      2. Click View summary
   5. Click Finish
2. Click the pencil icon on the Model output details tile to start the wizard.
   1. In the Specify model output details method page
      1. The defaults should be correct.
      2. Click Save
3. On the left pane, click on Quality under Evaluations and click the edit icon on the Quality thresholds tile
   1. In the Quality thresholds page set the following values:
      1. Accuracy 0.98
      2. Click Next
   2. In the Sample size page
      1. Set Minimum sample size to 10
      2. Click Save

On the left pane, Click on Go to model summary

The quality monitor (previously known as the accuracy monitor) reveals how well your model predicts outcomes.

As the tutorial uses a small dataset, configuring Fairness and Drift won't have any impact.

Evaluate the deployed model

In this section, you will evaluate the model by uploading a iris_retrain.csv file which contains 10 instances of each species. Download iris_retrain.csv.

1. Click on Actions and then Evaluate now.
2. Choose from CSV file as your import option and click on browse, upload the iris_retrain.csv file.
3. Click and click on Upload and evaluate.
4. After the evaluation is completed, you should see the dashboard with different metrics.

To understand the quality metrics, refer to Quality metric overview