Before you begin

• Set up your IBM Cloud account.

• Make sure that you have the following access roles to create a project and permission to create the project tool resources within the account:

  • The Editor role on the IBM Cloud Projects service.
  • The Editor and Manager role on the IBM Cloud® Schematics
  • The Viewer role on the resource group for the project
  • For more information about access and permissions, see Assigning users access to projects.

• Set up an authentication method. You can either use a Secrets Manager to manage API keys, or use Trusted Profiles to manage permission.

  • For more information, see Using an API key with Secrets Manager to authorize a project to deploy an architecture.
  • For more information, see Using trusted profiles to authorize a project to deploy an architecture.

• Generate RSA SSH keys by running command: ssh-keygen -t rsa -b 4096 -C "user_ID"

Provision Power Virtual Server environment in IBM Cloud

You can either provision the Power Virtual Server environment manually by following this documentation, or you can provision the Power Virtual Server environment by using deployable architecture provided by IBM Cloud. A deployable architecture (DA) is cloud automation for deploying a common architectural pattern that combines one or more cloud resources. We will use the Power Virtual Server with VPC landing zone Quickstart deployable architecture in this tutorial, and we will set up a Power Virtual Server instance with Linux.

This documentation page highlights the steps to use the deployable architecture, and I will show you how to deploy Linux instance using the Quickstart DA.

1. Log in to IBM Cloud and search for 'Power Virtual Server with VPC landing zone' in the Catalog.

   

2. Choose 'Create a new architecture' with the 'Quickstart' variation, and click 'Add to project'.

3. Choose the project you would like to use or create a new project, and provide the name for the configuration.

4. Under Security tab, provide the API key or Trusted Profile ID from the the prerequisite step.

5. In this tutorial, we will create a Power Virtual Server instance with RHEL 9.2 image. Choose custom image for 'tshirt_size' on the 'Required' tab, and specify values for zone, prefix, SSH keys, and resource group name.

6. Under Optional tab, choose 'Linux - RHEL9-SP2' for custom_profile_instance_boot_image, and edit the custom_profile JSON:

   {
   "sap_profile_id": null,
   "cores": "1",
   "memory": "2",
   "server_type": "s922",
   "proc_type": "shared",
   "storage": {
      "size": "",
      "tier": ""
   }
   }
   

7. Take the default for other fields, and click 'Save'.

8. Validate the configuration, approve and deploy. The environment will be deployed automatically. As you can see from the Quickstart DA diagram, it creates Edge VPC and PowerVS workspace. In Edge VPC, it creates Bastion host (jump server) in management security group, and proxy server in network service security group. It also creates a Power Virtual Server instance in the Power Virtual Server workspace. Other necessary components to connect Power Virtual Server workspace with IBM Cloud resources and secure the environment are also created, for example, Transit Gateway, VPN, VPE, etc.

9. Once the Quickstart DA finishes, you have the Power Virtual Server environment set up. You can find the details of the resources on the Outputs tab of your DA configuration. Make note of the following outputs:

   access_host_or_ip
   dns_host_or_ip
   nfs_host_or_ip_path
   powervs_instance_management_ip
   

   Where access_host_or_ip is the IP for the jump server, and the powervs_instance_management_ip is the IP for the Power Virtual Server instance.

10. Now you can connect to the Power Virtual Server instance via the following command. Make sure to replace the values of the placeholders with the values from your environment.

    ssh -A -o ServerAliveInterval=60 -o ServerAliveCountMax=600 -o ProxyCommand="ssh -W %h:%p root@<access_host_or_ip> -i <path_to_private_key>" root@<powervs_instance_management_ip> -i <path_to_private_key>
    

11. For the Quickstart DA, make sure to connect to Power Virtual Server instance and follow the Quickstart next steps.

    a. Add proxy settings in ~/.bashrc. As mentioned above, find the <proxy_host_or_ip_port> value in the outputs section of the deployment, and add the following entries at the end of the .bashrc file, and source .bashrc.

    export http_proxy=http://<proxy_host_or_ip_port>:3128
    export https_proxy=http://<proxy_host_or_ip_port>:3128
    export HTTP_PROXY=http://<proxy_host_or_ip_port>:3128
    export HTTPS_PROXY=http://<proxy_host_or_ip_port>:3128
    export no_proxy=161.0.0.0/0,10.0.0.0/8
    

    b. Mount file storage from VPC on PowerVS instance

    mkdir /nfs
    mount <nfs_host_or_ip_path> /nfs
    

    c. Config DNS on Power Virtual Server instance. Add the dns_host_or_ip_path value at the top in the /etc/resolv.conf file.

    nameserver dns_host_ip
    

    d. Now the Power Virtual Server instance should have internet access. You can try:

    curl google.com
    

Deploy target workloads on Power Virtual Server environment in IBM Cloud

I will set up two sample workloads on the Power Virtual Server instance, a MySQL database and a simple Nginx web server in podman containers.

1. Login to the Power Virtual Server instance, and install podman.

   dnf install podman -y
   

2. Start MySQL server

   podman run --name mysql-test-db -e MYSQL_ROOT_PASSWORD=<your-secret-password> -p 3306:3306 -d docker.io/ubuntu/mysql:latest
   

3. Let's download a sample MySQL database. As mentioned in this MySQL documentation page, we can use the sample Employees DB on GitHub.

   # create working directory
   mkdir projects && cd projects
   # Clone github project with sample MySQL test database
   git clone https://github.com/datacharmer/test_db.git
   cd test_db
   # Copy files into container
   podman cp . mysql-test-db:/test_db
   # Execute bash of container
   podman exec -it mysql-test-db bash
   cd /test_db
   # Import data from inside the container
   mysql -uroot -p<your-secret-password> < employees.sql
   
   

4. Now let's set up a simple Nginx web server.

   mkdir /tmp/nginx-test && cd /tmp/nginx-test
   echo "Hello world from `hostname`" > /tmp/nginx-test/index.html
   chmod go+r index.html
   # Start container
   podman run --name nginx-test -d -p 80:80 -v /tmp/nginx-test:/usr/share/nginx/html:ro,Z docker.io/library/nginx:latest
   

   Now curl command should run successfully.

   # curl --noproxy '*' http://localhost:80
   Hello world from yy0417-pi-qs
   

Provision Satellite Connector service in IBM Cloud

Satellite Connector can be created on IBM Cloud Console or via ibmcloud CLI. IBM Cloud Docs has instructions on how to create and config Satellite Connector with UI or CLI.

To create Satellite Connector on the IBM Cloud UI, click on Satellite -> Connectors, and then ‘Create connector’ button. Provide name, tag, Resource Group, IBM Cloud region, and create connector.

Once it is created, note down the ID.

SATELLITE_CONNECTOR_ID=MY_CONNECTOR_ID

Deploy Satellite Connector agent on Power Virtual Server

IBM Cloud Docs has instructions how to run the connector agent in docker. In this section, I will set up the agent on Power Virtual Server instance with RHEL 9.x using podman. You can refer to the IBM Cloud docs for more details.

1. First, install ibmcloud CLI by following instructions in IBM Cloud doc. On RHEL linux, I run the following command.

   # install ibmcloud cli
   curl -fsSL https://clis.cloud.ibm.com/install/linux | sh
   # install plugin container-registry(cr)
   ibmcloud plugin install cr
   

2. Create the local configuration files as explained in IBM Cloud documentation.

   a. Create a directory for the configuration files, in this example ~/agent/env-files

   mkdir -p ~/agent/env-files
   

   b. Create a file in the ~/agent/env-files directory called apikey with a single line value of your IBM Cloud API Key that can access the Satellite Connector.

   echo YOUR_API_KEY > ~/agent/env-files/apikey
   

   c. Create a file in the ~/agent/env-files directory called env.txt with the following values. Modify the SATELLITE_CONNECTOR_ID variable with your Satellite Connector ID. If you would like traffic to stay in your private network, you need to set SATELLITE_CONNECTOR_DIRECT_LINK_INGRESS. Refer to Configuraing the request path from your connector agent for more details. I am using the private endpoint in us-south in this demo.

   # Create a file called env.txt
   cat <<EOF > ~/agent/env-files/env.txt
   SATELLITE_CONNECTOR_ID=YOUR_SATELLITE_CONNECTOR_ID
   SATELLITE_CONNECTOR_IAM_APIKEY=/agent-env-files/apikey
   SATELLITE_CONNECTOR_TAGS=testsc
   SATELLITE_CONNECTOR_DIRECT_LINK_INGRESS=d-01-ws.private.us-south.link.satellite.cloud.ibm.com
   EOF
   

3. Pull the agent image

   # Log in to IBM Cloud and IBM Container Registry
   ibmcloud login --apikey YOUR_API_KEY
   ibmcloud cr region-set icr.io
   # login to container registry
   ibmcloud cr login --client podman
   # Pull image
   podman pull icr.io/ibm/satellite-connector/satellite-connector-agent:latest
   

4. Run the agent. Note that you may need the ':Z' access modifier to make sure that the container has the right permission to access the files.

   podman run -d --env-file ~/agent/env-files/env.txt -v ~/agent/env-files:/agent-env-files:Z icr.io/ibm/satellite-connector/satellite-connector-agent:latest
   # View podman containers:
   podman ps
   # view podman logs $CONTAINER_ID
   podman logs $CONTAINER_ID
   

   Make sure that there is no error in the logs. If everything works correctly, your agent should show up under ‘Active agents’ on the Satellite Connector cloud console, and the agent name should match the container ID.

   

   If you only have one agent and the agent stops, the connection is broken, so multiple agents are normally recommended. Refer to IBM Cloud doc for high availability setup. Only one agent is set up for demo purpose in this tutorial.

Config Satellite Connector Endpoint in Satellite Connector

Satellite Connector has endpoints that connect to the target workloads. Some IBM Cloud Services (eg. watsonx.ai) can help user to create System endpoint in Satellite Connector, in this case, user does not need to create User endpoint. Otherwise, user needs to create User endpoint themselves.

You can follow the instructions in IBM Cloud doc to create and manage Connector Endpoint either on UI or via CLI.

1. To create the User endpoint on UI, on Satellite Connector cloud console, under ‘User endpoints’, click on ‘Create endpoint’.

2. For destination resource, choose Agent location runs on the same network for this tutorial.

3. For resource details, enter a name for the endpoint, destination IP is the private IP of the Power Virtual Server instance, powervs_instance_management_ip on the DA config Outputs tab, and destination port is the port of the service.

   

4. Protocol is TCP, click next on the next few screens, and click on create endpoint. Once created, the endpoint will show up under User endpoints tab. You can click on the link to view details. Note down the endpoint address.

   

5. You can repeat the steps to add User endpoints for both Nginx and MySQL that are hosted on Power Virtual Server.

Connect from watsonx.data service to MySQL on Power Virtual Server in IBM Cloud

In this section, we will use watsonx.data as consumer to connect to the MySQL database hosted on the Power Virtual Server.

1. First provision the watsonx.data service if you haven't done so yet. Search for watsonx.data in IBM Cloud catalog, choose plan and location and provision the service.

2. Once the service is provisioned, open the web console. There are a few screens to help you set up starter engine and COS bucket used by watsonx.data.

3. Follow IBM documentation to add MySQL driver to watsonx.data.

   a. Click Configurations on the left, and click on the Driver manager tile.

   

   b. Click on Add driver.

   c. Drag the mysql-connector-j-8.2.0.jar file downloaded from MySQL site, and click Add. After the driver is successfully added, it undergoes a series of validation. If the validation is successful, it is set to 'inactive' status otherwise it is set to 'failed' status.

   

   d. Click the vertical ellipsis icon on the right to assign the driver to an engine.

   

   e. Click Assign, and confirm the assignment. The driver should be in Active status now.

   

4. Once the MySQL driver is active, let's create the connection. Click on the Infrastructure manager on the left.

   

5. Click on ‘Add component’, and choose MySQL. Fill in the connection information to the MySQL database hosted on Power Virtual Server. Note that the hostname and port information are from the Satellite Connector User endpoint address created in the previous step.

   

6. After MySQL connection is added, this should be what you see in Infrastructure manager.

   

7. Click on Manage Association on the MySQL catalog node.

   

8. Add to the starter engine, and click on save and restart engine.

   

9. You should see something like this once the engine is restarted.

   

10. Click on Data manager on the left, click on your data catalog and employees database, you should see the tables in the database. Click on departments table, and click on 'Data sample' tab on the right, you should see sample data from the table.

    

Congratulations! Your watsonx.data service successfully retrieved data from MySQL database hosted on your Power Virtual Server instance.

Connect from watsonx.ai service to the workloads on Power Virtual Server in IBM Cloud

In this section, we will use watsonx.ai service as consumer to connect to the MySQL database and Nginx services hosted on the Power Virtual Server.

1. First provision the watsonx.ai studio if you haven't done so yet. Search for watsonx.ai studio in IBM Cloud catalog, choose plan and location and provision the service.

   

2. Once the service is provisioned, launch in IBM Cloud Pak for Data.

   

3. View all projects and create one if needed. Click on the project you would like to use.

   

4. Under Assets tab, click New asset, and click on "Connect to a data source" tile.

   

5. Choose MySQL on the left, and click on MySQL tile. Watsonx.ai studio can manage the private connection for you, so that you don't have to manually create User endpoint. Refer to watsonx documentation for details.

   a. Fill in name of the connection. In this case, we would like watsonx.ai to manage the endpoint for us, so we fill in the private IP of the Power Virtual Server and the port of the MySQL service running on it.

   

   b. Fill in database name and credentials.

   

   c. Under Private connectivity section, choose the Satellite Connector create earlier in this tutorial, and click on the "Test connection" button on the top.

   

   d. You should see that the test connection was successful, and click the Create button to create the connection.

   

   e. Now if you go back to the Satellite Connector, and under System endpoint tab, you should see a new endpoint was created. Watsonx.ai will manage this system endpoint and delete it when the connection is deleted.

   

   f. Click on the system endpoint, you should be able to see the endpoint address.

   

6. Now let's add a new notebook and access the MySQL database.

   a. Under Assets tab, click "New asset", and click on the "Work with data and models in Python or R notebooks". Provide a name for the notebook, and select runtime. Runtime on Python 3.11 XS should be good enough for this tutorial. Click Create.

   

   b. Install the package needed, add this command to the notebook cell.

   !pip install mysql-connector-python
   

   c. The following test codes should work.

   

7. Access the Nginx server hosted on Power Virtual Server

   a. Add the following command to the cell, replace the host and port with your host and port for the Nginx service

   !curl http://<satellite-connector-private-host>:<Nginx-port>
   

   b. You should see result like this.

   

Congratulations! Your watsonx.ai service successfully accessed MySQL and Nginx services hosted on Power Virtual Server.