IBM Power Virtual Server and IBM AI Cloud Services Integration
This document provides reference architectures and design decisions for applications and services hosted on IBM Power Virtual Server to integrate with IBM managed watsonx SaaS services or client managed watsonx software in IBM Cloud.
Overview
This article discusses how to integrate Power Virtual Server and watsonx, and take advantages of both of them. The diagram below highlights the high-level communication between Power Virtual Server and watsonx services:
- To extend the mission critical workloads with watsonx, the applications hosted on Power Virtual Server can invoke inference APIs on watsonx service endpoint;
- To allow cloud native AI applications or watsonx services to make use of the data or applications hosted on Power Virtual Server, watsonx needs to access the private services hosted on Power Virtual Server. The watsonx.governance service can also use similar path to monitor AI models hosted on Power Virtual Server if needed.
Architecture diagram
For watsonx, client could choose to use the SaaS offering and create instances of the services they are interested in. In this case, IBM managed the watsonx stack and client consumes the corresponding cloud services.
There are also cases that clients would like to train or fine tune the AI models, and host the models themselves. In this case, client needs to set up their own stack and manage watsonx themselves.
We will discuss the reference architectures for both watsonx as SaaS and watsonx as software.
IBM Power Virtual Server and watsonx SaaS integration
First, let's discuss Power Virtual Server and watsonx SaaS integration. Let's take a look at two flavors of the reference architecture.
In this case, client has some workloads on Power Virtual Server, which could be databases, applications, or even AI models hosted on Power Virtual Server. If the client also has other cloud workloads in VPC, we would recommend multiple VPCs for separation of concerns between edge networking controls, provider management, and consumer workloads. Refer to VPC reference architecture in IBM Cloud Framework for Financial Services for recommended architecture and best practices.
IBM Power Virtual Server and watsonx SaaS connectivity:
- Applications and services on IBM Power Virtual Servers may need to use watsonx SaaS, for example, application running on Power Virtual Server needs to invoke LLM hosted in watsonx.ai
- 1.1 Direct Connectivity (NAT): Power Virtual Server provides a built-in NAT service for direct connectivity to IBM Cloud Services. This option offers a maximum speed of 10Gbps.
- 1.2 Virtual Private Endpoint (VPE): For IBM Cloud Services that has VPE enabled (eg. COS), you can create a VPE pointing directly to the Cloud Service. This method typically provides the highest throuput.
- IBM Cloud Services (eg. watsonx.data) needs to connect to databases or services (eg. DB2 or Oracle) hosted on Power Virtual Server
- 2.1 Satellite Connector enables private connectivity to Power Virtual Server workloads. Satellite Connector and agent need to be configured and deployed.
- 2.2 IBM Cloud Services connect to target services on Power Virtual Server via secure private channel.
This architecture includes Power Virtual Server Workspace and three VPCs, which provide segmentation for edge traffic control, management functionality, and consumer workloads.
If the client only has workloads on Power Virtual Server and no other workloads in VPC, we would recommend to set up edge VPC to control traffic flows and do not expose Power Virtual Server directly. The diagram below is a simplied version of the reference architecture, where network flow and bastion host can be set up in the edge VPC.
IBM provides different flavors of deployable architectures to automatically provision Power Virtual Server environment. Refer to Power Systems Virtual Server with VPC landing zone for details.
You can also refer to Gen AI pattern for watsonx on IBM Cloud, which discusses Gen AI patterns without Power Virtual Server.
IBM Power Virtual Server and watsonx software integration
If client chooses to deploy their own stack and have a single tenant watsonx to train, tune, or host their own models, they can deploy watsonx on OpenShift cluster at location of their choice. The reference architecture below shows the watsonx stack deployed in workload VPC in IBM Cloud.
IBM Power Virtual Server and watsonx software connectivity:
- Client deploys watsonx software in client owned VPC
- Power Workspace and VPC are connected via Transit Gateway
Design concepts
Following the Architecture Framework, this document covers the following solution aspects and domains:
Requirements
The following table outlines the requirements that are addressed in this architecture.
| Aspect | Requirements |
|---|---|
| Compute | Provide properly isolated compute resources with adequate compute capacity for the applications. |
| Storage | Provide storage that meets the application and database performance requirements. |
| Networking | Deploy workloads in isolated environment and enforce information flow policies. Provide secure, encrypted connectivity to the cloud’s private network for management purposes. Distribute incoming application requests across available compute resources. |
| Security | Ensure all operator actions are executed securely through a bastion host. Protect the boundaries of the application against denial-of-service and application-layer attacks. Encrypt all application data in transit and at rest to protect from unauthorized disclosure. Encrypt all security data (operational and audit logs) to protect from unauthorized disclosure. Encrypt all data using customer managed keys to meet regulatory compliance requirements for additional security and customer control. Protect secrets through their entire lifecycle and secure them using access control measures. Firewalls must be restrictively configured to prevent all traffic, both inbound and outbound, except that which is required, documented, and approved. |
| DevOps | Delivering software and services at the speed the market demands requires teams to iterate and experiment rapidly. They must deploy new versions frequently, driven by feedback and data. |
| Resiliency | Support application availability targets and business continuity policies. Ensure availability of the application in the event of planned and unplanned outages. Backup application data to enable recovery in the event of unplanned outages. Provide highly available storage for security data (logs) and backup data. |
| Service Management | Monitor system and application health metrics and logs to detect issues that might impact the availability of the application. Generate alerts/notifications about issues that might impact the availability of applications to trigger appropriate responses to minimize down time. Monitor audit logs to track changes and detect potential security problems. Provide a mechanism to identify and send notifications about issues found in audit logs. |
Components
The following table outlines the products or services used in the architecture for each aspect.
| Aspects | Architecture components | How the component is used |
|---|---|---|
| Data | watsonx.ai | Brings together new generative AI capabilities powered by foundation models and traditional machine learning (ML) into a powerful studio spanning the AI lifecycle |
| watsonx.data | Enables you to scale analytics and AI with all your data, wherever it resides | |
| watsonx.governance | Direct, manage and monitor the artificial intelligence activities | |
| Watsonx Assistant | Conversational artificial intelligence platform | |
| Watson Discovery | Automates the discovery of information and insights with advanced Natural Language Processing and Understanding | |
| watsonx Orchestrate | A digital assistant and platform that uses automation to help businesses streamline processes and save time | |
| Compute | IBM Power Virtual Server | Virtual Server offering on IBM Power systems |
| Virtual Servers for VPC | Web, App, and database servers | |
| Code Engine | Run your application, batch job, or container on a managed serverless platform | |
| Red Hat OpenShift Kubernetes Service (ROKS) | A managed offering to create your own cluster of compute hosts where you can deploy and manage containerized apps on IBM Cloud | |
| Storage | Cloud Object Storage | Provide flexible, cost-effective, and scalable cloud storage for unstructured data |
| IBM Cloud Block Storage for VPC | Persistent storage for use as boot and data storage for Virtual Servers in a VPC network | |
| Networking | VPC Virtual Private Network (VPN) | Remote access to manage resources in private network |
| Virtual Private Endpoint (VPE) | For private network access to Cloud Services, e.g., Key Protect, COS, etc. | |
| VPC Load Balancers | Application Load Balancing for web servers, app servers, and database servers | |
| Direct Link 2.0 | Seamlessly connect on-premises resources to cloud resources | |
| Transit Gateway (TGW) | Connects the Workload and Management VPCs within a region | |
| Cloud Internet Services (CIS) | Global load balancing between regions | |
| Access Control List (ACL) | To control all incoming and outgoing traffic in Virtual Private Cloud | |
| Security | IAM | IBM Cloud Identity & Access Management |
| Key Protect | A full-service encryption solution that allows data to be secured and stored in IBM Cloud | |
| BYO Bastion Host on VPC VSI | Remote access with Privileged Access Management | |
| App ID | Add authentication to web and mobile apps | |
| Secrets Manager | Certificate and Secrets Management | |
| Security and Compliance Center (SCC) | Implement controls for secure data and workload deployments, and assess security and compliance posture | |
| Hyper Protect Crypto Services (HPCS) | Hardware security module (HSM) and Key Management Service | |
| Virtual Network Function (VNF) | Virtualized network services running on virtual machines. | |
| Event Notifications | Get notified about critical events that occur in your IBM Cloud account. | |
| DevOps | Continuous Integration (CI) | A pipeline that tests, scans and builds the deployable artifacts from the application repositories |
| Continuous Deployment (CD) | A pipeline that generates all of the evidence and change request summary content | |
| Continuous Compliance (CC) | A pipeline that continuously scans deployed artifacts and repositories | |
| Container Registry | Highly available, and scalable private image registry | |
| Resiliency | VPC VSIs, VPC Block across multiple zones in two regions | Web, app, database high availability and disaster recovery |
| Service Management | IBM Cloud Monitoring | Apps and operational monitoring |
| IBM Cloud Logs | Scalable logging service that persists logs and provides users with capabilities for querying, tailing, and visualizing logs |