NSX ESG

在 IBM Cloud® for VMware Solutions 中配置的第一个组件是 Red Hat OpenShift，是一对NSX Edge设备。 NSX Edge设备配置为一对主动或被动超大型NSX Edge设备。

作为配置流程的一部分，NSX Edge连接到为 Red Hat OpenShift 集群订购的 IBM Cloud 公共和私有子网。

在内部或专用部署中，NSX 边缘被配置为主动或被动。 因此，用户必须创建 vSphere Distributed Resource Scheduler (DRS) antiaffinity 规则，以确保 NSX Edges 不会与各自的对等设备运行在同一主机上。

NSX ESG 接口

边缘部署了一个与 IBM Cloud 公共网络连接的上行接口和一个与 IBM Cloud 专用网络连接的上行接口。 此外，Transit网络连接分布式逻辑路由器（DLR）的接口也已可用。

NSX ESG 防火墙

配置规则，允许与互联网、IBM Cloud 网络和VXLAN网络通信。

NSX ESG DHCP

对于 Red Hat OpenShift 4.7 环境，引导、控制平面和计算节点需要访问 DHCP 服务器，以获取网络上的初始地址。 网络提供下载启动点火文件的途径。 初始设置后，将在节点上使用 Terraform 配置静态 IP 地址。

NSX ESG NAT

定义 NAT，以提供一种机制，允许 Red Hat OpenShift 网络访问公共和专用网络。

NSX ESG 路径

在边缘，将默认路由配置为公共互联网，然后添加静态路由以访问 IBM Cloud 专用网络。

NSX 负载均衡器

在 Red Hat OpenShift 环境中，需要两个负载均衡器，一个用于访问控制平面节点，另一个用于访问计算节点。 NSX Edge 将启用负载均衡，并通过应用程序概要文件进行配置，这些概要文件使用证书来处理来自源的入站连接。 NSX Edge还配置了负载平衡池，用于指向 Red Hat OpenShift 控制平面节点和 Red Hat OpenShift 计算节点。 此外，会创建虚拟服务器以使用专用接口上的虚拟 IP 地址 (VIP)，其中包含使用 VIP 来连接池的规则。

PowerNSX 命令

查看以下 PowerNSX 命令示例，您可以使用这些命令来自动配置和配置NSX ESG。

使用下表记录部署所需的参数。 显示的示例与先前描述的部署相匹配。

# Connect to NSX Manager
Connect-NsxServer -vCenterServer <IP_Address> -User <UserName> -Password '<Password>'

# Create Edge interface specification and create Edge
$priUplink = New-NsxEdgeInterfaceSpec -index 0 -Name "Private Uplink" -Type Uplink -ConnectedTo (Get-VDPortgroup SDDC-DPortGroup-Mgmt) -PrimaryAddress 10.208.242.130 -SubnetPrefixLength 26 -SecondaryAddresses 10.208.242.131,10.208.242.132,10.208.242.133
$pubUplink = New-NsxEdgeInterfaceSpec -index 1 -Name "Public Uplink" -Type Uplink -ConnectedTo (Get-VDPortgroup SDDC-DPortGroup-External) -PrimaryAddress 169.48.73.42 -SubnetPrefixLength 29 -SecondaryAddresses 169.48.73.43,169.48.73.44
$transit = New-NsxEdgeInterfaceSpec -index 2 -Name "Transit" -Type internal -ConnectedTo (Get-NsxLogicalSwitch OpenShift-Transit) -PrimaryAddress 192.168.100.1 -SubnetPrefixLength 24
New-NsxEdge -Name OpenShift-ESG -Cluster (Get-Cluster cluster1) -EnableHA -Datastore (Get-Datastore vsanDatastore) -Username admin -Password VMware12345! -FormFactor xlarge -Interface $priUplink,$pubUplink,$transit

# Set default gateway and add static routes to the ESG
Get-NsxEdge OpenShift-ESG | Get-NsxEdgeRouting | Set-NsxEdgeRouting -DefaultGatewayAddress 169.48.73.41 -DefaultGatewayVnic 1 -DefaultGatewayDescription "Public traffic"-confirm:$false
Get-NsxEdge OpenShift-ESG | Get-NsxEdgeRouting | New-NsxEdgeStaticRoute -Network 10.0.0.0/8 -NextHop 10.208.242.129 -Vnic 0 -Description "Private traffic" -confirm:$false
Get-NsxEdge OpenShift-ESG | Get-NsxEdgeRouting | New-NsxEdgeStaticRoute -Network 100.0.0.0/8 -NextHop 10.208.242.129 -Vnic 0 -Description "Private traffic" -confirm:$false
Get-NsxEdge OpenShift-ESG | Get-NsxEdgeRouting | New-NsxEdgeStaticRoute -Network 161.26.0.0/16 -NextHop 10.208.242.129 -Vnic 0 -Description "Private traffic" -confirm:$false
Get-NsxEdge OpenShift-ESG | Get-NsxEdgeRouting | New-NsxEdgeStaticRoute -Network 192.168.133.0/24 -NextHop 192.168.100.2 -Vnic 2 -Description "OpenShift Traffic" -confirm:$false

# Configure ESG firewall rules
Get-NsxEdge OpenShift-ESG | Get-NsxEdgeFirewall | New-NsxEdgeFirewallRule -Name "Private Outbound" -Source 10.208.242.128/26 -Service any -Action accept
Get-NsxEdge OpenShift-ESG | Get-NsxEdgeFirewall | New-NsxEdgeFirewallRule -Name "Public Outbound" -Source 169.48.73.40/29 -Service any -Action accept
Get-NsxEdge OpenShift-ESG | Get-NsxEdgeFirewall | New-NsxEdgeFirewallRule -Name "OpenShift Network" -Source 192.168.133.0/24 -Service any -Action accept
Get-NsxEdge OpenShift-ESG | Get-NsxEdgeFirewall | New-NsxEdgeFirewallRule -Name "Transit Network" -Source 192.168.100.0/24 -Service any -Action accept
Get-NsxEdge OpenShift-ESG | Get-NsxEdgeFirewall | New-NsxEdgeFirewallRule -Name "Private inbound to OpenShift" -Source 10.0.0.0/8 -Destination 10.208.242.131 -Service tcp/80,tcp/443 -Action accept
Get-NsxEdge OpenShift-ESG | Get-NsxEdgeFirewall | New-NsxEdgeFirewallRule -Name "SSH inbound to Bastion" -Source 10.0.0.0/8 -Destination 10.208.242.133 -Service tcp/22 -Action accept

# Configure ESG NAT rules
Get-NsxEdge OpenShift-ESG | Get-NsxEdgeNat | Set-NsxEdgeNat -enabled -confirm:$false
Get-NsxEdge OpenShift-ESG | Get-NsxEdgeNat | New-NsxEdgeNatRule -Vnic 1 -OriginalAddress 192.168.133.0/24 -TranslatedAddress 169.48.73.43 -Action snat -Protocol any -OriginalPort any -TranslatedPort any -Enabled -Description "OpenShift Network Public Outbound"
Get-NsxEdge OpenShift-ESG | Get-NsxEdgeNat | New-NsxEdgeNatRule -Vnic 0 -OriginalAddress 192.168.133.0/24 -TranslatedAddress 10.208.242.131 -Action snat -Protocol any -OriginalPort any -TranslatedPort any -Enabled -Description "OpenShift Network Private Outbound"
Get-NsxEdge OpenShift-ESG | Get-NsxEdgeNat | New-NsxEdgeNatRule -Vnic 0 -OriginalAddress 10.208.242.133 -TranslatedAddress 192.168.133.8 -Action dnat -Protocol any -OriginalPort any -TranslatedPort any -Enabled -Description "SSH to bastion node"

# Configure Load-balancer pool members
get-nsxedge OpenShift-ESG | get-nsxloadbalancer | Set-NsxLoadBalancer -Enabled
$poolMember1 = New-NsxLoadBalancerMemberSpec -name bootstrap-0 -IpAddress 192.168.133.9 -Port 6443
$poolMember2 = New-NsxLoadBalancerMemberSpec -name control-plane-0 -IpAddress 192.168.133.10 -Port 6443
$poolMember3 = New-NsxLoadBalancerMemberSpec -name control-plane-1 -IpAddress 192.168.133.11 -Port 6443
$poolMember4 = New-NsxLoadBalancerMemberSpec -name control-plane-2 -IpAddress 192.168.133.12 -Port 6443
$poolMember5 = New-NsxLoadBalancerMemberSpec -name bootstrap-0 -IpAddress 192.168.133.9 -Port 22623
$poolMember6 = New-NsxLoadBalancerMemberSpec -name control-plane-0 -IpAddress 192.168.133.10 -Port 22623
$poolMember7 = New-NsxLoadBalancerMemberSpec -name control-plane-1 -IpAddress 192.168.133.11 -Port 22623
$poolMember8 = New-NsxLoadBalancerMemberSpec -name control-plane-2 -IpAddress 192.168.133.12 -Port 22623
$poolMember9 = New-NsxLoadBalancerMemberSpec -name compute-0 -IpAddress 192.168.133.13 -Port 80
$poolMember10 = New-NsxLoadBalancerMemberSpec -name compute-1 -IpAddress 192.168.133.14 -Port 80
$poolMember11 = New-NsxLoadBalancerMemberSpec -name compute-2 -IpAddress 192.168.133.15 -Port 80
$poolMember12 = New-NsxLoadBalancerMemberSpec -name compute-0 -IpAddress 192.168.133.13 -Port 443
$poolMember13 = New-NsxLoadBalancerMemberSpec -name compute-1 -IpAddress 192.168.133.14 -Port 443
$poolMember14 = New-NsxLoadBalancerMemberSpec -name compute-2 -IpAddress 192.168.133.15 -Port 443

# Configure Load-balancer pools
$monitor = Get-NsxEdge OpenShift-ESG | Get-NsxLoadBalancer | Get-NsxLoadBalancerMonitor default_tcp_monitor
$pool1 = Get-NsxEdge OpenShift-ESG | Get-NsxLoadBalancer | New-NsxLoadBalancerPool -name api-pool-6443 -Description "OpenShift API TCP 6443" -Transparent:$false -Algorithm Round-Robin -Monitor $monitor -Memberspec $poolMember1,$poolMember2,$poolMember3,$poolMember4
$pool2 = Get-NsxEdge OpenShift-ESG | Get-NsxLoadBalancer | New-NsxLoadBalancerPool -name api-pool-22623 -Description "OpenShift API TCP 22623" -Transparent:$false -Algorithm Round-Robin -Monitor $monitor -Memberspec $poolMember5,$poolMember6,$poolMember7,$poolMember8
$pool3 = Get-NsxEdge OpenShift-ESG | Get-NsxLoadBalancer | New-NsxLoadBalancerPool -name app-pool-80 -Description "OpenShift Application HTTP" -Transparent:$false -Algorithm Round-Robin -Monitor $monitor -Memberspec $poolMember9,$poolMember10,$poolMember11
$pool4 = Get-NsxEdge OpenShift-ESG | Get-NsxLoadBalancer | New-NsxLoadBalancerPool -name app-pool-443 -Description "OpenShift Application HTTPS" -Transparent:$false -Algorithm Round-Robin -Monitor $monitor -Memberspec $poolMember12,$poolMember13,$poolMember14

# Configure Load-balancer application profile
$appProfile = Get-NsxEdge OpenShift-ESG | Get-NsxLoadBalancer | New-NsxLoadBalancerApplicationProfile -Name TCP-Source-Profile -Type TCP -PersistenceMethod sourceip

# Configure Load-balancer VIPs
Get-NsxEdge OpenShift-ESG | Get-NsxLoadBalancer | Add-NsxLoadBalancerVip -name api-6443-lb -Description "API / API-INT" -ipaddress 10.208.242.132 -Protocol tcp -Port 6443 -ApplicationProfile $appProfile -DefaultPool $pool1 -AccelerationEnabled
Get-NsxEdge OpenShift-ESG | Get-NsxLoadBalancer | Add-NsxLoadBalancerVip -name api-22623-lb -Description "API / API-INT" -ipaddress 10.208.242.132 -Protocol tcp -Port 22623 -ApplicationProfile $appProfile -DefaultPool $pool2 -AccelerationEnabled
Get-NsxEdge OpenShift-ESG | Get-NsxLoadBalancer | Add-NsxLoadBalancerVip -name app-80-lb -Description "Application HTTP" -ipaddress 10.208.242.131 -Protocol tcp -Port 80 -ApplicationProfile $appProfile -DefaultPool $pool3 -AccelerationEnabled
Get-NsxEdge OpenShift-ESG | Get-NsxLoadBalancer | Add-NsxLoadBalancerVip -name app-443-lb -Description "Application HTTPS" -ipaddress 10.208.242.131 -Protocol tcp -Port 443 -ApplicationProfile $appProfile -DefaultPool $pool4 -AccelerationEnabled

# Create DHCP pool
$xmlPayload = "
  <dhcp>
    <enabled>true</enabled>
    <ipPools>
      <ipPool>
        <ipRange>192.168.133.50-192.168.133.100</ipRange>
        <defaultGateway>192.168.133.1</defaultGateway>
        <subnetMask>255.255.255.0</subnetMask>
        <leaseTime>86400</leaseTime>
        <autoConfigureDNS>false</autoConfigureDNS>
        <primaryNameServer>10.187.214.66</primaryNameServer>
      </ipPool>
    </ipPools>
    <logging><enable>true</enable>
    <logLevel>info</logLevel></logging>
  </dhcp>"

$edgeID = (Get-NsxEdge -Name "OpenShift-ESG").Id
$uri = "/api/4.0/edges/$($edgeID)/dhcp/config"
$null = invoke-nsxwebrequest -method "put" `
      -uri $uri -body $xmlPayload -connection $nsxConnection

# Disconnect
Disconnect-NsxServer

相关链接

• Red Hat OpenShift堡垒节点设置
• Red Hat OpenShift4.7用户提供商基础架构安装