Monitoring parameters and components

Using the WAN Optimizer for monitoring

The Silver Peak WAN optimization appliance that is deployed as part of HCX™ does not get its management interface configured. The web user interface (UI) is a valuable tool to use when you base line traffic throughput and throttling network migration bandwidth use.

Only the HCX CGW Gateway WAN tunnel traffic flows through the WAN Optimizer appliance. Therefore, it cannot monitor stretched L2 traffic.

Configuring the UI

Outside of configuring an IP address for the web UI, do not make configuration changes to the WAN Optimizer appliance unless directed by the IBM® or VMware® support staff.

To configure the WAN Opt web UI:

Find the WAN Opt virtual machine (VM) appliance within the vCenter client.
Open up a console to the VM that uses the vCenter client.
Within the console, click F4 to configure the management interface.
Select to configure a static IP.
Use an IP address on the management IBM Cloud® assigned portable IP range and default gateway.
Click Apply on the next screen to save.
Click edit settings for the WAN Opt VM within the vCenter console.
Select Connected at Power On and Connected for Network Adapter 1.
Click OK to save.
Find the CGW-<xxx>-WANOPT in vCenter.
Edit settings on the VM.
Select the checkbox to connect Network adapter 1.
Click OK.
Go to https://<configured_WAN_OPT_IP>.
Log in with the admin default user and the admin password.

You can now use the WAN Opt web UI to monitor throughput rates, compression ratios, and to set bandwidth restrictions.

Migration bandwidth throttling

Before you migrate VMs, do an assessment of the network link that is used. Work with the networking specialists for the network that contains the source instance of vSphere or review weekly and monthly traffic use. Limit available bandwidth for migrations if that traffic traverses over a link that is critical to your business, especially if that link is less than 1 Gbps. Use the side with the most constrained bandwidth, which is typically the client side.

Do so when you deploy the fleet components in the HCX Client UI, but post deployment requires you go into the WAN opt UI.

Changes are lost if you redeploy the HCX CGW from the HCX web UI. This sets bandwidth limits for the migration traffic only. Stretched L2 traffic is not affected by this setting.

Log in to the WAN Opt Web UI.
From the Configuration tab, select Shaper from the menu.
In the Max bandwidth box, set the maximum bandwidth available to the WAN Opt appliance in Kbps. Do not exceed the maximum bandwidth of the WAN link. To set the value in Mbps, multiply by 1,000. To set the value in Gbps, multiply by 1,000,000. The default value is 10 Gbps (10,000,000 Kbps).
Click Apply.

For stretched L2 bandwidth throttling, QoS can be employed for UDP 500 and 4500 for the tunnel traffic between the L2C appliances.

Monitoring HCX components

Monitor HCX components such as HCX Manager, Cloud Gateway, WAN opt, and the Layer 2 Concentrator operations in the following ways:

Configure HCX Manager to send logs to a syslog server. Use the HCX Manager appliance management utility to run the https://<hcxhostname or IP>:9443 command.
Set up a ping to a VM that is migrated before network swing for each stretched L2 network.
Monitor the HCX component VM health with VMware Aria® Operations™ Manager or other VMware VM monitoring tools.

Bandwidth use

Use the following methods to monitor bandwidth use and latency.

vMotion traffic is best accomplished by using the WAN Opt web UI. The WAN Opt reduces the traffic that is going over the WAN and reduces packet loss by sending redundant packets. IThe typical ratio LAN to WAN bandwidth usage is approximately 3:1 (350 Mbps LAN = 90-120Mbps WAN).
Replication-based (bulk) migration of VMs within HCX results in VMs being moved with thick provisioning. While this method is not desirable, the WAN opt UI reveals a high ratio between LAN and WAN use when you move unused disk data. Conversely, it is observed that when non-compressible data is migrated, such as DB data and digital media, WAN use is at its highest and it comes closer to LAN use.

Observations

The vMotion migration of a VM within HCX does not generate more throughput than the vMotion networking for a single ESXi host.
As bulk migration can have multiple migrations in flight simultaneously, it achieves higher bandwidth use than a vMotion migration. The ratio observed at a customer side with 1 Gbps vMotion links to the ESX hosts was of 8 replications = bandwidth use of 1 vMotion.
As a result of moving empty space on disk, a high LAN use with a high ratio is displayed, therefore, low WAN use. 1 Gbps seems to be the limit. Indeed, in this particular case the vMotion network is only capable of 1 Gbps, which is the bottleneck.
For the vMotion migration of a multi TB Oracle database with a WAN link of 1 Gbps, the limitation is the vMotion network of 1 Gbps.

Stretched Layer 2 traffic

The HCX fleet component Layer 2 Concentrator has a bandwidth limitation of approximately 4 Gbps for all L2 network traffic that traverses it. Individually stretched networks have a bandwidth limit of approximately 1 Gbps or less depending on the traffic type. It is possible to have many stretched L2 networks across a single L2C pair (theoretical allowable max of 4096 networks per L2C pair). The L2C is engineered to detect and protect small traffic flows so that they are not overcome by large flows within the same L2C pair. It can be advantageous to identify if this situation is occurring and start more L2Cs to increase overall bandwidth capability.

Deploying multiple L2Cs can also be advantageous where multiple paths exist between the customer site and IBM Cloud®, such as direct link and internet. A single network cannot be made redundant or given increased bandwidth across multiple L2C pairs.

Monitor the traffic across all interfaces that use the Monitoring tab of the L2C VM. If the total data rate is approaching 8 Gbps (4 Gbps in / out) consider adding another L2 pair and redistribute stretched networks to rebalance.