Planning considerations for cluster networks

Considerations for cluster subnets

• Instance profiles have a recommended number of cluster subnets. The IBM Cloud® recommendation for cluster subnets for a given instance type is:

  Recommended number of cluster subnets for instance profiles.

  Instance Profile Type	Recommended Cluster Subnets to Cluster vNICs
  gx3d-160x1792x8h100	1:1

  • gx3d-160x1792x8h100 supports either 8, 16 or 32 cluster vNICs. As a result, you should have a corresponding number of cluster subnets.
  • If you are unsure about the correct number of cluster vNICs for your workload, IBM Cloud recommends using 8 for the gx3d-160x1792x8h100 profile.

  Ensure that the IP address ranges used for your cluster network subnets do not overlap with those of your cloud subnets. Overlapping ranges can lead to networking issues.

• Start by setting up your cluster network and subnets, then define instance templates to scale your training cluster with the desired number of nodes. For more information, see Getting started with cluster networks.

  • You can create an instance template to define instance details for provisioning one or more virtual servers. When you create your instance template, you can use it to provision single virtual server instances, or to provision multiple instances at the same time as part of an instance group.
  • Optionally, when you provision an instance, you can include the cluster network configuration within the instance provision request.

Considerations for IP address ranges (prefixes)

You can only apply the following considerations through the API and CLI.

• Cluster networks are isolated from the VPC network. You are allowed to specify cluster nework subnet_prefixes that overlap with the VPC's default or custom address prefixes. However, this may result in instances having two network interfaces (one on the VPC network, and one on the cluster network) with the same IP address. In such a scenario you must be careful to isolate the network interfaces in the guest operating system. For example, Linux guests can use network namespaces.

• The default subnet_prefixes for cluster networks (["cidr": "10.0.0.0/9"]) do not overlap with the default address prefixes for VPCs. To simplify the guest operating system configurations, either use the defaults, or plan your custom address ranges to avoid overlap.

• If you need to plan for statically-configured IP addresses, especially for a large number (such as 1000), it’s important to approach the task methodically. First, you should determine the total number of required IP addresses. Then consider how these addresses will be assigned to various instances. To manage this efficiently, implement a clear naming scheme that helps you organize and connect the IP addresses to their respective instances. For example, you might use a naming convention that reflects the function or location of each instance, making it easier to identify and manage them.

• When statically configuring IP addresses for network interfaces in a guest OS:

  • Ensure that the IP addresses for cluster network interfaces are in the address range of one of the subnet_prefixes in the cluster network. Cluster network interfaces with an IP address outside the address range of one of the subnet_prefixes will not receive packets.
  • If a statically configured IP address does not match the cluster network interface's primary reserved IP or one of its secondary reserved IPs, you must create a VPC routing table route to redirect traffic for the statically configured IP to one of the reserved IPs. Additionally, you must set allow_ip_spoofing to true on the associated cluster network interface to allow outbound packets with the statically configured IP as the source address.