Localnet UDN examples for Red Hat OpenShift Virtualization

Network, namespace, and virtual server details

The following table summarizes the example network layout.

Diagram of the final setup

Each code block in the guide can be copied to a file and applied with oc apply -f <file-name>.yml. Some code blocks might cross page boundaries in PDF and Word formats, and some long lines might wrap in those formats.

The guide uses ic as a shorthand for ibmcloud in IBM Cloud CLI commands. Define alias ic=ibmcloud in your shell, or substitute ibmcloud for ic in the commands that follow.

Creating reserved IPs in each subnet

First, create a reserved IP in each subnet for your virtual servers:

ic is subnet-reserved-ip-create zone1-vm-prod --address 198.18.0.20 --name zone1-web00 --auto-delete false

ic is subnet-reserved-ip-create zone1-vm-db --address 198.18.1.20 --name zone1-db00 --auto-delete false

ic is subnet-reserved-ip-create zone1-vm-app --address 198.18.2.20 --name zone1-app00 --auto-delete false

Creating VNIs using the reserved IPs

Now create the VNIs using the reserved IP names from the previous step:

ic is virtual-network-interface-create --name zone1-web00 --allow-ip-spoofing false --auto-delete false --protocol-state-filtering-mode disabled --enable-infrastructure-nat true --subnet zone1-vm-prod --vpc my-vpc --rip zone1-web00 --sgs handclap-roundworm-clique-eccentric --resource-group-name Default

ic is virtual-network-interface-create --name zone1-db00 --allow-ip-spoofing false --auto-delete false --protocol-state-filtering-mode disabled --enable-infrastructure-nat true --subnet zone1-vm-db --vpc my-vpc --rip zone1-db00 --sgs handclap-roundworm-clique-eccentric --resource-group-name Default

ic is virtual-network-interface-create --name zone1-app00 --allow-ip-spoofing false --auto-delete false --protocol-state-filtering-mode disabled --enable-infrastructure-nat true --subnet zone1-vm-app --vpc my-vpc --rip zone1-app00 --sgs handclap-roundworm-clique-eccentric --resource-group-name Default

Virtual server plant-web00 in the green namespace on vlan20-prod

# Bare YAML to stand up a Red Hat OpenShift on IBM Cloud virtual machine
---
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: "plant-web00"
  namespace: "green"
  annotations:
    description: "example vm plant-web00"
  labels:
    app: "plant-web00-green-server"
    kubevirt.io/dynamic-credentials-support: 'true'
    vm.kubevirt.io/template: "centos-stream9-server-small"
    vm.kubevirt.io/template.namespace: openshift
    vm.kubevirt.io/template.revision: '1'
    vm.kubevirt.io/template.version: v0.34.0
spec:
  dataVolumeTemplates:
    - apiVersion: cdi.kubevirt.io/v1beta1
      kind: DataVolume
      metadata:
        creationTimestamp: null
        name: "plant-web00-green-server"
      spec:
        sourceRef:
          kind: DataSource
          name: "centos-stream9"
          namespace: openshift-virtualization-os-images
        storage:
          resources:
            requests:
              storage: 30Gi
  runStrategy: RerunOnFailure
  template:
    metadata:
      annotations:
        vm.kubevirt.io/flavor: "small"
        vm.kubevirt.io/os: "centos-stream9"
        vm.kubevirt.io/workload: "server"
      labels:
        kubevirt.io/domain: example
        kubevirt.io/size: "small"
    spec:
      domain:
        cpu:
          cores: 1
          sockets: 1
          threads: 1
        devices:
          disks:
            - disk:
                bus: virtio
              name: rootdisk
            - disk:
                bus: virtio
              name: cloudinitdisk
          interfaces:
            - model: virtio
              name: default
              state: up
              bridge: {}
              macAddress: 02:00:01:00:98:A0 # <- replace with the MAC of VNI zone1-web00
          networkInterfaceMultiqueue: true
          rng: {}
        memory:
          guest: "2Gi"
      hostname: "plant-web00"
      networks:
        - name: default
          multus:
            networkName: vlan20-prod
      terminationGracePeriodSeconds: 180
      volumes:
        - dataVolume:
            name: "plant-web00-green-server"
          name: rootdisk
        - cloudInitNoCloud:
            userData: |-
              #cloud-config
              user: admin
              password: "aRandomPassword9292"
              ssh_authorized_keys:
                 - "ssh-ed25519 AAAA....BBBB....CCCCC.....DDDD put-your-key-here@example.local"
              chpasswd: { expire: False }
              runcmd:
                - [ dnf, install, -y, epel-release ]
                - [ dnf, install, -y, iperf3, nmap-ncat, darkhttpd ]
          name: cloudinitdisk

Virtual server plant-db00 in the green namespace on vlan21-db

---
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: "plant-db00"
  namespace: "green"
  annotations:
    description: "example vm plant-db00"
  labels:
    app: "plant-db00-green-server"
    kubevirt.io/dynamic-credentials-support: 'true'
    vm.kubevirt.io/template: "centos-stream9-server-small"
    vm.kubevirt.io/template.namespace: openshift
    vm.kubevirt.io/template.revision: '1'
    vm.kubevirt.io/template.version: v0.34.0
spec:
  dataVolumeTemplates:
    - apiVersion: cdi.kubevirt.io/v1beta1
      kind: DataVolume
      metadata:
        creationTimestamp: null
        name: "plant-db00-green-server"
      spec:
        sourceRef:
          kind: DataSource
          name: "centos-stream9"
          namespace: openshift-virtualization-os-images
        storage:
          resources:
            requests:
              storage: 30Gi
  runStrategy: RerunOnFailure
  template:
    metadata:
      annotations:
        vm.kubevirt.io/flavor: "small"
        vm.kubevirt.io/os: "centos-stream9"
        vm.kubevirt.io/workload: "server"
      labels:
        kubevirt.io/domain: example
        kubevirt.io/size: "small"
    spec:
      domain:
        cpu:
          cores: 1
          sockets: 1
          threads: 1
        devices:
          disks:
            - disk:
                bus: virtio
              name: rootdisk
            - disk:
                bus: virtio
              name: cloudinitdisk
          interfaces:
            - model: virtio
              name: default
              state: up
              bridge: {}
              macAddress: 02:00:01:00:98:A5 # <- replace with the MAC of VNI zone1-db00
          networkInterfaceMultiqueue: true
          rng: {}
        memory:
          guest: "2Gi"
      hostname: "plant-db00"
      networks:
        - name: default
          multus:
            networkName: vlan21-db
      terminationGracePeriodSeconds: 180
      volumes:
        - dataVolume:
            name: "plant-db00-green-server"
          name: rootdisk
        - cloudInitNoCloud:
            userData: |-
              #cloud-config
              user: admin
              password: "aRandomPassword9292"
              ssh_authorized_keys:
                 - "ssh-ed25519 AAAA....BBBB....CCCCC.....DDDD put-your-key-here@example.local"
              chpasswd: { expire: False }
              runcmd:
                - [ dnf, install, -y, epel-release ]
                - [ dnf, install, -y, iperf3, nmap-ncat, darkhttpd ]
          name: cloudinitdisk

Virtual server plant-app00 in the red namespace on vlan22-app

---
apiVersion: kubevirt.io/v1
kind: VirtualMachine
metadata:
  name: "plant-app00"
  namespace: "red"
  annotations:
    description: "example vm plant-app00"
  labels:
    app: "plant-app00-red-server"
    kubevirt.io/dynamic-credentials-support: 'true'
    vm.kubevirt.io/template: "centos-stream9-server-small"
    vm.kubevirt.io/template.namespace: openshift
    vm.kubevirt.io/template.revision: '1'
    vm.kubevirt.io/template.version: v0.34.0
spec:
  dataVolumeTemplates:
    - apiVersion: cdi.kubevirt.io/v1beta1
      kind: DataVolume
      metadata:
        creationTimestamp: null
        name: "plant-app00-red-server"
      spec:
        sourceRef:
          kind: DataSource
          name: "centos-stream9"
          namespace: openshift-virtualization-os-images
        storage:
          resources:
            requests:
              storage: 30Gi
  runStrategy: RerunOnFailure
  template:
    metadata:
      annotations:
        vm.kubevirt.io/flavor: "small"
        vm.kubevirt.io/os: "centos-stream9"
        vm.kubevirt.io/workload: "server"
      labels:
        kubevirt.io/domain: example
        kubevirt.io/size: "small"
    spec:
      domain:
        cpu:
          cores: 1
          sockets: 1
          threads: 1
        devices:
          disks:
            - disk:
                bus: virtio
              name: rootdisk
            - disk:
                bus: virtio
              name: cloudinitdisk
          interfaces:
            - model: virtio
              name: default
              state: up
              bridge: {}
              macAddress: 02:00:01:00:98:AA # <- replace with the MAC of VNI zone1-app00
          networkInterfaceMultiqueue: true
          rng: {}
        memory:
          guest: "2Gi"
      hostname: "plant-app00"
      networks:
        - name: default
          multus:
            networkName: vlan22-app
      terminationGracePeriodSeconds: 180
      volumes:
        - dataVolume:
            name: "plant-app00-red-server"
          name: rootdisk
        - cloudInitNoCloud:
            userData: |-
              #cloud-config
              user: admin
              password: "aRandomPassword9292"
              ssh_authorized_keys:
                 - "ssh-ed25519 AAAA....BBBB....CCCCC.....DDDD put-your-key-here@example.local"
              chpasswd: { expire: False }
              runcmd:
                - [ dnf, install, -y, epel-release ]
                - [ dnf, install, -y, iperf3, nmap-ncat, darkhttpd ]
          name: cloudinitdisk

Tests

Run the following tests to verify the network setup.

1. Start a simple HTTP server on each virtual server.

   • From 198.18.0.20, run darkhttpd /usr/share/doc/bash --daemon.
   • From 198.18.1.20, run darkhttpd /usr/share/doc/bash --daemon.
   • From 198.18.2.20, run darkhttpd /usr/share/doc/bash --daemon.

2. From 198.18.0.20, run curl -v http://198.18.1.20:8080.

3. From 198.18.1.20, run curl -v http://198.18.0.20:8080.

4. From the jump virtual server, run curl -v http://198.18.2.20:8080.

5. From 198.18.1.20, start an iperf3 server: iperf3 -s -p 9090.

6. From 198.18.0.20, connect to the iperf3 server in the green namespace: iperf3 -c 198.18.1.20 -p 9090.

7. Ping from 198.18.1.20 to 198.18.0.20 and vice versa.

8. Rerun the preceding tasks by using 198.18.2.20 and the jump virtual server's IP address.

Applying a multi-network policy and rerunning the tests

Apply a multi-network policy that allows only TCP port 8080 for the vlan20-prod network. Save the following manifest to a file named my-policy.yml. Then, run oc apply -f my-policy.yml to apply it.

---
apiVersion: k8s.ovn.org/v1alpha1
kind: MultiNetworkPolicy
metadata:
  name: allow-8080-tcp
  namespace: green
  annotations:
    k8s.v1.cni.cncf.io/policy-for: vlan20-prod
spec:
  podSelector: {}
  ingress:
    - ports:
        - protocol: TCP
          port: 8080
  egress:
    - ports:
        - protocol: TCP
          port: 8080
  policyTypes:
    - Ingress
    - Egress

Rerun the Tests against plant-web00 on vlan20-prod. TCP port 8080 still works, but other traffic, such as ping and the iperf3 test on port 9090, no longer work on that network attachment.

Run oc delete -f my-policy.yml to remove the policy and restore connectivity. The tests pass again.