Storage design considerations

Network Block or File storage for VMware on Classic Infrastructure

Using VMware for SAP workloads on IBM Cloud is certified. However it requires choice of the storage and would use the "TDI" delivery model for which you would need to run validation checks to gain SAP Support. Therefore, it is important to consider the correct storage for your VMware hosts when they are running SAP workloads.

For VMware clusters, where SAP workloads are run across multiple VMware vSphere hypervisor nodes, storage must be shared across these hypervisor nodes.

VMware is available to work with Block storage or File storage from IBM Cloud. To help you select Block storage or File storage for running SAP on VMware, see VMware Technical Paper on Storage Protocol Comparison.

When you are using Network Block or File storage, do not expect that certification performance benchmarks to remain the same. Particularly after factoring in the hypervisor overheads as described in Compute Profiles of SAP-certified VMware on Classic Infrastructure.

For VMware datastores (where the virtual machine .VMDK virtual disks are located), the recommendations are:

• For SAP HANA, use Local SDD disks for the datastore in a RAID10 configuration
• For SAP HANA, with network storage, use 10 IOPS/GB with each vSphere node that hosts SAP that uses a network interface card with 10 Gbps connection
• For SAP NetWeaver or SAP AnyDB, with network storage, use at least 4 IOPS/GB with each vSphere node that hosts SAP that uses a network interface card with 10 Gbps connection

To achieve maximum IOPS on a volume, adequate network resources need to be in place. Other considerations include private network usage outside of storage and host side, and application-specific tunings (for example, IP stacks and queue depths). For more information, see Getting started with Block Storage and Getting started with File Storage for more information on storage tiers and performance.

The storage to use with either the VMware manual setup (Bare Metal with VMware OS Image) or VMware automated setup (IBM Cloud for VMware Solutions Dedicated), is described in:

• Storage to use with VMware vSphere on IBM Cloud Bare Metals provides further direction on how to integrate storage in an ESX environment.
• Storage to use with IBM Cloud for VMware Solutions Dedicated

Block Storage for Virtual Servers on VPC Infrastructure

For network storage, IOPS per GB is limited and performance depends on the workload. For relational database management systems (RDBMS), it might be advisable to use the same volume for both the database's log and data storage. This setup depends on the behavior of your application.

In general, for typical RDBMS-based applications, a 5 IOPS/GB profile is reasonable.

If your application uses dedicated key performance indicators (KPIs) on storage performance, test the storage throughput before you begin software deployment. By using volume manager-based software RAID (like LVM), you meet almost every KPI.

Disk mount points and volumes for IBM Db2

After you attach the two data volumes, two new virtual disks will appear in the Virtual Server, see the following table. In this example, those disks are vdd, vde, and vdf.

Table 1 shows a basic layout of the file system to support an IBM Db2 installation. Generally, an IBM Db2 installation uses subdirectories that can be segmented into independent volumes.

For example, "/db2/<DBSID>", "/db2/<DBSID>/log_dir", and several "sapdata<n>", where the folder "log_dir" contains the online logs files of the database and the "sapdata<n>" contains the data itself. For example, see the Db2 documentation here: Required File Systems for IBM Db2 for Linux, UNIX, and Windows.

mx2-8x64, mx2-16x128 and mx2-32x256 profiles

The mx2-8x64 profile is certified for SAP Business One on HANA only.

For Virtual Server created based on the mx2-8x64, mx2-16x128 and mx2-32x256 profiles, there are:

• 3x 500 GB volumes; three block storage volumes of 500 GB size, with a custom volume profile that supports up to 10,000 Max IOPS attached to the Virtual Server
• 1x 2,000 GB volume; one block storage volume of 2,000 GB size, with a lower 4,000 IOPS (medium performance) attached to the Virtual Server for backups

After attaching the three data volumes, three new virtual disks will appear in the virtual server, see the table that follows. In this example, those disks are vdd, vde and vdf.

The disks are visible in the operating system of the virtual server as follows:

[root@hana256-vsi ~]# fdisk -l

Disk /dev/vdd: 536.9 GB, 536870912000 bytes, 1048576000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/vde: 536.9 GB, 536870912000 bytes, 1048576000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/vdf: 536.9 GB, 536870912000 bytes, 1048576000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

These three disks must be managed under the Linux® Logical Volume Manager (LVM), and deployed as logical volumes. In order to achieve that, first put the three devices under LVM control. For example, make them physical volumes:

[root@hana256-vsi ~]# pvcreate /dev/vdd /dev/vde /dev/vdf

Then, create a volume group from the physical volumes. The name of the volume group can be chosen according to your preferences, in our sample it is hana_vg:

[root@hana256-vsi ~]# vgcreate hana_vg /dev/vdd /dev/vde /dev/vdf

After creating the volume group, three logical volumes need to be defined on top. These logical volumes reflect the file system size requirements for SAP HANA. The following commands are for a 256 GB virtual server:

[root@hana256-vsi ~]# lvcreate -i 3 -I 64K -L 256GB -n hana_log_lv hana_vg
[root@hana256-vsi ~]# lvcreate -i 3 -I 64K -L 256GB -n hana_shared_lv hana_vg
[root@hana256-vsi ~]# lvcreate -i 3 -I 64K -l 100%FREE -n hana_data_lv hana_vg

For a 128 GB virtual server, in the example above -L 256GB must be replaced by -L 128GB and for 64 GB by -L 64GB accordingly. These commands will not result in the smallest possible file system size, but they create the smallest configuration, which will fulfill the SAP HANA KPIs. Finally, a file system needs to be created on top of each volume group:

[root@hana256-vsi ~]# mkfs.xfs /dev/mapper/hana_vg-hana_log_lv
[root@hana256-vsi ~]# mkfs.xfs /dev/mapper/hana_vg-hana_data_lv
[root@hana256-vsi ~]# mkfs.xfs /dev/mapper/hana_vg-hana_shared_lv

The following entries to /etc/fstab will mount the file systems after their mount points (/hana/data, /hana/log and /hana/shared) have been created:

/dev/mapper/hana_vg-hana_log_lv    /hana/log xfs defaults,swalloc,nobarrier,inode64
/dev/mapper/hana_vg-hana_shared_lv /hana/shared xfs defaults,inode64 0 0
/dev/mapper/hana_vg-hana_data_lv   /hana/data xfs defaults,largeio,swalloc,inode64 0 0

mx2-48x384 profile

For Virtual Server created based on the mx2-48x384 profile there are:

• 3x 500 GB volumes; three block storage volumes of 500 GB size, with a custom volume profile that supports up to 10,000 Max IOPS attached to the Virtual Server is required
• 4x 100 GB volumes; four block storage volumes of 100 GB size, with a custom volume profile that supports up to 6,000 Max IOPS attached to the Virtual Server is required
• Optional: 1x 2,000 GB volume; one block storage volume of 2,000 GB size, with a lower 4,000 IOPS (medium performance) attached to the Virtual Server for backups

After attaching the seven data volumes, seven new virtual disks will appear in the Virtual Server, see the table that follows. In this example, those disks are vdd, vde, vdf, vdg, vdh, vdi, vdj.

These three disks must be managed under the Linux® Logical Volume Manager (LVM), and deployed as logical volumes. In order to achieve that, first put the three devices under LVM control. For example, make them physical volumes:

[root@hana384-vsi ~]# pvcreate /dev/vd[d,e,f,g,h,i,j]

Then, two different volume groups need to be created:

[root@hana384-vsi ~]# vgcreate hana_vg /dev/vdh /dev/vdi /dev/vdj
[root@hana384-vsi ~]# vgcreate hana_log_vg /dev/vdd /dev/vde /dev/vdf /dev/vdg

Next, three logical volumes need to be defined on top. These logical volumes reflect the file system size requirements for SAP HANA. The following commands are for a 384 GB virtual server:

[root@hana384-vsi ~]# lvcreate -l 100%VG -i 4 -I 64K  -n hana_log_lv hana_log_vg
[root@hana384-vsi ~]# lvcreate -i 3 -L 384G -I 64K -n hana_shared_lv hana_vg
[root@hana384-vsi ~]# lvcreate -i 3 -l 100%FREE  -I 64K -n hana_data_lv hana_vg

Finally, a file system needs to be created on top of each volume group:

[root@hana384-vsi ~]# mkfs.xfs /dev/mapper/hana_log_vg-hana_log_lv
[root@hana384-vsi ~]# mkfs.xfs /dev/mapper/hana_vg-hana_data_lv
[root@hana384-vsi ~]# mkfs.xfs /dev/mapper/hana_vg-hana_shared_lv

The following entries to /etc/fstab mount the file systems after their mount points (/hana/data, /hana/log and /hana/shared) have been created:

/dev/mapper/hana_log_vg-hana_log_lv    /hana/log xfs defaults,swalloc,nobarrier,inode64
/dev/mapper/hana_vg-hana_shared_lv /hana/shared xfs defaults,inode64 0 0
/dev/mapper/hana_vg-hana_data_lv   /hana/data xfs defaults,largeio,swalloc,inode64 0 0