SAP S/4HANA HA Deployment using IBM Schematics

Description

This automation solution is designed for the deployment of SAP S/4HANA HA cluster solution using IBM Cloud Schematics. The SAP solution will be deployed on top of Red Hat Enterprise Linux 8.x in an existing IBM Cloud Gen2 VPC, using an existing bastion host with secure remote SSH access.

The solution is based on Terraform remote-exec and Ansible playbooks executed by Schematics and it is implementing a 'reasonable' set of best practices for SAP VSI host configuration.

It contains:

• Terraform scripts for deploying one Power Placement group to include all the 4 VMs involved in this solution.
• Terraform scripts for deploying four VSIs in an EXISTING VPC with Subnet and Security Group configs. The VSIs scope: two for the HANA database cluster instance and two for the SAP application cluster.
• Terraform scripts for deploying and configuring three Application Load Balancers like HANA DB, SAP ASCS/ERS.
• Terraform scripts for deploying and configuring one VPC DNS service used to map the ALB FQDN to the SAP ASCS/ERS and Hana Virtual hostnames.
• Terraform scripts for deploying and configuring seven File shares for VPC.
• Bash scripts used for checking the prerequisites required by SAP VSIs deployment and for the integration into a single step in IBM Schematics GUI of the VPC virtual resources provisioning and the SAP S/4HANA HA cluster solution installation.
• Ansible scripts for OS requirements installation and configuration for SAP applications.
• Ansible scripts for cluster components installation.
• Ansible scripts for SAP application cluster configuration and SAP HANA cluster configuration.
• Ansible scripts for HANA installation.
• Ansible scripts for HANA DB backup.
• Ansible scripts for HANA system replica configuration.
• Ansible scripts for ASCS and ERS instances installation.
• Ansible scripts for DB load.
• Ansible scripts for primary and additional application servers installation.

Installation media

SAP HANA installation media used for this deployment is the default one for SAP HANA, platform edition 2.0 SPS05 available at SAP Support Portal under INSTALLATION AND UPGRADE area and it has to be provided manually in the input parameter file.

SAP S/4HANA installation media used for this deployment is the default one for SAP S/4HANA 2020 available at SAP Support Portal under INSTALLATION AND UPGRADE area and it has to be provided manually in the input parameter file.

SAP Software Provisioning Manager used for this solution is 2.0 SP13 and it's recommended to use the same version or higher.

VSI Configuration

The VSIs are configured with Red Hat Enterprise Linux 8 for SAP HANA (amd64) and they have: at least two SSH keys configured to access as root user and the following storage volumes created for DB and SAP APP VSI:

HANA DBs VSI Disks:

• 3 x 500 GB disks with 10 IOPS / GB - DATA
• 1 x 10 GB disk - SWAP

SAP APPs VSI Disks:

• 1x 40 GB disk with 10 IOPS / GB - SWAP

File Shares:

• 6 x 20GB file shares - DATA
• 1 x 80GB file shares -DATA

IBM Cloud API Key

The IBM Cloud API Key should be provided as input value of type sensitive for "ibmcloud_api_key" variable, in IBM Schematics -> Workspaces -> -> Settings menu.
The IBM Cloud API Key can be created here.

Input parameters

The following parameters can be set in the Schematics workspace: VPC, Subnet, Security group, Resource group, Hostnames, Domain Name, Profile, Image, File Shares, SSH Keys and your SAP system configuration variables, as below:

VPC Infra input parameters:

Parameter	Description
ibmcloud_api_key	IBM Cloud API key (Sensitive* value).
private_ssh_key	id_rsa private key content (Sensitive* value).
SSH_KEYS	List of SSH Keys UUIDs that are allowed to SSH as root to the VSI. Can contain one or more IDs. The list of SSH Keys is available here. Sample input (use your own SSH UUIDs from IBM Cloud): [ "r010-57bfc315-f9e5-46bf-bf61-d87a24a9ce7a" , "r010-3fcd9fe7-d4a7-41ce-8bb3-d96e936b2c7e" ]
BASTION_FLOATING_IP	The FLOATING IP from the Bastion Server.
RESOURCE_GROUP	The name of an EXISTING Resource Group for VSIs and Volumes resources. Default value: "Default". The list of Resource Groups is available here.
REGION	The cloud region where to deploy the solution. The regions and zones for VPC are listed here. Review supported locations in IBM Cloud Schematics here. Sample value: eu-de.
ZONE	The cloud zone where to deploy the solution. Sample value: eu-de-2.
VPC	The name of an EXISTING VPC. The list of VPCs is available here
SUBNET	The name of an EXISTING Subnet. The list of Subnets is available here.
SECURITY_GROUP	The name of an EXISTING Security group. The list of Security Groups is available here.
DOMAIN_NAME	The Domain Name used for DNS and ALB. Duplicates are not allowed. The list with DNS resources can be searched here. Sample value: "example.com". (See Obs.*)
SHARE PROFILES	IOPS per GB tier for File Share storage. Valid values are 3, 5, and 10. For more info about file share profiles, check here. Default value: share_profile = "tier-5iops".
SHARE SIZES	Custom File Shares Sizes for SAP mounts. Sample values: usrsap-sapmnt = "20" , usrsap-trans = "80".
[DB/APP]- VIRT-HOSTNAMES	ASCS/ERS/HANA virtual hostnames. Default values: "sap($your_sap_sid)ascs/ers" , "sap($your_sap_sid)ers" , "db($your_hana_sid)hana".
[DB/APP]-HOSTNAMES	SAP HANA/APP Cluster VSI Hostnames. Each hostname should be up to 13 characters as required by SAP. For more information on rules regarding hostnames for SAP systems, check SAP Note 611361: Hostnames of SAP ABAP Platform servers. Default values: APP-HOSTNAME-1/2 = "sapapp-$your_sap_sid-1/2" , DB-HOSTNAME-1/2 = "hanadb-$your_hana_sid-1/2".
[DB/APP]-PROFILES	The profile used for the HANA/APP VSI. A list of profiles is available here. For more information about supported DB/OS and IBM Gen 2 Virtual Server Instances (VSI), check SAP Note 2927211: SAP Applications on IBM Virtual Private Cloud Default values: DB-PROFILE = "mx2-16x128" , APP-PROFILE = "bx2-4x16".
[DB/APP]-IMAGE	The OS image used for the HANA/APP VSI. You must use the Red Hat Enterprise Linux 8 for SAP HANA (amd64) image for all VMs as this image contains the required SAP and HA subscriptions. A list of images is available here Default value: "ibm-redhat-8-6-amd64-sap-hana-2"

SAP input parameters:

Parameter	Description	Requirements
hana_sid	The SAP system ID identifies the SAP HANA system. (See Obs.*)	Consists of exactly three alphanumeric characters Has a letter for the first character Does not include any of the reserved IDs listed in SAP Note 1979280
hana_sysno	Specifies the instance number of the SAP HANA system	Two-digit number from 00 to 97 Must be unique on a host
hana_system_usage	System Usage	Default: custom Valid values: production, test, development, custom
hana_components	SAP HANA Components	Default: server Valid values: all, client, es, ets, lcapps, server, smartda, streaming, rdsync, xs, studio, afl, sca, sop, eml, rme, rtl, trp
kit_saphana_file	Path to SAP HANA ZIP file	As downloaded from SAP Support Portal. Default value: "/storage/HANADB/51055299.ZIP"
sap_sid	The SAP system ID identifies the entire SAP system. (See Obs.*)	Consists of exactly three alphanumeric characters Has a letter for the first character Does not include any of the reserved IDs listed in SAP Note 1979280
sap_ascs_instance_number	Technical identifier for internal processes of ASCS	Two-digit number from 00 to 97 Must be unique on a host
sap_ers_instance_number	Technical identifier for internal processes of ERS	Two-digit number from 00 to 97 Must be unique on a host
sap_ci_instance_number	Technical identifier for internal processes of PAS	Two-digit number from 00 to 97 Must be unique on a host
sap_aas_instance_number	Technical identifier for internal processes of AAS	Two-digit number from 00 to 97 Must be unique on a host
hdb_concurrent_jobs	Number of concurrent jobs used to load and/or extract archives to HANA Host	Default: 23
kit_sapcar_file	Path to sapcar binary	As downloaded from SAP Support Portal. Default value: "/storage/S4HANA/SAPCAR_1010-70006178.EXE"
kit_swpm_file	Path to SWPM archive (SAR)	As downloaded from SAP Support Portal. Default value: "/storage/S4HANA/SWPM20SP13_1-80003424.SAR"
kit_sapexe_file	Path to SAP Kernel OS archive (SAR)	As downloaded from SAP Support Portal. Default value: "/storage/S4HANA/SAPEXE_100-70005283.SAR"
kit_sapexedb_file	Path to SAP Kernel DB archive (SAR)	As downloaded from SAP Support Portal. Default value: "/storage/S4HANA/SAPEXEDB_100-70005282.SAR"
kit_igsexe_file	Path to IGS archive (SAR)	As downloaded from SAP Support Portal. Default value: "/storage/S4HANA/igsexe_1-70005417.sar"
kit_igshelper_file	Path to IGS Helper archive (SAR)	As downloaded from SAP Support Portal. Default value: "/storage/S4HANA/igshelper_17-10010245.sar"
kit_saphostagent_file	Path to SAP Host Agent archive (SAR)	As downloaded from SAP Support Portal. Default value: "/storage/S4HANA/SAPHOSTAGENT51_51-20009394.SAR"
kit_hdbclient_file	Path to HANA DB client archive (SAR)	As downloaded from SAP Support Portal. Default value: "/storage/S4HANA/IMDB_CLIENT20_009_28-80002082.SAR"
kit_s4hana_export	Path to S/4HANA Installation Export dir	The archives downloaded from SAP Support Portal should be present in this path. Default value: "/storage/S4HANA/export"

SAP Passwords:
(Sensitive* values)

Parameter	Description	Requirements
sap_main_password	Common password for all users that are created during the installation	It must be 8 to 14 characters long It must contain at least one digit (0-9) It must not contain \ (backslash) and " (double quote)
hana_main_password	HANA system master password	It must be 8 to 14 characters long It must contain at least one digit (0-9) It must not contain \ (backslash) and " (double quote) Master Password must contain at least one upper-case character
ha_password	HA cluster password	It must be 8 to 14 characters long It must contain at least one digit (0-9) It must not contain \ (backslash) and " (double quote)

Obs*:

• Sensitive - The variable value is not displayed in your Schematics logs and it is hidden in the input field.
  
• The following parameters should have the same values as the ones set for the BASTION server: REGION, ZONE, VPC, SUBNET, SECURITYGROUP.
• DOMAIN_NAME variable rules:
  • it should contain at least one "." as a separator. It is a private domain and it is not reacheable to and from the outside world.
  • it could be like a subdomain name. Ex.: staging.example.com
  • it can only use letters, numbers, and hyphens.
  • hyphens cannot be used at the beginning or end of the domain name.
  • it can't be used a domain name that is already in use.
  • domain names are not case sensitive.
• The following SAP "SID" values are reserved and are not allowed to be used: ADD, ALL, AMD, AND, ANY, ARE, ASC, AUX, AVG, BIT, CDC, COM, CON, DBA, END, EPS, FOR, GET, GID, IBM, INT, KEY, LOG, LPT, MAP, MAX, MIN, MON, NIX, NOT, NUL, OFF, OLD, OMS, OUT, PAD, PRN, RAW, REF, ROW, SAP, SET, SGA, SHG, SID, SQL, SUM, SYS, TMP, TOP, UID, USE, USR, VAR",

VPC Configuration

The Security Rules inherited from BASTION deployment are the following:

• Allow all traffic in the Security group for private networks.
• Allow outbound traffic (ALL for port 53, TCP for ports 80, 443, 8443)
• Allow inbound SSH traffic (TCP for port 22) from IBM Schematics Servers.

Files description and structure:

• modules - directory containing the terraform modules
• main.tf - contains the configuration of the VSI for the deployment of the current SAP solution.
• output.tf - contains the code for the information to be displayed after the VSI is created (Hostname, Private IP)
• integration*.tf - contains the integration code that makes the SAP variabiles from Terraform available to Ansible.
• provider.tf - contains the IBM Cloud Provider data in order to run terraform init command.
• terraform.tfvars - contains the IBM Cloud API key referenced in provider.tf (dynamically generated)
• variables.tf - contains variables for the VPC and VSI
• versions.tf - contains the minimum required versions for terraform and IBM Cloud provider.

Steps to follow:

1. Make sure that you have the required IBM Cloud IAM
   permissions to
   create and work with VPC infrastructure and you are assigned the
   correct
   permissions to
   create the workspace in Schematics and deploy resources.

2. Generate an SSH
   key.
   The SSH key is required to access the provisioned VPC virtual server
   instances via the bastion host. After you have created your SSH key,
   make sure to upload this SSH key to your IBM Cloud
   account in
   the VPC region and resource group where you want to deploy the SAP solution

3. Create the Schematics workspace:

   1. From the IBM Cloud menu
      select Schematics.
   • Click Create a workspace.
   • Enter a name for your workspace.
   • Click Create to create your workspace.
   2. On the workspace Settings page, enter the URL of this solution in the Schematics examples Github repository.
   • Select the latest Terraform version.
   • Click Save template information.
   • In the Input variables section, review the default input variables and provide alternatives if desired.
   • Click Save changes.

4. From the workspace Settings page, click Generate plan

5. Click View log to review the log files of your Terraform
   execution plan.

6. Apply your Terraform template by clicking Apply plan.

7. Review the log file to ensure that no errors occurred during the
   provisioning, modification, or deletion process.

The output of the Schematics Apply Plan will list the public/private IP addresses
of the VSI host, the hostname and the VPC.

Related links:

• How to create a BASTION/STORAGE VSI for SAP in IBM Schematics
• Securely Access Remote Instances with a Bastion Host
• VPNs for VPC overview: Site-to-site gateways and Client-to-site servers.
• IBM Cloud Schematics