Migrating SAP S/4HANA to IBM Power Virtual Server
Steps before you migrate an SAP S/4HANA database
The following sections cover several important advisories to prepare for the SAP HANA database migration. Read and implement the relevant SAP notes.
Before you attempt any data migration or replication action, check the source database for any existing issues.
If issues exist, it might be one of the following issues.
- Interrupted or failed garbage collection.
- Source databases still contain entries, tables, or data from actions such as client deletion, which leads to a false positive on the true size of the database.
- Reported inconsistencies during the database check.
- Extreme load or unload actions that lead to orphaned entries.
- Hardware issues that occurred during a delta merge from memory to disk.
- Excessive page memory dumps were detected, which can indicate page corruptions.
- Alerts that are displayed during an SAP HANA Mini Check.
Incorrectly performing any data migration or replication action can result in data loss and application inconsistencies. Make sure that you read and understand the associated SAP Notes and correction notes before you perform any related task. IBM Cloud® is not liabil for any data loss or application integrity.
The following sections contain SAP-advised pre-steps to help make sure that the source database is in a consistent state. Before any migration or backup or recovery operation begins that the consistency (such as Row store, Column Store, Pages) and trace files that are on the source database are closely examined for any existing issues. These recommended steps must be completed before you start the migration.
Checking and confirming database health
Check the health of your database to reduce the risk of the transferring existing issues to your target system. Health checks prevent pre-existing issues (such as consistency or block corruption) from migrating onto the target SAP HANA system. SAP HANA System Replication can't help you in this scenario, so it is important to perform these necessary checks. Use the following SAP Notes to assist you.
- SAP Note 2116157 - FAQ: SAP HANA Consistency Checks and Corruptions
- SAP Note 2272121 - How To: Analyzing Physical Corruptions with the SAP HANA Persistence Diagnosis Tool
- SAP Note 2380176 - FAQ: SAP HANA Database Trace
Checking the database trace files
The database trace is written to service specific files on operating system level. The trace directory is located here:
/usr/sap/<SID>/HDB<inst>/<host>/trace/DB_<SID>/
The following alias in the environment of the <sid>adm
user helps you to quickly switch to the trace directory on the OS level:
cdtrace
The database trace files use the following naming convention:
<service>_<host>.<port>.<counter>.trc
In the context of dynamic tiering, a file with the following convention can exist (SAP Note 2871785):
esserver_console_<host>.<port>.<counter>.trc
Example:
indexserver_saphana01.30003.024.trc
You can access these files either directly, at the operating system level, or in one of the following ways:
SAP HANA Studio -> Administration -> Diagnosis Files
DBACOCKPIT -> Diagnostics -> Diagnosis Files
More checking information
The SQL statements that are in the following SAP notes indicate whether a database reorganization is required and the amount of space that is saved after the reorg action takes place.
This check serves two purposes.
- Highlights whether the reorg action is required on the SAP HANA database.
- If a reorg action is required, it provides an estimated size after space saving actions are completed.
Reorganizing database row store
If your database is heavily fragmented, a row store reorganization is required.
Starting from SAP HANA 2.0 SPS04, online row store reorganization automatically triggers for large row store (allocated size >= 3.2 GB) if the utilization ratio is less than the defined threshold.
By default, the threshold is 60% and the utilization ratio is checked in the background once an hour.
If your database is version is less than SAP HANA 2.0 SPS04
, then follow the instructions that are in
-
SAP Note 1977584 - Technical Consistency Checks for SAP HANA Databases This SAP Note contains useful SQL statements to check the CATALOG, DEPENDENCY, and TABLE CONSISTENCY.
Make sure that you pay attention to the instructions that are in these SAP Notes and follow each step that relates to your existing SAP HANA version.
SQL mini checks
Use this SQL statement to show the current size of the SAP HANA database.
SELECT HOST, PORT, TO_DECIMAL( SUM(FREE_SIZE)*100 / SUM(ALLOCATED_SIZE), 10,2) "Free Space Ratio in %",TO_DECIMAL(
SUM(ALLOCATED_SIZE)/1048576, 10, 2) "Allocated Size in MB",TO_DECIMAL( SUM(FREE_SIZE)/1048576, 10, 2) "Free Size in MB" FROM
M_RS_MEMORY WHERE ( CATEGORY = 'TABLE' ) and ( ALLOCATED_SIZE > 0 ) GROUP BY HOST, PORT
For more useful SQL statements, you can use the following SAP Note. This SAP Note includes some useful SQL statements that you can run from the command line by using the hdbsql
executable file. Or, you can use the SQL Console
that is built into SAP HANA Studio.
Commands from SQLStatements_Internal.zip impose an increased risk of instabilities such as crashes or terminations. If you run these commands, run them with care. Perhaps running them first on a DEV or POC system first.
The following SQL statement helps you identify critical technical issues. When you download the SQL Collection compressed files, search for SQL: "HANA_Configuration_MiniChecks"
.
SQL: "HANA_Configuration_MiniChecks"
performs several mini checks and returns C = 'X'
if it finds a potentially critical situation. You can use the following SAP Note to interpret the results.
Scheduling an SAP HANA sizing report on the source system
If you plan to migrate an existing SAP system from an on-premise site to your IBM Cloud® environment, you need fristto run an SAP Sizing report. The current version for the SAP HANA memory sizing report is Advanced correction version 17
.
If you want to run the SAP HANA Sizing report, see the following SAP Note.
It is advised that you use the most recent Advanced Correction of the SAP Sizing report. When you run the report, make sure that you include the forecast for SAP HANA database growth. The generated report states the anticipated required CPU, memory, and storage recommendations for your PowerVS instance target. Go to IBM Cloud® and select the most recent certified profile that is available for IBM Power Virtual Servers.
- SAP 2947579 - SAP HANA on IBM Power Virtual Servers
- SAP 2188482 - SAP HANA on IBM Power Systems: Supported hardware and features
Extra sizing SAP Notes
Using the EarlyWatch Alert reports as an early indicator
If your on-premises landscape has an SAP Solution Manager set up, you can generate the EarlyWatch Alert report for your source system.
The report outlines specific issues that your on-premises source system might have. You must address SQL performance indicators, Urgent performance KPI indicators immediately. Issues that are classified as Red
or Severe problems detected
must be handled as soon as possible.
Check the EarlyWatch Alert report for existing issues with the source SAP HANA database and act upon each finding in the Service Summary
or Alert Overview
sections, based on its severity.
More related SAP Notes for EarlyWatch Alert reports
Source database credentials
When you add an SAP HANA system to the SAP HANA System Replication setup, remember that the replication process from source primary to the target secondary server overwrites the MDC User Tables SAP${sid}.USR02
. So, it is important
to know (by checking the SAP HANA Studio) what the current user with SYS privileges that was used to register the MDC in SAP HANA Studio on the source. As a Best Practice
, make sure that you know the login credentials for the
database user and the password for the source system. If, for example, you forget the passwords and proceed with the SAP HANA System Replication from the source to the target you can test the secondary target by swapping the primary and
secondary servers. If you do not know the login credentials for the database user and the password on the source system then you can't register the system in either SAP HANA Studio or an SAP HANA Cockpit setup.
Creating the Target SAP HANA System on IBM Power Virtual Server
Planning the IBM Power Virtual Server deployment
A Power Server workspace in your IBM Cloud account is a prerequisite for the following steps. Read details in Hybrid Cloud Network Considerations for SAP on IBM Power Virtual Server.
A hybrid cloud network connection needs to be in place, as described in Hybrid Cloud Network Considerations for SAP on IBM Power Virtual Server.
The Planning for a deployable SAP HANA infrastructure is described in Planning your deployment.
The sizing aspect of the target system is vital to your planning. Follow the recommendations mentioned in the SAP HANA Sizing report on the source system. Also, consider the findings of the EarlyWatch Alert report (EWA report). Both factors provide a realistic approach on the recommended size of your target system, see Sizing process for SAP Systems.
Comparing the required CPU, cores and storage for your target system
Check that the certified profiles in IBM Cloud® are close to or match the recommendations that are mentioned in the source system sizing report and also consider the EWA report summary.
Select the correct IBM Power Systems Virtual Server Certified Profile from the following two links:
Target server must have equal or greater storage capacity than the source system and be sized correctly
Remember to take SAP HANA database growth into consideration and the need to follow the IBM System Storage Architecture and Configuration Guidelines for SAP HANA TDI.
The following document outlines the required storage configuration for the target server in IBM Cloud:
Consider the extra space that you need to create a file system mount point to store software executable files and the initial SAP HANA system backup. Depending on your planned IBM Cloud PowerVS infrastructure, you can create the file system as an NFS mount to export to other systems in the architecture.
Creating the software repository file system and transfer the installation packages
As used in previous demo systems, the mount point swrepo
is created with at least 200 GB of free space. Download the SAP HANA Software from SAP Marketplace - the version that matches your SAP HANA version from the source system.
-
- Access Software Downloads in SAP for Me
- Enter your SAP "S" user ID and password to proceed.
-
- Support Packages & Upgrades
- By Alphabetical Index (A-Z)
- "H"
- SAP HANA PLATFORM EDITION
- SAP HANA PLATFORM EDITION 2.0
- SAP HANA DATABASE 2.0
- Make sure that the Selection box shows
LINUX ON POWER LE 64BIT
- Select the IMDB_SERVER20 that is installed on the Source System, and download to your laptop or PC
- Go back to the SAP HANA PLATFORM EDITION 2.0 page
- SAP HANA CLIENT 2.0
- Make sure that the Selection box shows
LINUX ON POWER LE 64BIT
- Select the release that you installed on the source system (or one version higher if your version is not on the list)
- Navigate back to * Software Center Catalog view
- Support Packages & Upgrades
- On the right side, is a search box, search for
SAPCAR
- On the Displayed results list, select
SAPCAR 7.53
Maintenance Software Component - Select the file
SAPCAR_1200-70007726.EXE
and make sure that the Selection box showsLINUX ON POWER LE 64BIT
- Download to your laptop or PC or Jump Host
Create a directory /swrepo
on the target system.
sudo mkdir /swrepo
Make sure that your user owns this directory, so the user can work and extract files.
sudo chown $USER: /swrepo
Transfer the installation files and sapcar utility downloads to the target SAP HANA server /swrepo
mount point. The SAPCAR utility needs executable permissions to unpack the .SAR archive files.
chmod -R 755 /swrepo/SAPCAR_1200-70007726.EXE
You can set an alias SAPCAR for this utility in the .bash_profile
. This setting enables the SAPCAR command from any directory.
To add a line to your bash profile, use the following command.
echo "alias SAPCAR='/swrepo/SAPCAR_1200-70007726.EXE'" >>$HOME/.bash_profile
Use the source command to enable the new defined alias.
source $HOME/.bash_profile
Check whether it works by running SAPCAR -v
to get the version list:
SAPCAR -v
Unpacking the files
Use the following examples to unpack the files.
SAPCAR -xvf IMDB_CLIENT20_XXX_XX-XXXXXXXX.SAR -manifest /SAP_HANA_CLIENT/SIGNATURE.SMF
The sapcar file extraction output looks like the following example.
x SAP_HANA_CLIENT/SIGNATURE.SMF
SAPCAR: 98 file(s) extracted
SAPCAR -xvf IMDB_SERVER20_XXX_XX-XXXXXXXX.SAR -manifest /SAP_HANA_DATABASE/SIGNATURE.SMF
x /SAP_HANA_DATABASE/SIGNATURE.SMF
SAPCAR: 355 file(s) extracted
During extraction directories /swrepo/SAP_HANA_DATABASE
and /swrepo/SAP_HANA_CLIENT
are created and contain the files that are required for the installation.
Making sure that the target server OS and patch level match the source server
Check the operating system version and patch level on the target system. For productive systems, the same level makes sure that the installation performs similar and migrating runs with ease. For nonproductive systems, for example, a proof of concept system (POC) in IBM PowerVS, a higher operating system version is a valid option.
Target server - Both RHEL and SLES
To determine the operating system version and patch level, run the following command.
cat /etc/os-release
Alternatively on Red Hat Linux systems you can use a second file.
cat /etc/redhat-release
On SUSE Linux Enterprise Server (for SAP Applications) the release and patch level can be listed with the following command.
lsb_release -a
Making sure that the file system and mount points match the source system
The source and target systems must have the identical mapping for storage, LVM, and file systems. Only on target is the larger storage capacity that is needed or the Migration. File system structure requirements are also highlighted in the
beginning of this section with the TDI requirements. Also, consider that the mount point and file ownership UID and GUID match the source system. Also, mount points need the same <SID>
defined on both systems. When you
install SAP HANA on the target system, the same <SID>
and <instance number>
from the source system are used.
export SID=<SID> # SAP HANA System ID (uppercase)
export sid=<sid> # SAP HANA System ID (lowercase)
export INSTNO=<INSTNO> # SAP HANA Instance Number
export SiteOnPrem=<PrimarySiteName> # HANA System Replication Site Name 1 - Migration from On-Prem - Source
export SiteOnCloud=<secondarySiteName> # HANA System Replication Site Name 2 - Migration to On-Cloud - Target
export NODE1=<Hostname 1> # Hostname of On-Prem Server
export NODE2=<Hostname 2> # Hostname of IBM Cloud PowerVS Instance
Entries in /etc/hosts
for all systems involved in the migration project
The /etc/hosts
file needs to contain entries for the Source System and any dependent SAP Netweaver or S/4 FES Application Server. You can use a DNS server for your network resource resolution, but it helps if you include the
IP addresses, short name, FQDN, and description to help identify servers in the landscape in the /etc/hosts
file, especially if issues occur with network resolution or the DNS services.
Preparing and tuning the OS for SAP HANA
Use the following SAP Notes to begin the preparation phase of the target system for the installation of SAP HANA.
- SAP Note 2777782 - SAP HANA DB: Recommended OS Settings for RHEL 8
- SAP Note 2772999 - Red Hat Enterprise Linux 8.x: Installation and Configuration
- SAP Note 3018133 - Linux: Running SAP applications compiled with GCC 10.x
Make sure that you completed the tasks that are mentioned in the Recommended OS Settings for RHEL 8
as these tasks are important tuning and performance settings that need to be applied. If ignored, it can impact the installation
of SAP products and the performance thereafter.
Pre-SAP HANA checks by using the hcmt tool
The SAP HANA hardware and cloud measurement tools hcmt
help measure and analyze your hardware or cloud systems before you deploy SAP HANA or apply for SAP HANA certification. The tools consist of the following components:
- SAP HANA hardware and cloud measurement tool
- SAP HANA hardware and cloud measurement analysis
Use the following SAP Note to check and verify the OS and configuration before you install SAP HANA.
If you have a port issue when you run hcmt
, open a second Terminal session. Navigate to the setup directory of hcmt
, now start a session that keeps the required port open.
To run hcmt
in server-client mode, you need to start two sessions:
hcmt
server mode on - a jump host to collect test result from remote servershcmt
client on the target systems that are intended to run SAP HANA, runhcmt
performance test by using the full execution plan.
Hcmt server session
The hcmt server collects data that is measured on hcmt client systems. A typical system to run the hcmt server is a jump host or similar system. Navigate to the directory where hcmt is installed, and run the following command
sudo ./hcmt -v -S
The following example is the expected output.
hcmt-2.00.062.00.1650891137 (2022-04-25 15:12:20)
Server started, listening on port 50000 ...
Hcmt client session
On the target system that you want to be the SAP HANA server, run the hcmt
command as a client by using the full execution plan.
sudo ./hcmt -v -p /swrepo/HCMT/setup/config/full_executionplan.json
System output:
hcmt-2.00.062.00.1650891137 (2022-04-25 15:12:20)
Loading executionplan
LogVolume (/hana/log):
DataVolume (/hana/data):
Hosts: <`Leave Blank!!!!`> Leave this field blank, otherwise it will affect the test.
Start execution of plan
Executing Test C9C9F832-854F-492D-8E7EFB4609AC435C
Note: CPU Micro Benchmark
If you receive an error 'Port 50000 is already used', SAP HANA is probably installed already. Stop the SAP HANA system and then run the hcmt
command again.
Plan Variant: CPU Performance
This command generates a hcmresult-YYYMMDDHHMMSS.zip file in the setup directory. Upload this file to the HCMT SAP website and review the results to make sure that the HANA is set up and configured correctly.
If you experience issues, you can still use the old check tool.
Installing SAP HANA on the target system
Remember the following variables:
export SID=<SID> # SAP HANA System ID (uppercase)
export sid=<sid> # SAP HANA System ID (lowercase)
export INSTNO=<INSTNO> # SAP HANA Instance Number
For this example, the installation is up to the point where you need to enter "Y" to continue. Navigate back to the HANA_DATABASE directory.
Run the SAP HANA database lifecycle manager command.
sudo ./hdblcm
The following example is the expected output.
SAP HANA Lifecycle Management - SAP HANA Database 2.00.061.00.1644229038
************************************************************************
This will scan the directories for the required software.
Scanning software locations...
Detected components:
SAP HANA Database (2.00.061.00.1644229038) in /swrepo/HANA/SAP_HANA_DATABASE/server
SAP HANA Database Client (2.11.20.1644165757) in /swrepo/HANA/SAP_HANA_CLIENT/client
Do you want to specify additional components location? (y/n) [n]: `n`
Choose n
for no additional components location and continue.
Choose an action
Index | Action | Description
-------------------------------------------------------------
1 | install | Install new system
2 | extract_components | Extract components
3 | print_detected_components | Print detected components
4 | Exit (do nothing) |
Enter selected action index [4]: `1`
Enter 1
and press <enter>
key to install a new system.
Output continues with the following example.
SAP HANA Database version '2.00.061.00.1644229038' will be installed.
Select additional components for installation:
Index | Components | Description
--------------------------------------------------------------------------------
1 | all | All components
2 | server | No additional components
3 | client | Install SAP HANA Database Client version 2.11.20.1644165757
Enter comma-separated list of the selected indices [3]: `1`
Enter 1
and press <enter>
to install all SAP HANA components.
Output continues with the following example.
SAP HANA Database version '2.00.061.00.1644229038' will be installed.
Select additional components for installation:
Index | Components | Description
--------------------------------------------------------------------------------
1 | all | All components
2 | server | No additional components
3 | client | Install SAP HANA Database Client version 2.11.20.1644165757
Enter comma-separated list of the selected indices [3]: `1`
Enter 1
and press <enter>
to install all components. Accept a series of defaults on the next line in the output.
Enter Installation Path [`/hana/shared`]:
Enter Local Host Name [`Yourhostname`]:
Do you want to add hosts to the system? (y/n) [`n`]:
Enter `n' for no additional systems. Check the source SAP HANA database system parameters:
- source SAP HANA
SID
- source SAP HANA
Instance Number
Continue with the same values for the target system:
Enter SAP HANA System ID: `<Needs to match the source system>`
Enter Instance Number [00]: `<Needs to match the source system>`
Enter Local Host Worker Group [default]:
Index | System Usage | Description
-------------------------------------------------------------------------------
1 | production | System is used in a production environment
2 | test | System is used for testing, not production
3 | development | System is used for development, not production
4 | custom | System usage is neither production, test nor development
Select System Usage / Enter Index [4]: 2
Enter a number that represents the planned function. In the example, 2
indicates a system for testing.
Accept more default values:
Do you want to enable data and log volume encryption? [n]:
Enter Location of Data Volumes [/hana/data/<SID>]:
Enter Location of Log Volumes [/hana/log/<SID>]:
Restrict maximum memory allocation? [n]:
Apply System Size Dependent Resource Limits? (SAP Note 3014176) [y]:
Determine these passwords as set on the source system:
sapadm
password<sid>adm
password- System Database User
SYSTEM
password
Set the same passwords on the target system:
Enter SAP Host Agent User (sapadm) Password: <Use the same password used on the source system>
Confirm SAP Host Agent User (sapadm) Password: <Use the same password used on the source system>
Enter System Administrator (<sid>adm) Password: <Use the same password used on the source system>
Confirm System Administrator (<sid>adm) Password: <Use the same password used on the source system>
Enter System Administrator Home Directory [/usr/sap/<SID>/home]:
Enter System Administrator Login Shell [/bin/sh]:
Enter System Administrator User ID [1001]: <check that the user ID number matches the source system>
Enter ID of User Group (sapsys) [79]: <Check that the GUID number matches the source system>
Enter System Database User (SYSTEM) Password: <Use the same password used on the source system>
Confirm System Database User (SYSTEM) Password: <Use the same password used on the source system>
Restart system after machine reboot? [n]:
Summary before execution
At the summary, you can check to make sure that the selections that you made for the installation are correct. Then, select "Y" to begin. After about 20 minutes, you see the following message.
Registering SAP HANA Database Components on Local Host...
- Deploying SAP Host Agent configurations...
Creating Component List...
SAP HANA Database System installed
Log file written to xxxxxx
Checking that SAP HANA is running and determining the version
Run the following HDB proc
command to verify that all services started on the primary and secondary SAP HANA system.
sudo -i -u ${sid}adm -- HDB proc
SAP HANA version needs to be equal or greater than the primary server
To verify the SAP HANA database version, use the following command on both nodes.
sudo -i -u ${sid}adm -- HDB version
Initial backup of the MDC/SYSTEMDB SAP HANA database
Backup SYSTEMDB
Add both the SYSTEMDB entry and the MDC to the HANA Studio Application. Or, If you have an SAP HANA Cockpit in your landscape, you can add the target system to your HANA Cockpit instead. After both systems are added, complete an initial system backup. On the SYSTEMDB entry, -> right click and select.
- Backup & Recovery
- Backup Up System Database
- Backup Type
Complete Data Backup
- Destination
File
- Backup Destination
/swrepo/backup/data/SYSTEMDB
make sure that this directory structure exists and is writable with user${sid}adm
. - Backup Prefix
COMPLETE_DATA_BACKUP_INITIAL_DDMMYYY
Next Review Backup Setup
and then selectFinish
Make sure that the SYSTEMDB@${SID}
backup completes successfully.
Backup MDC
Backup & Recovery
- Backup Up Tenant Database
- Specify the Tenant Database
${sid}
. Next - Backup Type
Complete Data Backup
. - Destination
File
. - Backup Destination
/swrepo/backup/data/DB_${sid}
make sure that this directory structure exists and is writable with user${sid}adm
. - Backup Prefix
COMPLETE_DATA_BACKUP_INITIAL_DDMMYYY
. Next Review Backup Setup
and then selectFinish
.
Make sure that the backup of DB_${sid}
completes without errors.
Check backup status
On the SYSTEMDB entry, -> right click and select the following actions.
- Backup & Recovery
- Select Open
Backup Console
- Select the tab
Backup Catalog
- In the Database Field select
${sid}
for the MDC - In the Database Field select
<SYSTEMDB>
for the SYSTEMDB
Optional check of the trace log files
The database trace is written to service specific files on operating system level. The trace directory is in the following location:
/usr/sap/${sid}/HDB<inst>/<host>/trace/DB_${sid}/
The following alias in the environment of the ${sid}adm
user allows to you quickly switch to the trace directory on the OS level:
cdtrace
The database trace files use the following naming convention:
<service>_<host>.<port>.<counter>.trc
In the context of dynamic tiering, a file with the following convention can exist (SAP Note 2871785):
esserver_console_<host>.<port>.<counter>.trc
- Example:
indexserver_saphana01.30003.024.trc
You can access these files either directly on the operating system level or in one of the following ways:
SAP HANA Studio -> Administration -> Diagnosis Files
DBACOCKPIT -> Diagnostics -> Diagnosis Files
Migrating SAP S/4HANA by using SAP HANA System Replication
Pre-checks before you configure SAP HANA System Replication
Before you configure SAP HANA System Replication, a few prerequisites must be checked. The described Steps are valid for Red Hat Enterprise Linux 8 (RHEL) and SUSE Enterprise Linux (SLES).
Check the SAP HANA database user on the source system
Check with your SAP basis administration team or SAP HANA administrators, which SAP HANA database user is used to access the system. Typically this user the SYSTEM
user, or the SAP schema owner user if your SAP basis administration
team implemented the SAP security advisories.
SAP HANA landscape pre-steps for activating SAP HANA System Replication
Set the environment variables on the primary and secondary SAP HANA systems
To simplify the setup, prepare the following environment variables for ${sid}adm
on both nodes. These environment variables are used in subsequent commands in the remainder of the examples.
On both nodes, run the following commands. Remember that the variables must be the same on both systems, source and target.
export SID=<SID> # SAP HANA System ID (uppercase)
export sid=<sid> # SAP HANA System ID (lowercase)
export INSTNO=<INSTNO> # SAP HANA Instance Number
export DIR_INSTANCE=/usr/sap/${SID}/HDB${INSTNO} # "${sid}adm" home directory
export SiteOnPrem=<PrimarySiteName> # HANA System Replication Site Name 1 - Migration from On-Prem - Source
export SiteOnCloud=<secondarySiteName> # HANA System Replication Site Name 2 - Migration to On-Cloud - Target
export NODE1=<Hostname 1> # Hostname of On-Prem Server
export NODE2=<Hostname 2> # Hostname of IBM Cloud PowerVS Instance
Make sure that SAP HANA is running on both systems
As the operating system user ${sid}adm
, the command HDB proc
can be used to verify that all services are started.
Run the following command on both systems, primary and secondary SAP HANA server.
sudo -i -u ${sid}adm -- HDB proc
SAP HANA version must be equal or greater than the primary server
Run the following command on each node to determine the SAP HANA server version.
sudo -i -u ${sid}adm -- HDB version
The target system version must be equal or larger compared with the source system version. The only exception for the version is for an Active/Active
read enabled configuration, here the HDB version must be identical on source
and target system.
-
Therefore, make sure that the system configuration is identical on both the source and target servers. Then, compare the settings in the ini-files on both systems.
-
For a scale-out configuration, make sure that the number of worker nodes (scale-out) and roles are identical on both the source and target servers.
-
The same
${sid}
andinstance numbers
must be used on both systems. -
Back up
PKI SSFS .key and the .dat files
from the primary and secondary systems. -
Copy existing PKI keys from the primary to the secondary system.
To make sure that you can recover to the original installed state, if needed, back up the existing keys on both primary and secondary systems.
sudo -i -u ${sid}adm -- cp -p /usr/sap/${SID}/SYS/global/security/rsecssfs/data/SSFS_${SID}.DAT /usr/sap/${SID}/SYS/global/security/rsecssfs/data/SSFS_${SID}.DAT_<hostname>
After the backup of the existing PKI SSFS .key
and .dat
files is done, you now need to copy the PKI SSFS .key
and .dat
files from primary system to the target system.
The SAP HANA 2.0 data and log transmission channels for the replication process require authentication by using the system PKI SSFS storage certificate files.
The system PKI SSFS storage certificate files are stored in /usr/sap/${SID}/SYS/global/security/rsecssfs/
in subdirectories data
and key
.
On NODE2, run the following commands to copy files SSFS_${SID}.DAT
and SSFS_${SID}.KEY
from NODE1.
As ${sid}adm
user, run the following two commands on NODE2.
scp ${NODE1}:/usr/sap/${SID}/SYS/global/security/rsecssfs/data/SSFS_${SID}.DAT /usr/sap/${SID}/SYS/global/security/rsecssfs/data/SSFS_${SID}.DAT
scp ${NODE1}:/usr/sap/${SID}/SYS/global/security/rsecssfs/key/SSFS_${SID}.KEY /usr/sap/${SID}/SYS/global/security/rsecssfs/key/SSFS_${SID}.KEY
The copied PKI SSFS storage certificates on NODE2 become active during the start of the SAP HANA system.
Check that the configuration parameter log_mode
is set to normal
Make sure that the configuration parameter log_mode
is set to normal in the persistence section of the global.ini
on both the primary and secondary SAP HANA Servers.
Run the following command on both systems to verify the log_mode
setting.
sudo -i -u ${sid}adm -- grep -i 'log_mode' /usr/sap/${SID}/HDB${INSTNO}/exe/config/global.ini
The following output is expected.
log_mode=normal
Register the primary server first
On the primary SAP HANA system, run the following command to register this node as the primary
for SAP HANA System Replication.
sudo -i -u ${sid}adm -- hdbnsutil -sr_enable --name=${SiteOnPrem}
The following output is expected.
nameserver is active, proceeding ...
successfully enabled system as system replication source site
done.
Check whether the primary system is registered
Verify that the primary system is successfully registered by using the following command.
sudo -i -u ${sid}adm -- hdbnsutil -sr_state
The following output is expected.
System Replication State
~~~~~~~~~~~~~~~~~~~~~~~~
online: true
mode: primary
operation mode: primary
site id: 1
site name: SiteOnPrem
is source system: true
is secondary/consumer system: false
has secondaries/consumers attached: false
is a takeover active: false
is primary suspended: false
Host Mappings:
~~~~~~~~~~~~~~
Site Mappings:
~~~~~~~~~~~~~~
SiteCloud (primary/)
Tier of SiteCloud: 1
Replication mode of SiteCloud: primary
Operation mode of SiteOnPrem :
Hint based routing site:
done.
Make sure that SAP HANA is not active on the secondary site
The secondary site must not be an active SAP HANA server. Stop SAP HANA database services by using the following command.
sudo -i -u ${sid}adm -- HDB stop
The following output is expected.
hdbdaemon will wait maximal 300 seconds for NewDB services finishing.
Stopping instance using: /usr/sap/${SID}/SYS/exe/hdb/sapcontrol -prot NI_HTTP -nr 10 -function Stop 400
10.08.2023 10:32:07
Stop
OK
Waiting for stopped instance using: /usr/sap/${SID}/SYS/exe/hdb/sapcontrol -prot NI_HTTP -nr 10 -function WaitforStopped 600 2
10.08.2023 10:32:51
WaitforStopped
OK
hdbdaemon is stopped.
Register the secondary system
Now register the secondary system.
sudo -i -u ${sid}adm -- hdbnsutil -sr_register \
--name=<secondarySiteName> \
--remoteHost=<primary_host> \
--remoteInstance=<primary_systemnr> \
--replicationMode=[sync|syncmem|async] \
--operationMode=[delta_datashipping|logreplay|logreplay_readaccess]
For example, if you use
SiteOnCloud
as the secondary site namesyncmem
as replication mode andlogreplay
as operation mode
The last command looks like the following example.
sudo -i -u ${sid}adm -- hdbnsutil -sr_register \
--name=${SiteOnCloud} \
--remoteHost=${NODE1} \
--remoteInstance=${INSTNO} \
--replicationMode=syncmem \
--operationMode=logreplay
The following output is expected.
Thu 10 Aug 10:36:13 CEST 2023
adding site ...
collecting information ...
updating local ini files ...
done.
Troubleshoot hdbnsutil errors with SELinux enabled
If security-enhanced Linux (SELinux) is enabled, the output of hdbnsutil
is not as expected. You can see one of the following two symptoms.
Command is not recognized
error message- Usage information displayed
SELinux, when set to enforcing
, prevents the command hdbnsutil
from restarting the saphostagent in the ${sid}adm
user context. You can either add proper SELinux security policies or as SAP recommends.
Then, disable SELinux.
Check the current SELinux status with the following command.
sestatus
The following output is an example.
SELinux status: enforcing
If sestatus
command returns with enforcing
, then commands even when run with root privileges can be blocked, depending on security policy.
To disable SELinux temporarily, run the following command.
sudo setenforce 0
SELinux is now temporarily disabled until the next restart.
Now check with sestatus
again, the status shows disabled
.
Check whether the saphostagent process is running with the following command.
sudo ps -ef | grep -i host
If output is empty and no process is displayed, manually restart the saphostagent.
sudo -i -u ${sid}adm -- /usr/sap/hostctrl/exe/saphostexec -restart /usr/sap/hostctrl/exe/host_profile
Check the state on both sides of the SAP HANA System Replication
Check the primary system state
Verify the system replication state on the primary node. Run the following command on the primary server:
sudo -i -u ${sid}adm -- hdbnsutil -sr_state
The following output is expected.
System Replication State
~~~~~~~~~~~~~~~~~~~~~~~~
online: true
mode: primary
operation mode: primary
site id: 1
site name: SiteOnPrem_hostname
is source system: true
is secondary/consumer system: false
has secondaries/consumers attached: true
is a takeover active: false
is primary suspended: false
Host Mappings:
~~~~~~~~~~~~~~
<SiteOnCloud_hostname> -> [SiteOnPrem_hostname] <SiteOnPrem_hostname_hostname>
<SiteOnCloud_hostname> -> [SiteOnCloud] <SiteOnPrem_hostname_hostname>
Site Mappings:
~~~~~~~~~~~~~~
SiteOnPrem_hostname (primary/primary)
|---SiteOnCloud (syncmem/logreplay)
Tier of SiteCloud: 1
Tier of SiteOnPrem_hostname: 2
Replication mode of SiteOnPrem_hostname: primary
Replication mode of SiteOnCloud: syncmem
Operation mode of SiteOnPrem_hostname: primary
Operation mode of SiteOnCloud logreplay
Mapping: SiteOnPrem_hostname -> SiteOnCloud
Hint based routing site:
done.
Check the secondary system state
Now check the system replication state on the second node. Run the same command on the second server.
sudo -i -u ${sid}adm -- hdbnsutil -sr_state
The following output is expected.
System Replication State
~~~~~~~~~~~~~~~~~~~~~~~~
online: false
mode: syncmem
operation mode: unknown
site id: 2
site name: SiteOnCloud
is source system: unknown
is secondary/consumer system: true
has secondaries/consumers attached: unknown
is a takeover active: false
is primary suspended: false
is timetravel enabled: false
replay mode: auto
active primary site: 1
primary masters: <SiteOnPrem_hostname_hostname>
done.
Restart the secondary server
So far both SAP HANA servers are configured as replication partners. Now restart the secondary SAP HANA server to complete the replication setup.
Run the following command on the secondary server.
sudo -i -u ${sid}adm -- HDB start
The following output is expected.
StartService
OK
Starting instance using: /usr/sap/${SID}/SYS/exe/hdb/sapcontrol -prot NI_HTTP -nr 10 -function StartWait 2700 2
OK
10.08.2023 10:38:47
Start
OK
10.08.2023 10:40:17
StartWait
OK
Check HDB info
or HDB proc
on the secondary side to confirm that SAP HANA is running again. When successful, run the sr_state
command on the primary system.
sudo -i -u ${sid}adm -- hdbnsutil -sr_state
The following output is expected.
System Replication State
~~~~~~~~~~~~~~~~~~~~~~~~
online: true
mode: primary
operation mode: primary
site id: 1
site name: SitePrem
is source system: true
is secondary/consumer system: false
has secondaries/consumers attached: true
is a takeover active: false
is primary suspended: false
Host Mappings:
~~~~~~~~~~~~~~
<SiteOnCloud_hostname> -> [SiteOnPrem] <SiteOnPrem>
<SiteOnCloud_hostname> -> [SiteOnCloud] <SiteOnPrem>
Site Mappings:
~~~~~~~~~~~~~~
SiteOnPrem (primary/primary)
|---SiteOnCloud (syncmem/logreplay)
Tier of SiteOnprem : 1
Tier of SiteOnCloud: 2
Replication mode of SiteOnPrem: primary
Replication mode of SiteOnCloud: syncmem
Operation mode of SiteOnPrem: primary
Operation mode of SiteOnCloud: logreplay
Mapping: SiteOnprem_hostname -> SiteOnCloud
Hint based routing site:
done.
Check replication status
After the secondary system is configured and SAP HANA is started on the secondary server, the replication process automatically starts synchronizing data with a full replica
. You can verify the initial replication on the primary
server and watch the current completion status of the full replication action.
Run the Python script with the following command.
sudo -i -u ${sid}adm -- python ${DIR_INSTANCE}/exe/python_support/systemReplicationStatus.py
The following output is expected.
|Database |Host |Port |Service Name |Volume ID |Site ID |Site Name |Secondary |Secondary |Secondary |Secondary |Secondary |Replication |Replication |Replication |Secondary |
| | | | | | | |Host |Port |Site ID |Site Name |Active Status |Mode |Status |Status Details |Fully Synced |
|-------- |-------- |----- |------------ |--------- |------- |--------- |----------|--------- |--------- |---------- |------------- |----------- |----------- |-------------- |-------------|
|SYSTEMDB |<NODE1> |31001 |nameserver | 1 | 1 |SiteOnPrem| <NODE2> | 31001 | 2 |SiteOnCloud|YES |SYNCMEM |ACTIVE | | True|
|S4H |<NODE1> |31007 |xsengine | 2 | 1 |SiteOnPrem| <NODE2> | 31007 | 2 |SiteOnCloud|YES |SYNCMEM |ACTIVE | | True|
|S4H |<NODE1> |31040 |docstore | 5 | 1 |SiteOnPrem| <NODE2> | 31040 | 2 |SiteOnCloud|YES |SYNCMEM |ACTIVE | | True|
|S4H |<NODE1> |31003 |indexserver | 3 | 1 |SiteOnPrem| <NODE2> | 31003 | 2 |SiteOnCloud|YES |SYNCMEM |ACTIVE | | True|
|S4H |<NODE1> |31011 |dpserver | 4 | 1 |SiteOnPrem| <NODE2> | 31011 | 2 |SiteOnCloud|YES |SYNCMEM |ACTIVE | | True|
status system replication site "1": ACTIVE
overall system replication status: ACTIVE
Local System Replication State
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
mode: PRIMARY
site id: 1
site name: SiteOnPrem_hostname
Four methods to check the system replication status
Option 1. landscapeHostConfiguration.py
The first option uses the Python script landscapeHostConfiguration.py
for a server point of view. This script displays a status line per SAP HANA server system.
Run the following command.
sudo -i -u ${sid}adm -- python ${DIR_INSTANCE}/exe/python_support/landscapeHostConfiguration.py
Make sure that each server that is listed in the output shows OK
in the host status column.
Option 2. systemReplicationStatus.py
The second alternative option uses the Python script systemReplicationStatus.py
for a database view of the SAP HANA system replication. This script displays one status line for each database and an overall status after the
database table.
Run the Python script with the following command.
sudo -i -u ${sid}adm -- python ${DIR_INSTANCE}/exe/python_support/systemReplicationStatus.py
Make sure that each database listed shows an ACTIVE
in replication status column. The expected script output contains the following line:
overall system replication status: ACTIVE
Option 3. hdbcons
Check the detailed status of the system replication with the hdbcons
command and run as ${sid}adm
user. This third option is a technical per server and per service view.
Run the SAP HANA DB Management Client Console hdbcons
with the following command.
sudo -i -u ${sid}adm -- hdbcons -e hdbindexserver "replication info"
Option 4. Sql script
The fourth alternative uses an SQL statement that can be run, for example, in SAP HANA studio or cockpit. This option is a hosts-per-site-view of SAP HANA system replication.
Check by running the following SQL statement.
select host, SECONDARY_HOST, PORT, SITE_NAME, SECONDARY_SITE_NAME, REPLICATION_MODE, REPLICATION_STATUS, REPLICATION_STATUS_DETAILS,SECONDARY_ACTIVE_STATUS from M_SERVICE_REPLICATION;
Check especially columns REPLICATION_STATUS
and REPLICATION_STATUS_DETAILS
in the SQL output.
Post replication completion
Before you disable the replication setup, check the trace logs for any inconsistencies or anomalies after the replication action is performed. After the replication completes, the database contains all active services on the primary system only. But you can still examine the trace logs for any inconsistencies or issues.
Checking the database trace files
The database trace is written to service specific files on operating system level. The trace directory is located here:
/usr/sap/<SID>/HDB<inst>/<host>/trace/DB_<SID>/
The following alias in the ${sid}adm
user environment allows the ${sid}adm
user to quickly change to the trace directory on the operating system level:
cdtrace
The database trace files have the following naming convention:
<service>_<host>.<port>.<counter>.trc
In the context of dynamic tiering also a file with the following convention can exist (SAP Note 2871785):
esserver_console_<host>.<port>.<counter>.trc
Example:
indexserver_saphana01.30003.024.trc
You can access database trace files in three ways:
- Directly on the operating system level
SAP HANA Studio -> Administration -> Diagnosis Files
DBACOCKPIT -> Diagnostics -> Diagnosis Files
SAP HANA System Replication resources
For more information, see the following links:
- SAP Note 1999880 - HANA System Replication FAQ
- SAP Note 11969700 - SQL Statement Collection for SAP HANA
- SAP Note 3357978 - Configuring SAP HANA Multi Target System Replication
- SAP HANA Replication Setups
- SAP HANA System Replication
- SAP HANA multitarget System Replication
- SAP HANA System Replication
- SAP Help Portal Documentation HSR HANA 2.0 SP07