IBM Cloud Docs
Overview of IBM Spectrum LSF

Overview of IBM Spectrum LSF

IBM® Spectrum LSF is a scheduling software to enable High-Performance Computing (HPC) clusters. This offering uses deployable architecture to provision and configure IBM Cloud resources. With simple steps to define configuration properties and use automated deployment, you can build your own HPC clusters in minutes by using your choice of an Intel x86 based VPC virtual server instance profile type for the worker nodes in the cluster. Spectrum LSF also enables configuration for auto scaling, so Spectrum LSF clusters can automatically add and remove worker nodes based on workload specifications. This allows to take full advantage of consumption-based pricing and pay for cloud resources only when they are needed. 

A deployable architecture involves components, modules, and dependencies in a way that allows for seamless deployment and makes it easy for developers and operations teams to quickly deploy new features and updates to the system, without requiring extensive manual intervention. Refer the Deployable architecture document for more detailed information.

IBM Spectrum LSF offers the option of a public virtual system, or virtual systems that are deployed on dedicated hosts, for static compute nodes only. The management nodes and dynamic compute nodes use public virtual machines only. The dedicated host option allows you to have systems that are assigned just for your workloads and avoids issues like a noisy neighbor. You can pack a dedicated host to full capacity before spilling to another instance or spread the virtual server instances evenly across all dedicated hosts. Go to Dedicated Hosts for Virtual Server Instances for more information.

In addition, Spectrum LSF provides two shared storage options to manage your application data:

  • File storage for VPC or
  • Storage Scale

The Storage Scale feature is designed to work with Spectrum LSF cluster nodes. To leverage this functionality, users must first deploy an IBM Storage Scale cluster with CES enabled as a prerequisite. Once set up, CES-based NFS mount points can be exported to the LSF cluster as shared mount points. This integration allows the LSF cluster to access the same mount points and share data with the Storage Scale cluster, enabling the deployment of a high-performance file system within your HPC cluster.

The offering supports the bring-your-own-license (BYOL) model for IBM Spectrum LSF to deploy an HPC cluster on IBM Cloud. Make sure that you have sufficient software licenses to deploy the required capacity on the IBM Cloud cluster. For evaluation purposes, IBM Cloud does enable limited access. Contact your IBM Cloud sales or support team for evaluation licenses.

The Spectrum LSF enables all three interfaces: UI, API, and CLI.

IBM Spectrum LSF also offers the LSF Application Center, which provides a flexible, easy-to-use interface for cluster users and administrators. It is available as an add-on module to IBM Spectrum LSF, the LSF Application Center enables users to interact with intuitive, self-documenting, standardized interfaces. You can access the LSF Application Center through the GUI, and you can also access the API calls with Python and curl.

The LSF cluster is configured not only with the Application Center feature but also with the Application Center High Availability (HA) functionality. In the event of a failover, the PAC feature remains operational, ensuring that users can still access and interact with the GUI. Jobs continue to run as long as at least one LSF management host is available. The cluster deploys GUI services across three GUI servers, with the database hosted on one of these GUI hosts.

The offering enables the initial Spectrum LSF-based HPC cluster creation. Any updates that are needed post-deployment regarding LSF configuration or setup must be performed by using LSF tools and commands. If you use the Schematics interface to change configuration properties and reapply those changes, you can cause disruptions to the running Spectrum LSF cluster. Restoring it back to a working state might not be easy.

The current solution no longer requires ibm_customer_number(ICN) for entitlement check before deploying the solution for non-production use. The solution is now available for use without ICN validation. Users can provision up to a maximum of 10 static worker nodes for evaluation or non-production use cases. If the number of worker nodes exceeds 10, it becomes the user responsibility to obtain the necessary entitlement check and licensing for those additional nodes in the production environment. For production use or for evaluating greater than 10 worker nodes, the user must purchase the necessary LSF licenses. To purchase the license, go to Purchasing licenses.