Provisioning a serverless Spark engine for Lite plan

You can add serverless Spark engine for IBM® watsonx.data Lite plan instance. Native serverless Spark engine is a compute engine that resides within IBM® watsonx.data.

You cannot register an External Spark engine for watsonx.data Lite plan.

To add a Spark engine, complete the following steps.

1. Log in to watsonx.data console.

2. From the navigation menu, select Infrastructure manager.

3. To add a Spark engine, click Add component and click Next.

4. In the Add component page, from the Engines section, select IBM Spark.

5. In the Add component - IBM Spark page, configure the following details:

   a. In the Add component - IBM Spark window, enter the Display name for your Spark engine.

   b. The Registration mode is by default Create a native Spark engine.

   c. Configure the following details:

   Provisioning Spark engine

   Field	Description
   Default Spark version	Select the Spark runtime version that must be considered for processing the applications. For supported Spark versions, see Supported Spark version. Lite version of watsonx.data does not support Spark version 4.0.
   Engine home bucket	Select the registered Cloud Object Storage bucket from the list to store the Spark events and logs that are generated while running spark applications. Note Make sure you do not select the IBM-managed bucket as Spark Engine home. If you select an IBM-managed bucket, you cannot access it to view the logs. For more information, see Before you begin.
   Instance resource quota	Displays the memory limit of the Spark engine. For Lite plan, it will be 8 vCPU and 32 GB. You cannot modify the value.
   Associated catalogs (optional)	Select the catalogs that must be associated with the engine.

6. Click Create. The engine is provisioned and is displayed in the Infrastructure Manager page.