Introduction to Native Spark engine

Native Spark engine is a compute engine in IBM® watsonx.data. You can use Native Spark engine to submit applications that involve complex analytical operations.

In watsonx.data, Native Spark engine is used to achieve the following use cases:

• Ingest large volumes of data into watsonx.data tables.
• Handle complex analytical workload.
• Table maintenance operation to enhance watsonx.data performance of the table
• Develop, run, and debug applications written in Python, R, and Scala.

You can also register an external Spark engine from the Add engine window of the watsonx.data UI. IBM® Analytics Engine on Cloud is the external Spark engine here. To provision an external Spark engine on Cloud, see Provisioning Analytics Engine. For more information about registering the engine, see Registering an engine.