Spark user interface
The Spark user interface (Spark UI) helps you to keep track of various aspects of a running Spark application.
The following list includes a few examples:
- current running stage
- number of tasks in a stage
- reason for a longer running stage
- strangler task in a stage
- whether the executors in the application are used optimally
- inspect into memory and disk consumption of the driver and executors
Use the Spark history server to inspect the run of a completed Spark application. To access the Spark history server, see the Access Spark history server.
Accessing the Spark UI
-
Log in to watsonx.data console.
-
From the navigation menu, select Infrastructure manager.
-
Click the name of the Spark engine (from list view or topology view). Engine information window opens.
-
In the Applications tab, select an application and click the application ID link. The Spark UI opens.
-
You can view the following details :
- The Event Time line displays a graphical view of the timeline and the events.
- Different stages of execution, the storage used, the Spark environment and executor (memory and driver) details.