SuperMap iManager K8S
简体中文

GIS Cloud Suite

The Spark Cluster Management and Monitoring

GIS Cloud Suite supports to monitor and manage the environment of Spark Cluster.

Spark Cluster Management

In the Spark Cluster page, all the added spark cluster environments are shown in the list, as the screenshow below.

Users can view the configuration of the spark cluster, delete external spark cluster, disable built-in spark cluster, and enter the console of the built-in spark cluster.

sparklist

Service Management

Clicks on Console to enter the page of Spark Cluster services list. As the screenshot below, manage the services by the following functions:

servicelist

  • Name: The list displays the main services that support the Spark Cluster environment, clicks on the name to enter the containers management page.
  • Address: The address of the service, clicks the link to see the service details of spark master/worker.
  • Description: Introduce the service.
  • Status: Shows the number of running/total replicas in the service.
  • Redeploy: Redeploys the service.

  • Scaling: Manual/Automatic scaling the service node.

    • Manual Scaling: Set the number of nodes, the system would scaling the nodes to the expected number.

    sparkmanualelastic

    • Automatic Scaling: Set the monitoring type, threshold, minimum node(s), and maximum nodes, the system would automatically scaling the nodes according to the loads. The automatic scaling could be deleted by clicking on the button beside the service name.

    sparkautoelastic

  • Adjust Spec: Adjusts the spec of CPU or Memory.

Container Management

Clicks on the name of Spark Cluster Service to enter the container page, the page lists the information of container name, IP, status, duration, and host machine. If the container malfunctioned, you can recreate the container, the service would stop working until finishing recreating.

Clicks on Logs to see the container’s log, the log has the running record of the container from creation to the present.

sparklog

Clicks on Command pad to enter the container’s command interface.

The screenshot below is an example of using ‘ls’ command to view the directory list of the container.

sparkcontrollpad

Notes:
Using [shift + insert] to paste the commands into the Command Pad.

Spark Cluster Monitoring

Service Monitoring

Clicks on Computing Resources > Spark Cluster > Console to enter the page of the Spark Cluster Console and roll to the monitoring panels.

The monitoring panels record the real-time indicators of CPU useage, memory useage, network in/out, and filesystem useage of the services. Clicks on the service name in the legend to check the specific service recording. The monitoring panels could be enlarged or narrowed, and draged to other places of the page. More functions are listed below:

  • Select recording range: Choose the time range of the monitor recording.
  • Set refresh time: Set the refresh interval of the panels.
  • Refresh: Click the button to refresh the panels.
  • Save dashboard: After changing the panels’ size or draging the panels, click the button to save the current layout.
  • Versions: All the layout styles are saved in the Versions, the Versions has the ability of restoring the layout to any style.

Container Monitoring

On the Spark Cluster Console page, clicks on the name of service to enter the container page. The monitoring panels record the real-time indicators of CPU useage, memory useage, network in/out, and filesystem useage for the containers. The monitoring panels could be enlarged or narrowed, and draged to other places of the page. Users could also save layout, manage versions, set custom time range, set refresh time, and refresh the panel manually.