HistoryServer is a web interface for completed and running (aka incomplete) Spark applications.
You can start a HistoryServer instance by executing $SPARK_HOME/sbin/start-history-server.sh script. See Starting HistoryServer.
|
Tip
|
Use EventLoggingListener to collect events. |
|
Tip
|
Enable Add the following line to Refer to Logging. |
You can start a HistoryServer instance by executing $SPARK_HOME/sbin/start-history-server.sh script.
$ ./sbin/start-history-server.sh
starting org.apache.spark.deploy.history.HistoryServer, logging to .../spark/logs/spark-jacek-org.apache.spark.deploy.history.HistoryServer-1-japila.out|
Note
|
To stop the server execute stop-history-server.sh or kill it.
|
When started, it prints out the following INFO message to the logs:
INFO HistoryServer: Started daemon with process name: [processName]It registers signal handlers (using SignalUtils) for TERM, HUP, INT to log their execution:
ERROR HistoryServer: RECEIVED SIGNAL [signal]It inits security if enabled (using spark.history.kerberos.enabled setting).
|
Caution
|
FIXME Describe initSecurity
|
It creates a SecurityManager.
It creates a ApplicationHistoryProvider (by reading spark.history.provider).
It reads spark.history.ui.port.
It creates a HistoryServer and requests to bind.
It registers a shutdown hook to call stop on the HistoryServer instance.
-
spark.history.provider(default: FsHistoryProvider) is a fully-qualified class name for a ApplicationHistoryProvider that comes with a single-arg constructor acceptingSparkConf.
-
spark.history.ui.port(default:18080) — the port of the History Server’s UI.