add user guide for accessing spark Jupyter lab #33
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #33 +/- ##
=======================================
Coverage 48.57% 48.57%
=======================================
Files 4 4
Lines 105 105
=======================================
Hits 51 51
Misses 54 54 ☔ View full report in Codecov by Sentry. |
| Please be aware that your session will automatically close after `1 hour`. Should you require an extension, simply invoke | ||
| `get_spark_session()` to initiate a new session. |
There was a problem hiding this comment.
this means the prior session is still sitting around wasting resources, right?
There was a problem hiding this comment.
no. after 1 hr, the prior session will be closed by the timer.
There was a problem hiding this comment.
But for that hour it is, that's what I'm talking about
There was a problem hiding this comment.
yea. for the hour the session is open, it's taking resources.
There was a problem hiding this comment.
I suppose we could keep track of open sessions and close them when a new session is requested but that's probably overkill for now
| This error occurs when there is an issue with the SparkMonitor kernel extension, specifically related to the | ||
| communication between the kernel and the front end. The error indicates that the comm attribute is missing from the | ||
| ScalaMonitor object. |
There was a problem hiding this comment.
From the extension developer:
This is a difficult error to debug; the kernel extension is complaining that the Scala Listener has no comms; this is usually because the listener wasn't able to find a Spark Instance.
itsjafer/jupyterlab-sparkmonitor#16
There was a problem hiding this comment.
Hmm, that ticket says the bug is fixed
There was a problem hiding this comment.
yea. It didn't happen very often. And restart kernel always works.
No description provided.