add JUPYTER_MODE for JupyterHub #77
Conversation
| @@ -179,6 +181,65 @@ services: | |||
| volumes: | |||
| - ./cdr/cdm/jupyter/user_shared_workspace:/cdm_shared_workspace/user_shared_workspace | |||
|
|
|||
| dev_jupyterhub: | |||
There was a problem hiding this comment.
We probably only need one jupyterhub instance eventually and control permission based on different user groups. But I haven't completely sort it out yet.
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #77 +/- ##
=======================================
Coverage 48.76% 48.76%
=======================================
Files 4 4
Lines 121 121
=======================================
Hits 59 59
Misses 62 62 ☔ View full report in Codecov by Sentry. |
| - POSTGRES_USER=hive | ||
| - POSTGRES_PASSWORD=hivepassword | ||
| - POSTGRES_DB=hive | ||
| - POSTGRES_URL=postgres:5432 |
There was a problem hiding this comment.
Not really the point of this PR but it makes me nervous that users have access to these creds...
There was a problem hiding this comment.
yea. We do have a todo item for this.
# TODO: create postgres user w/ only write access to the hive tables
There was a problem hiding this comment.
Even that would make me nervous. It means that any user could blow away the hive tables
There was a problem hiding this comment.
I updated the todo item to
# TODO: create postgres user r/ only read access to the hive tables
There was a problem hiding this comment.
Users won't need to create tables?
There was a problem hiding this comment.
I guess another question is if users and devs are all in the same jupyterhub instance is it possible to have different environments
There was a problem hiding this comment.
Users won't since they don't have minIO write permission. I am hoping it can be configured. But TBD.
There was a problem hiding this comment.
Ok, so users = read only, devs = write, essentially.
There was a problem hiding this comment.
If we could ever get the remote metastore working that might provide some protection as well, not sure
docker-compose.yaml
Outdated
| volumes: | ||
| - ./cdr/cdm/jupyter/home:/home |
There was a problem hiding this comment.
no. Each user will have their own dir and python environment.
There was a problem hiding this comment.
Should dev_jupyterhub have the shared workspace mounted in that case?
There was a problem hiding this comment.
both instances mount - ./cdr/cdm/jupyter/jupyterhub/users_home:/jupyterhub/users_home. Each user will have their own dir inside of /jupyterhub/users_home and create notebook there.
There was a problem hiding this comment.
My point is that dev-jupyterhub also mounts /cdr/cdm/jupyter:/cdm_shared_workspace. I'm asking if that line needs to be there or if it should be removed to match user_juypterhub
There was a problem hiding this comment.
I don't see any issues with using - ./cdr/cdm/jupyter:/cdm_shared_workspace for the dev environment.
There was a problem hiding this comment.
I'm not saying there's an issue, I just wasn't sure why it was there. can you remind me what's in the hive_metastore and cdm-postgres dirs? Are they both still relevant with the standalone postgres service?
There was a problem hiding this comment.
I think cdm-postgres is not longer used, it's not getting updated for 2 months.
In hive_metastore there are <db_name>.db files. I think they are used when users run spark select statement.
There was a problem hiding this comment.
Ok, maybe cdm-postgres was the local DB, and hive_metastore is cache data?
There was a problem hiding this comment.
not exactly sure. But I want to worry about those later.
No description provided.