Swap ml-workspace container and add admin cookbook

deploy/install.sh

@@ -19,7 +19,7 @@ bash script/base.sh || exit
 printf "\n \n Download the required docker images...\n "
 printf ".........\n "
 docker pull traefik:v2.10
-docker pull mltooling/ml-workspace:0.13.2
+docker pull mltooling/ml-workspace-minimal:0.13.2
 printf "\n\n docker images successfully downloaded...\n \n \n "
 
 
@@ -59,7 +59,7 @@ docker run -d \
   --env WORKSPACE_BASE_URL="user1" \
   --shm-size 512m \
   --restart always \
-  mltooling/ml-workspace:0.13.2 || true
+  mltooling/ml-workspace-minimal:0.13.2 || true
 
 docker run -d \
  -p 8091:8080 \
@@ -70,7 +70,7 @@ docker run -d \
   --env WORKSPACE_BASE_URL="user2" \
   --shm-size 512m \
   --restart always \
-  mltooling/ml-workspace:0.13.2 || true
+  mltooling/ml-workspace-minimal:0.13.2 || true
 
 #-------------
 printf "\n \n Start the traefik gateway server\n "

deploy/single-script-install.sh

@@ -97,7 +97,7 @@ printf "\n\n End of installing dependencies...\n\n\n "
 printf "Download the required docker images...\n "
 printf ".........\n\n\n "
 docker pull traefik:v2.10
-docker pull mltooling/ml-workspace:0.13.2
+docker pull mltooling/ml-workspace-minimal:0.13.2
 printf "\n\n docker images successfully downloaded...\n \n \n "
 #----
 
@@ -165,7 +165,7 @@ docker run -d \
   --env WORKSPACE_BASE_URL="user1" \
   --shm-size 512m \
   --restart always \
-  mltooling/ml-workspace:0.13.2 || true
+  mltooling/ml-workspace-minimal:0.13.2 || true
 
 docker run -d \
  -p 8091:8080 \
@@ -176,7 +176,7 @@ docker run -d \
   --env WORKSPACE_BASE_URL="user2" \
   --shm-size 512m \
   --restart always \
-  mltooling/ml-workspace:0.13.2 || true
+  mltooling/ml-workspace-minimal:0.13.2 || true
 
 #-------------
 printf "\n\n Start the traefik gateway server\n "

deploy/vagrant/make_boxes/dtaas/user.sh

@@ -72,7 +72,7 @@ cat /vagrant/vagrant.pub >> /root/.ssh/authorized_keys
 
 # get the required docker images
 docker pull traefik:v2.10
-docker pull mltooling/ml-workspace:0.13.2
+docker pull mltooling/ml-workspace-minimal:0.13.2
 docker pull grafana/grafana:10.1.4
 docker pull influxdb:2.7
 docker pull rabbitmq:3-management

docs/admin/client/CLIENT.md

@@ -125,7 +125,7 @@ docker run -d \
   --env WORKSPACE_BASE_URL="alice" \
   --shm-size 512m \
   --restart always \
-  mltooling/ml-workspace:0.13.2
+  mltooling/ml-workspace-minimal:0.13.2
 
 docker run -d \
  -p 8091:8080 \
@@ -136,7 +136,7 @@ docker run -d \
   --env WORKSPACE_BASE_URL="bob" \
   --shm-size 512m \
   --restart always \
-  mltooling/ml-workspace:0.13.2
+  mltooling/ml-workspace-minimal:0.13.2
 ```
 
 Given that multiple services are running at different routes,

docs/admin/client/auth.md

@@ -102,4 +102,4 @@ DTaaS application URL: https://foo.com/bar
 Gitlab instance URL: https://foo.gitlab.com
 Callback URL: https://foo.com/bar/Library
 Logout URL: https://foo.com/bar
-```
+```

docs/admin/guides/add_service.md

@@ -0,0 +1,112 @@
+# Add other services
+
+<!-- prettier-ignore -->
+!!! Pre-requisite
+    You should read the documentation about
+    the already available [services](../services.md)
+
+This guide will show you how to add more services.
+In the following example we will be adding **mongodb** as a service,
+but this could be used for other services as well.
+
+:fontawesome-solid-circle-info:
+**Adding other services requires more RAM and CPU power.**
+**Please make sure the host machine meets the hardware requirements**
+**for running all the services.**
+
+## 1. Add the configuration
+
+You should add the following configuration variables.
+
+| Configuration Variable Name | Description                                                       |
+| :-------------------------- | :---------------------------------------------------------------- |
+| username                    | the username of the root user in the mongodb                      |
+| password                    | the password of the root user in the mongodb                      |
+| port                        | the mapped port on the host machine (default is 27017)            |
+| datapath                    | path on host machine to mount the data from the mongodb container |
+
+Open the file `/deploy/services/services.yml` and add the configuration for mongodb:
+
+```yml
+services:
+    rabbitmq:
+        username: "dtaas"
+        password: "dtaas"
+        vhost: "/"
+        ports:
+            main: 5672
+            management: 15672
+    ...
+    mongodb:
+        username: <username>
+        password: <password>
+        port: <port>
+        datapath: <datapath>
+    ...
+```
+
+## 2. Add the script
+
+The next step is to add the script that set up the mongodb container with the configuraiton.
+
+Create following file `/deploy/services/mongodb.js` and add the following code:
+
+```js
+#!/usr/bin/node
+/* Install the optional platform services for DTaaS */
+import { $ } from "execa";
+import chalk from "chalk";
+import fs from "fs";
+import yaml from "js-yaml";
+
+const $$ = $({ stdio: "inherit" });
+const log = console.log;
+let config;
+
+try {
+  log(chalk.blue("Load services configuration"));
+  config = await yaml.load(fs.readFileSync("services.yml", "utf8"));
+  log(
+    chalk.green(
+      "configuration loading is successful and config is a valid yaml file"
+    )
+  );
+} catch (e) {
+  log(chalk.red("configuration is invalid. Please rectify services.yml file"));
+  process.exit(1);
+}
+
+log(chalk.blue("Start MongoDB server"));
+const mongodbConfig = config.services.mongodb;
+
+try {
+  log(
+    chalk.green(
+      "Attempt to delete any existing MongoDB server docker container"
+    )
+  );
+  await $$`docker stop mongodb`;
+  await $$`docker rm mongodb`;
+} catch (e) {}
+
+log(chalk.green("Start new Mongodb server docker container"));
+await $$`docker run -d -p ${mongodbConfig.port}:27017 \
+--name mongodb \
+-v ${mongodbConfig.datapath}:/data/db \
+-e MONGO_INITDB_ROOT_USERNAME=${mongodbConfig.username} \
+-e MONGO_INITDB_ROOT_PASSWORD=${mongodbConfig.password} \
+mongo`;
+log(chalk.green("MongoDB server docker container started successfully"));
+```
+
+## 3. Run the script
+
+Go to the directory `/deploy/services/`
+and run services script with the following commands:
+
+```bash
+yarn install
+node mongodb.js
+```
+
+The MongoDB should now be available on **services.foo.com:<port\>**.

docs/admin/guides/add_user.md

@@ -0,0 +1,82 @@
+# Add a new user
+
+This page will guide you on, how to add more users to the DTaas. Please do the following:
+
+<!-- prettier-ignore -->
+!!! important
+  Make sure to replace **<username\>** and **<port\>**
+  Select a port that is not already being used by the system.
+
+## 1. Add user
+
+Add the new user on the Gitlab instance.
+
+## 2. Setup a new workspace
+
+The above code creates a new workspace for the new user based on "user2".
+
+```bash
+cd DTaaS/files
+cp -R user2 <username>
+cd ..
+docker run -d \
+ -p <port>:8080 \
+  --name "ml-workspace-<username>" \
+  -v "${TOP_DIR}/files/<username>:/workspace" \
+  -v "${TOP_DIR}/files/<username>:/workspace/common" \
+  --env AUTHENTICATE_VIA_JUPYTER="" \
+  --env WORKSPACE_BASE_URL="<username>" \
+  --shm-size 512m \
+  --restart always \
+  mltooling/ml-workspace-minimal:0.13.2
+
+```
+
+## 3. Add username and password
+
+The following code adds basic authentication for the new user.
+
+```bash
+cd DTaaS/servers/config/gateway
+htpasswd auth <username>
+```
+
+## 4. Add 'route' for new user
+
+Now we just need to add a new route to the servers ingress.
+
+Open the following file with your preffered editor (e.g. VIM/nano).
+
+```bash
+vi DTaaS/servers/config/gateway/dynamic/fileConfig.yml
+```
+
+Now add the new route and service for the user.
+
+<!-- prettier-ignore -->
+!!! important
+  foo.com should be replaced with your own domain.
+
+```yml
+http:
+  routers:
+    ....
+    <username>:
+      entryPoints:
+        - http
+      rule: 'Host(`foo.com`) && PathPrefix(`/<username>`)'
+      middlewares:
+        - basic-auth
+      service: <username>
+
+  services:
+    ...
+    <username>:
+      loadBalancer:
+        servers:
+          - url: 'http://localhost:<port>'
+```
+
+## 5. Access the new user
+
+Log into the DTaaS application as new user

docs/admin/guides/common_workspace_readonly.md

@@ -0,0 +1,38 @@
+# Make common asset area read only
+
+## Why
+
+In some cases you might want to restrict the users rights to the common workspace.
+In order to make the common area read only,
+you have to change the install script for setting up the user workspace.
+
+## How
+
+To make the common area read-only for user2,
+the following changes were made to the install script,
+which is located one of the following places.
+
+- trial installation: `single-script-install.sh`
+
+- production installation: `DTaas/deploy/install.sh`
+
+The line `-v "${TOP_DIR}/files/common:/workspace/common:ro"`
+was added to make the common workspace read-only for user2.
+
+Here's the updated code:
+
+```sh
+docker run -d \
+  -p 8091:8080 \
+  --name "ml-workspace-user2" \
+  -v "${TOP_DIR}/files/user2:/workspace" \
+  -v "${TOP_DIR}/files/common:/workspace/common:ro" \
+  --env AUTHENTICATE_VIA_JUPYTER="" \
+  --env WORKSPACE_BASE_URL="user2" \
+  --shm-size 512m \
+  --restart always \
+  mltooling/ml-workspace-minimal:0.13.2 || true
+```
+
+This ensures that the common area is read-only for user2,
+while the user's own files are still writable.

docs/admin/guides/hosting_site_without_https.md

@@ -0,0 +1,6 @@
+# Hosting site without https
+
+In the default setup,
+the https connection is provided by the reverse proxy.
+The DTaaS application runs in http mode.
+So removing the reverse proxy removes the https mode.

docs/admin/guides/link_service.md

@@ -0,0 +1,38 @@
+# Link services to local ports
+
+<!-- prettier-ignore -->
+!!! Requirements
+
+    - User needs to have an account on server2.
+    - SSH server must be running on server2
+
+To link a port from the service machine (server2)
+to the local port on the user workspace.
+You can make a ssh link between these two.
+
+## 1. Step
+
+Go to the user workspace, on which you want to map
+from localhost to the services machine
+
+- e.g. `foo.com/user1`
+
+## 2. Step
+
+Open a terminal in your user workspace.
+![Open terminal in user workspace](open_terminal_in_workspace.gif)
+
+## 3. Step
+
+Run the following command to map a port:
+
+```sh
+ssh -fNT -L <local_port>:<destination>:<destination_port> <user>@<ssh_server>
+
+```
+
+Here's an example mapping the service port 5672 to localhost port 5672.
+
+```sh
+ssh -fNT -L 5672:localhost:5672 [email protected]
+```