Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

301 run dtaas with localhost domain name #348

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .codeclimate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,3 +35,4 @@ exclude_patterns:
- "**/public/"
- client/config/
- servers/lib/src/types.ts
- "deploy/config/client/env*.js"
19 changes: 19 additions & 0 deletions deploy/config/client/env.local.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
window.env = {
nichlaes marked this conversation as resolved.
Show resolved Hide resolved
nichlaes marked this conversation as resolved.
Show resolved Hide resolved
REACT_APP_ENVIRONMENT: "dev",
REACT_APP_URL: "http://localhost/",
REACT_APP_URL_BASENAME: "",
REACT_APP_URL_DTLINK: "/lab",
REACT_APP_URL_LIBLINK: "",
REACT_APP_WORKBENCHLINK_TERMINAL: "/terminals/main",
REACT_APP_WORKBENCHLINK_VNCDESKTOP: "/tools/vnc/?password=vncpassword",
REACT_APP_WORKBENCHLINK_VSCODE: "/tools/vscode/",
REACT_APP_WORKBENCHLINK_JUPYTERLAB: "/lab",
REACT_APP_WORKBENCHLINK_JUPYTERNOTEBOOK: "",

REACT_APP_CLIENT_ID:
"1be55736756190b3ace4c2c4fb19bde386d1dcc748d20b47ea8cfb5935b8446c",
REACT_APP_AUTH_AUTHORITY: "https://gitlab.com/",
REACT_APP_REDIRECT_URI: "http://localhost/Library",
REACT_APP_LOGOUT_REDIRECT_URI: "http://localhost/",
REACT_APP_GITLAB_SCOPES: "openid profile read_user read_repository api",
};
18 changes: 18 additions & 0 deletions deploy/config/client/env.trial.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
window.env = {
REACT_APP_ENVIRONMENT: 'dev',
REACT_APP_URL: 'https://foo.com/',
REACT_APP_URL_BASENAME: '',
REACT_APP_URL_DTLINK: '/lab',
REACT_APP_URL_LIBLINK: '',
REACT_APP_WORKBENCHLINK_TERMINAL: '/terminals/main',
REACT_APP_WORKBENCHLINK_VNCDESKTOP: '/tools/vnc/?password=vncpassword',
REACT_APP_WORKBENCHLINK_VSCODE: '/tools/vscode/',
REACT_APP_WORKBENCHLINK_JUPYTERLAB: '/lab',
REACT_APP_WORKBENCHLINK_JUPYTERNOTEBOOK: '',

REACT_APP_CLIENT_ID: '934b98f03f1b6f743832b2840bf7cccaed93c3bfe579093dd0942a433691ccc0',
REACT_APP_AUTH_AUTHORITY: 'https://gitlab.foo.com/',
REACT_APP_REDIRECT_URI: 'https://foo.com/Library',
REACT_APP_LOGOUT_REDIRECT_URI: 'https://foo.com/',
REACT_APP_GITLAB_SCOPES: 'openid profile read_user read_repository api',
};
35 changes: 35 additions & 0 deletions deploy/config/gateway/fileConfig.local.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
http:
routers:
dtaas:
entryPoints:
- http
rule: "Host(`localhost`)"
service: dtaas

user1:
entryPoints:
- http
rule: "Host(`localhost`) && PathPrefix(`/user1`)"
nichlaes marked this conversation as resolved.
Show resolved Hide resolved
service: user1

libms:
entryPoints:
- http
rule: "Host(localhost`) && PathPrefix(`/lib`)"
service: libms

services:
dtaas:
loadBalancer:
servers:
- url: "http://localhost:4000"

user1:
loadBalancer:
servers:
- url: "http://localhost:8090"

libms:
loadBalancer:
servers:
- url: "http://localhost:4001"
48 changes: 48 additions & 0 deletions deploy/config/gateway/fileConfig.trial.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
http:
routers:
dtaas:
entryPoints:
- http
rule: 'Host(`foo.com`)'
middlewares:
- basic-auth
service: dtaas

user1:
entryPoints:
- http
rule: 'Host(`foo.com`) && PathPrefix(`/user1`)'
middlewares:
- basic-auth
service: user1

libms:
entryPoints:
- http
rule: 'Host(`foo.com`) && PathPrefix(`/lib`)'
service: libms


# Middleware: Basic authentication
middlewares:
basic-auth:
basicAuth:
usersFile: "/etc/traefik/auth"
removeHeader: true


services:
dtaas:
loadBalancer:
servers:
- url: "http://localhost:4000"

user1:
loadBalancer:
servers:
- url: "http://localhost:8090"

libms:
loadBalancer:
servers:
- url: "http://localhost:4001"
6 changes: 6 additions & 0 deletions deploy/config/lib.local
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
PORT='4001'
MODE='local'
LOCAL_PATH ='filepath'
LOG_LEVEL='debug'
APOLLO_PATH='/lib'
GRAPHQL_PLAYGROUND='true'
9 changes: 9 additions & 0 deletions deploy/config/lib.trial
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
PORT='4001'
MODE='local' or 'gitlab'
LOCAL_PATH ='filepath'
GITLAB_GROUP ='dtaas'
GITLAB_URL='https://gitlab.foo.com/api/graphql'
TOKEN='123-sample-token'
LOG_LEVEL='debug'
APOLLO_PATH='/lib' or ''
GRAPHQL_PLAYGROUND='false' or 'true'
2 changes: 1 addition & 1 deletion deploy/install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ TOP_DIR=$(pwd)
printf "\n \n Build, configure and run the react website\n "
printf ".....\n "
cd "${TOP_DIR}/client" || exit
yarn install
yarn install --production
yarn build

#one of the environments; specify only one; "dev" used the REACT_APP_ENV is not set
Expand Down
51 changes: 35 additions & 16 deletions deploy/single-script-install.sh
Original file line number Diff line number Diff line change
@@ -1,5 +1,23 @@
#!/bin/bash
set -eu
while [[ "$#" -gt 0 ]]; do
case $1 in
--env)
env_variable="$2"
shift
;;
*)
echo "Unknown parameter passed: $1"
exit 1
;;
esac
shift
done

set -e

if [ -n "$env_variable" ] ; then
printf "environment: %s.\n" "$env_variable"
fi

printf "Install script for DTaaS software platform.\n"
printf "You can run the script multiple times until the installation succeeds.\n "
Expand Down Expand Up @@ -125,17 +143,26 @@ TOP_DIR=$(pwd)
printf "\n\n Build, configure and run the react website\n "
printf ".....\n "
cd "${TOP_DIR}/client" || exit
yarn install
yarn install --production
yarn build

yarn configapp dev
cp "${TOP_DIR}/deploy/config/client/env.js" build/env.js
if [ -n "$env_variable" ] ; then
cp "${TOP_DIR}/deploy/config/client/env.${env_variable}.js" build/env.js
else
cp "${TOP_DIR}/deploy/config/client/env.js" build/env.js
fi
nohup serve -s build -l 4000 & disown

#-------------
printf "\n\n Build, configure and run the lib microservice\n "
printf "...........\n "
cd "${TOP_DIR}/servers/lib" || exit
if [ -n "$env_variable" ] ; then
cp "${TOP_DIR}/deploy/config/lib.${env_variable}" .env
else
cp "${TOP_DIR}/deploy/config/lib" .env
fi
yarn install
yarn build

Expand Down Expand Up @@ -167,24 +194,16 @@ docker run -d \
--restart always \
mltooling/ml-workspace-minimal:0.13.2 || true

docker run -d \
-p 8091:8080 \
--name "ml-workspace-user2" \
-v "${TOP_DIR}/files/user2:/workspace" \
-v "${TOP_DIR}/files/common:/workspace/common" \
--env AUTHENTICATE_VIA_JUPYTER="" \
--env WORKSPACE_BASE_URL="user2" \
--shm-size 512m \
--restart always \
mltooling/ml-workspace-minimal:0.13.2 || true

#-------------
printf "\n\n Start the traefik gateway server\n "
printf "...........\n "
cd "${TOP_DIR}/servers/config/gateway" || exit
cp "${TOP_DIR}/deploy/config/gateway/auth" auth
cp "${TOP_DIR}/deploy/config/gateway/fileConfig.yml" "dynamic/fileConfig.yml"

if [ -n "$env_variable" ] ; then
cp "${TOP_DIR}/deploy/config/gateway/fileConfig.${env_variable}.yml" "dynamic/fileConfig.yml"
else
cp "${TOP_DIR}/deploy/config/gateway/fileConfig.yml" "dynamic/fileConfig.yml"
fi
docker run -d \
--name "traefik-gateway" \
--network=host -v "$PWD/traefik.yml:/etc/traefik/traefik.yml" \
Expand Down
79 changes: 79 additions & 0 deletions docs/admin/localhost.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
# Localhot Installation

To try out the software, you can install it on Ubuntu Server 22.04
Operating System. The setup requires a
machine which can spare 16GB RAM, 8 vCPUs and 50GB Hard Disk
space to the vagrant box.
A successful installation will create a setup
similar to the one shown in the figure.

![Single host install](./single-host.png)

A one-step installation script is provided on this page. This script sets up
the DTaaS software with a user.
You can use it to check a test installation of DTaaS software.

## Pre-requisites

### 1. Gitlab OAuth application

The DTaaS react website requires Gitlab OAuth provider.
If you need more help with this step, please see
the [Authentication page](client/auth.md).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is sufficient to have user-owned oauth application for localhost and trial installations. Pæease mention this point in information box both here and on trial installation page.

Please also create a section at the 3nd of auth.md mentioning about the use of user-owned oauth application.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@prasadtalasila I don't understand the second part about the auth.md for the use of user-owned oauth

Also what the 3nd section of auth.md is


!!! Information
<!-- markdownlint-disable-file MD013 -->
It is sufficient to have [user-owned oauth](https://docs.gitlab.com/ee/integration/oauth_provider.html#create-a-user-owned-application)
application.

nichlaes marked this conversation as resolved.
Show resolved Hide resolved
You need the following information from the OAuth application registered on Gitlab:

| Gitlab Variable Name | Variable name in Client env.js | Default Value |
| :------------------- | :----------------------------- | :----------------------------------------------- |
| OAuth Provider | REACT_APP_AUTH_AUTHORITY | <https://gitlab.foo.com/> |
| Application ID | REACT_APP_CLIENT_ID |
| Callback URL | REACT_APP_REDIRECT_URI | <http://localhost/Library> |
| Scopes | REACT_APP_GITLAB_SCOPES | openid, profile, read_user, read_repository, api |

You can also see
[Gitlab help page](https://docs.gitlab.com/ee/integration/oauth_provider.html)
for getting the Gitlab OAuth application details.

Remember to create gitlab accounts for `user1`.

## Install

<!-- prettier-ignore -->
!!! note
While installing you might encounter multiple dialogs asking,
which services should be restarted. Just click **OK** to all of those.

Run the following scripts. To setup the installation to use localhost,
the following argument is added to the script `--env local`.

```bash
wget https://raw.githubusercontent.com/INTO-CPS-Association/DTaaS/feature/distributed-demo/deploy/single-script-install.sh
bash single-script-install.sh --env local
```

<!-- prettier-ignore -->
!!! warning
This test installation has default credentials and is thus highly insecure.

## Post-install Check

Now when you visit your domain, you should be able to login through your
OAuth Provider and be able to access the DTaas web UI.

If you can following all the screenshots from
[user website](../user/website/index.md).
Everything is correctly setup.

## References

Image sources: [Ubuntu logo](https://logodix.com/linux-ubuntu),
[Traefik logo](https://www.laub-home.de/wiki/Traefik_SSL_Reverse_Proxy_f%C3%BCr_Docker_Container),
[ml-workspace](https://github.com/ml-tooling/ml-workspace),
[nodejs](https://www.metachris.com/2017/01/how-to-install-nodejs-7-on-ubuntu-and-centos/),
[reactjs](https://krify.co/about-reactjs/),
[nestjs](https://camunda.com/blog/2019/10/nestjs-tx-email/)
12 changes: 9 additions & 3 deletions docs/admin/trial.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,11 @@ A one-step installation script is provided on this page. This script sets up
the DTaaS software with default credentials and users.
You can use it to check a test installation of DTaaS software.

!!! Information
<!-- markdownlint-disable-file MD013 -->
It is sufficient to have [user-owned oauth](https://docs.gitlab.com/ee/integration/oauth_provider.html#create-a-user-owned-application)
application.

nichlaes marked this conversation as resolved.
Show resolved Hide resolved
## Pre-requisites

### 1. Domain name
Expand Down Expand Up @@ -40,7 +45,7 @@ You can also see
[Gitlab help page](https://docs.gitlab.com/ee/integration/oauth_provider.html)
for getting the Gitlab OAuth application details.

Remember to create gitlab accounts for `user1` and `user2`.
Remember to create gitlab accounts for `user1`.

## Install

Expand All @@ -49,11 +54,12 @@ Remember to create gitlab accounts for `user1` and `user2`.
While installing you might encounter multiple dialogs asking,
which services should be restarted. Just click **OK** to all of those.

Run the following scripts.
Run the following scripts. To setup the installation to use trial,
the following argument is added to the script `--env trial`.

```bash
wget https://raw.githubusercontent.com/INTO-CPS-Association/DTaaS/feature/distributed-demo/deploy/single-script-install.sh
bash single-script-install.sh
bash single-script-install.sh --env trial
prasadtalasila marked this conversation as resolved.
Show resolved Hide resolved
```

<!-- prettier-ignore -->
Expand Down
1 change: 1 addition & 0 deletions mkdocs-github.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ nav:
- Overview: admin/overview.md
- Authentication: admin/client/auth.md
- Host Install:
- Localhost: admin/localhost.md
- Trial: admin/trial.md
- Production: admin/host.md
- Vagrant:
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ nav:
- Overview: admin/overview.md
- Authentication: admin/client/auth.md
- Host Install:
- Localhost: admin/localhost.md
- Trial: admin/trial.md
- Production: admin/host.md
- Vagrant:
Expand Down
Loading