Long term Themes

Target Features by 1-Nov-2024

1. Fully functional DT Runner.
2. Feature complete CLI for admin tasks. Tasks to be taken care of are: installation, backup, update and uninstall. Python frameworks to be used for the CLI.
3. JWT-based authentication for lib and runner microservices when run in standalone mode.
4. Kasm Ubuntu-based docker container with five services: terminal, file server (cloudcmd), VSCode server, Jupyter Lab, and VNC. These are to be used as user workspaces.
5. Complete code coverage for tests. Feature tests at the top-level of the application.
6. Feature complete DT Lifecycle manager and DT Configuration Validator microservices.
7. Kubernetes based execution manager. The execution manager should be able to manage DTs running inside docker containers.
8. Keep support for relative paths in DTs. This means that all of user assets need to be cloned when a DT is executed in DevOps pipeline.
9. OAuth protection for all components in complete DTaaS installation. (done)
10. docker compose for both developers and users. (done)
11. Ten examples of different application domains. (done)

ML Workspace

• Rebuild the docker image using the latest forks so that
  • the library are updated
  • use the slim version of ML workspace images to see if the problems are less
  • to only include Terminal, VS Code, Jupyter Lab, Jupyter Notebook (with required libraries, kernels reinstalled)
• Problems seen so far (only with ml-workspace but not with ml-workspace-minimal):
  • the docker container loses network connectivity after three days (this is not the case on vagrant machines)
  • the terminal tool doesn't seem to have the ability to refresh itself. If there is an issue, the only solution is to close and reopen the terminal from "open tools" drop down of notebook
  • terminal app does not show at all after some time: terminal always comes if it is open from drop-down menu of Jupyter Notebook, but not as a direct link.
• ml-workspace-minimal only slows down after ten days or so.
  • The completed processes within docker containers become zombies (defunct state). This is because of the absence of init service within docker containers which usually cleans up the zombie process.
• Tools that must run in the new container are: Prosys OPC-UA server, GNU Octave 8.4

Gitlab

• The logout on gitlab does not effect the logged in status on the react app. Gitlab does not have the capability to make a webhook call to oauth end points for logging them out.
• Two concurrent sessions at different instances of DTaaS do not work properly with OAuth. The OAuth for the instance launched next takes the OAuth configuration from the first application.

Security

• Secure folders for all user folders. Generate security keys and use them to encrypt user folders on the server. Only users with their keys can see the files but no one else can.
• Provide SSH access to user workspaces. Users can complete the development on their desktops and then push assets to user workspace (use TCP proxy servers - HA Proxy or Traefik)

1. Use port access feature of ml-workspace to provide access to different DT Runners running in user workspace.

Documentation

• The markdown to html is good, but not markdown to html to pdf. The mathematical expressions are missing from the pdf documentation. The notices/warnings/tips etc. are not rendered correctly as well.

Other priorities

• Integrate Thingsboard and thingsboard-gateway into the DTaaS. Prefer these two over Eclipse Ditto / Hono.