Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Developer Documentation added #114

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .codeclimate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ plugins:
fixme:
enabled: true
markdownlint:
enabled: true
enabled: true
shellcheck:
enabled: true

Expand Down
4 changes: 4 additions & 0 deletions .mdl_style.rb
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
all
rule 'MD013', :ignore_code_blocks => true
rule 'MD013', :tables => false
rule 'MD046', :indented => true
6 changes: 1 addition & 5 deletions .mdlrc
Original file line number Diff line number Diff line change
@@ -1,5 +1 @@
git_recurse true
all
rule 'MD013', :ignore_code_blocks => true
rule 'MD013', :tables => false
rule 'MD046', :indented
style '.mdl_style.rb'
2 changes: 1 addition & 1 deletion deploy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,4 +114,4 @@ You can run this script multiple times until the installation is successful.

## Access the application

Now you should be able to access the DTaaS application at: <http:>_https://foo.com_</http:>
Now you should be able to access the DTaaS application at: _https://foo.com_
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bare URL used

71 changes: 71 additions & 0 deletions docs/developer/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# A Developer's Guide

This guide is to help developers get familiar with the project. Please see
developer-specific
[Slides](https://odin.cps.digit.au.dk/into-cps/dtaas/assets/DTaaS-overview.pdf),
[Video](https://odin.cps.digit.au.dk/into-cps/dtaas/assets/videos/DTaaS-overview.mkv),
and [Research paper](https://arxiv.org/abs/2305.07244).

## Operating Softwares

Ideally, developers should work on Ubuntu/Linux. Other operating systems are not supported inherently and may require additional steps.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line length


## Code Editing

Any popular code editors can be used to work on the project. VS Code, Sublime Text are a few examples.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line length


## Development Workflow

To manage collaboration by multiple developers on the software, a simple [Fork, Branch, PR](https://gun.io/news/2017/01/how-to-github-fork-branch-and-pull-request/) development workflow is in place. Each developer should follow these steps:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line length


1. Have an updated fork of the main repository on their account. This should also be added to codeclimate.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trailing spaces

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line length

1. Clone your personal fork onto your computer.
```
git clone https://github.com/<yourgithubusername>/DTaaS.git
```
1. Work on your issue/feature on your personal computer.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trailing spaces


1. Any updates/additions to the software should first be committed to your personal fork.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line length


Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trailing spaces

1. To check status of your changes:
```
git status
```
2. Add the changes to the commit:
```
git add --all *
```
3. Finally, commit these to the repository on your PC.
```
git commit -m <commit message>
```
1. Push any commits onto your fork of the repository
```
git push
```

1. Any issues that arise codeclimate should also be resolved.

1. Once changes are made, they should be tested on personal systems or the [integration server](https://github.com/INTO-CPS-Association/DTaaS/wiki/DTaaS-Integration-Server) .
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Line length


1. Once changes are verified, a PR should be made to the appropriate branch of the main repository.
1. Any issues raised in the PR review should be resolved.
1. Finally, the PR will be merged.

Remember that every commit should be meaningful and satisfies the requirements.

## Code Quality

Quality checks are performed by CodeClimate to ensure the best possible quality of code to add to our project.

While any new issues introduced in your code would be shown in the PR page itself, to address any specific issue, you can visit the Issues or Code section of the CodeClimate page.

It is highly recommended that any code you add does not introduce new quality issues. If they are introduced, they should be fixed immediately using the appropriate suggestions from CodeClimate, or in worst case, adding a ignore flag (To be used with caution).

## Testing

For information about testing and workflow related to that, please see the [testing page](testing/intro.md).

## Live Demo Server

A demo server is up and running at [https://sandbox.cps.digit.au.dk/](https://sandbox.cps.digit.au.dk/). Developers will need credentials to log in.
46 changes: 46 additions & 0 deletions docs/developer/servers/lib/lib-class.mmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
classDiagram
class FilesResolver {
-filesService: IFilesService
+listDirectory(path: string): Promise<Project>
+readFile(path: string): Promise<Project>
}

class FilesServiceFactory {
-configService: ConfigService
-gitlabFilesService: GitlabFilesService
-localFilesService: LocalFilesService
+create(): IFilesService
}

class GitlabFilesService {
-configService: ConfigService
-parseArguments(path: string): Promise<domain: string; parsedPath: string>
-sendRequest(query: string): Promise<Project>
-executeQuery(path: string, getQuery: QueryFunction): Promise<Project>
+listDirectory(path: string): Promise<Project>
+readFile(path: string): Promise<Project>
}

class LocalFilesService {
-configService: ConfigService
-getFileStats(fullPath: string, file: string): Promise<Project>
+listDirectory(path: string): Promise<Project>
+readFile(path: string): Promise<Project>
}

class ConfigService {
+get(propertyPath: string): any
}

class IFilesService{
listDirectory(path: string): Promise<Project>
readFile(path: string): Promise<Project>
}

IFilesService <|-- FilesResolver: uses
IFilesService <|.. GitlabFilesService: implements
IFilesService <|.. LocalFilesService: implements
IFilesService <|-- FilesServiceFactory: creates
ConfigService <|-- FilesServiceFactory: uses
ConfigService <|-- GitlabFilesService: uses
ConfigService <|-- LocalFilesService: uses
99 changes: 99 additions & 0 deletions docs/developer/servers/lib/lib-sequence.mmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
sequenceDiagram
actor Client
actor Traefik

box LightGreen RAMS
participant FR as FilesResolver
participant FSF as FilesServiceFactory
participant CS as ConfigService
participant IFS as IFilesService
participant LFS as LocalFilesService
participant GFS as GitlabFilesService
end

participant FS as Local File System DB
participant GAPI as GitLab API DB

Client ->> Traefik : HTTP request
Traefik ->> FR : GraphQL query
activate FR

FR ->> FSF : create()
activate FSF

FSF ->> CS : getConfiguration("MODE")
activate CS

CS -->> FSF : return configuration value
deactivate CS

alt MODE = Local
FSF ->> FR : return filesService (LFS)
deactivate FSF

FR ->> IFS : listDirectory(path) or readFile(path)
activate IFS

IFS ->> LFS : listDirectory(path) or readFile(path)
activate LFS

LFS ->> CS : getConfiguration("LOCAL_PATH")
activate CS

CS -->> LFS : return local path
deactivate CS

LFS ->> FS : Access filesystem
alt Filesystem error
FS -->> LFS : Filesystem error
LFS ->> LFS : Throw new InternalServerErrorException
LFS -->> IFS : Error
else Successful file operation
FS -->> LFS : Return filesystem data
LFS ->> IFS : return Promise<Project>
end
deactivate LFS
else MODE = GitLab
FSF ->> FR : return filesService (GFS)
%%deactivate FSF

FR ->> IFS : listDirectory(path) or readFile(path)
activate IFS

IFS ->> GFS : listDirectory(path) or readFile(path)
activate GFS

GFS ->> GFS : parseArguments(path)
GFS ->> GFS : executeQuery()

GFS ->> CS : getConfiguration("GITLAB_API_URL", "GITLAB_TOKEN")
activate CS

CS -->> GFS : return GitLab API URL and Token
deactivate CS

GFS ->> GAPI : sendRequest()
alt GitLab API error
GAPI -->> GFS : API error
GFS ->> GFS : Throw new Error("Invalid query")
GFS -->> IFS : Error
else Successful GitLab API operation
GAPI -->> GFS : Return API response
GFS ->> IFS : return Promise<Project>
end
deactivate GFS
end

alt Error thrown
IFS ->> FR : return Error
deactivate IFS
FR ->> Traefik : return Error
Traefik ->> Client : HTTP error response
else Successful operation
IFS ->> FR : return Promise<Project>
deactivate IFS
FR ->> Traefik : return Promise<Project>
Traefik ->> Client : HTTP response
end

deactivate FR
102 changes: 102 additions & 0 deletions docs/developer/system/architecture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
# System Overview

## User Requirements

The DTaaS software platform users expect a single platform
to support the complete DT lifecycle. To be more precise, the platform users expect the following features:

1. **Author** – create different assets of the DT on the
platform itself. This step requires use of some software
frameworks and tools whose sole purpose is to author
DT assets.
1. **Consolidate** – consolidate the list of available DT assets
and authoring tools so that user can navigate the library
of reusable assets. This functionality requires support
for discovery of available assets.
1. **Configure** – support selection and configuration of
DTs. This functionality also requires support for validation of a given configuration.
1. **Execute** – provision computing infrastructure on demand to support execution of a DT.
1. **Explore** – interact with a DT and explore the results
stored both inside and outside the platform. Exploration
may lead to analytical insights.
1. **Save** – save the state of a DT that’s already in the
execution phase. This functionality is required for ondemand saving and re-spawning of DTs.
1. **What-if analysis** – explore alternative scenarios to (i)
plan for an optimal next step, (ii) recalibrate new DT
assets, (iii) automated creation of new DTs or their
assets; these newly created DT assets may be used to
perform scientifically valid experiments.
1. **Share** – share a DT with other users of their organisation.

astitva1905 marked this conversation as resolved.
Show resolved Hide resolved
## System Architecture

![System architecture](architecture.png)

## System Components

The figure shows the system architecture of the the DTaaS software platform. The main domains of this architecture are:

1. [The Website](./client.md)

1. [The Gateway](https://github.com/INTO-CPS-Association/DTaaS/tree/feature/distributed-demo/servers/config/gateway#the-gateway-server) - This is the single point of entry for direct access to the platform services. The gateway is responsible for controlling user access to the microservice components.

1. [The Library Microservice (Reusable Assets)](./lib-ms.md)

### Microservices

1. **The security microservice** implements
role-based access control (RBAC) in the platform.
1. **The accounting microservice** is responsible for keeping track of the
platform, DT asset and infrastructure usage. Any licensing,
usage restrictions need to be enforced by the accounting
microservice. Accounting is a pre-requisite to commercialisation of the platform.
Due to significant use of external
infrastructure and resources via the platform, the accounting
microservice needs to interface with accounting systems of
the external services.
1. **The data microservice** is a frontend to all the databases
integrated into the platform. A time-series database and a
graph database are essential. These two databases store timeseries
data from PT, events on PT/DT, commands sent by
DT to PT. The PTs uses these databases even when their
respective DTs are not in the execute phase.
1. **The visualisation microservice** is again a frontend to
visualisation software that are natively supported inside the platform.
Any visualisation software running either on external
systems or on client browsers do not need to interact with
this microservice. They can directly use the data provided by
the data microservice.

## C4 Architectural Diagrams

The C4 architectural diagrams of the DTaaS software are presented here.

### Level 1

This Level 1 diagram only shows the users and the roles they play in the DTaaS software.

![Alt text](c4l1.png)

### Level 2

This Level 2 diagram shows the software containers of the DTaaS software.

![Detailed C4 architecture](c4l3.png)

A mapping of containers to system components is also available in the table.

| System Component | Container(s) |
|:---|:---|
| Gateway | Service Router |
| Unified Interface | React Webapplication |
| Reusable Assets | Library Microservice |
| Data | MQTT, InfluxDB, and RabbitMQ (not shown in the C4 Level 2 diagram) |
| Visualization | InfluxDB (not shown in the C4 Level 2 diagram) |
| DT Lifecycle | DT Lifecycle Manager and DT Configuration Validator |
| Security | Gitlab |
| Accounting | None |
| Execution Manager | Execution Manager |

## Disclaimer
Only the web client and library microservice components are functional at present.
Everything else is a work-in-progress.
Binary file added docs/developer/system/architecture.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/developer/system/c4l1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/developer/system/c4l3.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
9 changes: 9 additions & 0 deletions docs/developer/system/client.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Website

The [Website](https://github.com/INTO-CPS-Association/DTaaS/tree/feature/distributed-demo/client#readme) is how the end-users interact with the software platform.

This is the Client side (frontend) for Digital Twin as a Service (DTaaS) software. The software provides a React single page web application for the Digital Twin support platform.

## Client Architecture

astitva1905 marked this conversation as resolved.
Show resolved Hide resolved
![Client architecture](package-diagram.png)
Loading