-
Notifications
You must be signed in to change notification settings - Fork 57
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Developer Documentation added #114
Changes from all commits
3ffbeca
6ecfdac
f47ce99
da1968a
01c5d7b
0aa9f0f
35451e0
c007cbe
60e9478
3ce67a8
ee2c0ee
aad4679
5809a6b
636b362
214e438
980c87d
b36b647
ba31a95
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -20,7 +20,7 @@ plugins: | |
fixme: | ||
enabled: true | ||
markdownlint: | ||
enabled: true | ||
enabled: true | ||
shellcheck: | ||
enabled: true | ||
|
||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
all | ||
rule 'MD013', :ignore_code_blocks => true | ||
rule 'MD013', :tables => false | ||
rule 'MD046', :indented => true |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1 @@ | ||
git_recurse true | ||
all | ||
rule 'MD013', :ignore_code_blocks => true | ||
rule 'MD013', :tables => false | ||
rule 'MD046', :indented | ||
style '.mdl_style.rb' |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,71 @@ | ||
# A Developer's Guide | ||
|
||
This guide is to help developers get familiar with the project. Please see | ||
developer-specific | ||
[Slides](https://odin.cps.digit.au.dk/into-cps/dtaas/assets/DTaaS-overview.pdf), | ||
[Video](https://odin.cps.digit.au.dk/into-cps/dtaas/assets/videos/DTaaS-overview.mkv), | ||
and [Research paper](https://arxiv.org/abs/2305.07244). | ||
|
||
## Operating Softwares | ||
|
||
Ideally, developers should work on Ubuntu/Linux. Other operating systems are not supported inherently and may require additional steps. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Line length |
||
|
||
## Code Editing | ||
|
||
Any popular code editors can be used to work on the project. VS Code, Sublime Text are a few examples. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Line length |
||
|
||
## Development Workflow | ||
|
||
To manage collaboration by multiple developers on the software, a simple [Fork, Branch, PR](https://gun.io/news/2017/01/how-to-github-fork-branch-and-pull-request/) development workflow is in place. Each developer should follow these steps: | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Line length |
||
|
||
1. Have an updated fork of the main repository on their account. This should also be added to codeclimate. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Trailing spaces There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Line length |
||
1. Clone your personal fork onto your computer. | ||
``` | ||
git clone https://github.com/<yourgithubusername>/DTaaS.git | ||
``` | ||
1. Work on your issue/feature on your personal computer. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Trailing spaces |
||
|
||
1. Any updates/additions to the software should first be committed to your personal fork. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Line length |
||
|
||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Trailing spaces |
||
1. To check status of your changes: | ||
``` | ||
git status | ||
``` | ||
2. Add the changes to the commit: | ||
``` | ||
git add --all * | ||
``` | ||
3. Finally, commit these to the repository on your PC. | ||
``` | ||
git commit -m <commit message> | ||
``` | ||
1. Push any commits onto your fork of the repository | ||
``` | ||
git push | ||
``` | ||
|
||
1. Any issues that arise codeclimate should also be resolved. | ||
|
||
1. Once changes are made, they should be tested on personal systems or the [integration server](https://github.com/INTO-CPS-Association/DTaaS/wiki/DTaaS-Integration-Server) . | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Line length |
||
|
||
1. Once changes are verified, a PR should be made to the appropriate branch of the main repository. | ||
1. Any issues raised in the PR review should be resolved. | ||
1. Finally, the PR will be merged. | ||
|
||
Remember that every commit should be meaningful and satisfies the requirements. | ||
|
||
## Code Quality | ||
|
||
Quality checks are performed by CodeClimate to ensure the best possible quality of code to add to our project. | ||
|
||
While any new issues introduced in your code would be shown in the PR page itself, to address any specific issue, you can visit the Issues or Code section of the CodeClimate page. | ||
|
||
It is highly recommended that any code you add does not introduce new quality issues. If they are introduced, they should be fixed immediately using the appropriate suggestions from CodeClimate, or in worst case, adding a ignore flag (To be used with caution). | ||
|
||
## Testing | ||
|
||
For information about testing and workflow related to that, please see the [testing page](testing/intro.md). | ||
|
||
## Live Demo Server | ||
|
||
A demo server is up and running at [https://sandbox.cps.digit.au.dk/](https://sandbox.cps.digit.au.dk/). Developers will need credentials to log in. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
classDiagram | ||
class FilesResolver { | ||
-filesService: IFilesService | ||
+listDirectory(path: string): Promise<Project> | ||
+readFile(path: string): Promise<Project> | ||
} | ||
|
||
class FilesServiceFactory { | ||
-configService: ConfigService | ||
-gitlabFilesService: GitlabFilesService | ||
-localFilesService: LocalFilesService | ||
+create(): IFilesService | ||
} | ||
|
||
class GitlabFilesService { | ||
-configService: ConfigService | ||
-parseArguments(path: string): Promise<domain: string; parsedPath: string> | ||
-sendRequest(query: string): Promise<Project> | ||
-executeQuery(path: string, getQuery: QueryFunction): Promise<Project> | ||
+listDirectory(path: string): Promise<Project> | ||
+readFile(path: string): Promise<Project> | ||
} | ||
|
||
class LocalFilesService { | ||
-configService: ConfigService | ||
-getFileStats(fullPath: string, file: string): Promise<Project> | ||
+listDirectory(path: string): Promise<Project> | ||
+readFile(path: string): Promise<Project> | ||
} | ||
|
||
class ConfigService { | ||
+get(propertyPath: string): any | ||
} | ||
|
||
class IFilesService{ | ||
listDirectory(path: string): Promise<Project> | ||
readFile(path: string): Promise<Project> | ||
} | ||
|
||
IFilesService <|-- FilesResolver: uses | ||
IFilesService <|.. GitlabFilesService: implements | ||
IFilesService <|.. LocalFilesService: implements | ||
IFilesService <|-- FilesServiceFactory: creates | ||
ConfigService <|-- FilesServiceFactory: uses | ||
ConfigService <|-- GitlabFilesService: uses | ||
ConfigService <|-- LocalFilesService: uses |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,99 @@ | ||
sequenceDiagram | ||
actor Client | ||
actor Traefik | ||
|
||
box LightGreen RAMS | ||
participant FR as FilesResolver | ||
participant FSF as FilesServiceFactory | ||
participant CS as ConfigService | ||
participant IFS as IFilesService | ||
participant LFS as LocalFilesService | ||
participant GFS as GitlabFilesService | ||
end | ||
|
||
participant FS as Local File System DB | ||
participant GAPI as GitLab API DB | ||
|
||
Client ->> Traefik : HTTP request | ||
Traefik ->> FR : GraphQL query | ||
activate FR | ||
|
||
FR ->> FSF : create() | ||
activate FSF | ||
|
||
FSF ->> CS : getConfiguration("MODE") | ||
activate CS | ||
|
||
CS -->> FSF : return configuration value | ||
deactivate CS | ||
|
||
alt MODE = Local | ||
FSF ->> FR : return filesService (LFS) | ||
deactivate FSF | ||
|
||
FR ->> IFS : listDirectory(path) or readFile(path) | ||
activate IFS | ||
|
||
IFS ->> LFS : listDirectory(path) or readFile(path) | ||
activate LFS | ||
|
||
LFS ->> CS : getConfiguration("LOCAL_PATH") | ||
activate CS | ||
|
||
CS -->> LFS : return local path | ||
deactivate CS | ||
|
||
LFS ->> FS : Access filesystem | ||
alt Filesystem error | ||
FS -->> LFS : Filesystem error | ||
LFS ->> LFS : Throw new InternalServerErrorException | ||
LFS -->> IFS : Error | ||
else Successful file operation | ||
FS -->> LFS : Return filesystem data | ||
LFS ->> IFS : return Promise<Project> | ||
end | ||
deactivate LFS | ||
else MODE = GitLab | ||
FSF ->> FR : return filesService (GFS) | ||
%%deactivate FSF | ||
|
||
FR ->> IFS : listDirectory(path) or readFile(path) | ||
activate IFS | ||
|
||
IFS ->> GFS : listDirectory(path) or readFile(path) | ||
activate GFS | ||
|
||
GFS ->> GFS : parseArguments(path) | ||
GFS ->> GFS : executeQuery() | ||
|
||
GFS ->> CS : getConfiguration("GITLAB_API_URL", "GITLAB_TOKEN") | ||
activate CS | ||
|
||
CS -->> GFS : return GitLab API URL and Token | ||
deactivate CS | ||
|
||
GFS ->> GAPI : sendRequest() | ||
alt GitLab API error | ||
GAPI -->> GFS : API error | ||
GFS ->> GFS : Throw new Error("Invalid query") | ||
GFS -->> IFS : Error | ||
else Successful GitLab API operation | ||
GAPI -->> GFS : Return API response | ||
GFS ->> IFS : return Promise<Project> | ||
end | ||
deactivate GFS | ||
end | ||
|
||
alt Error thrown | ||
IFS ->> FR : return Error | ||
deactivate IFS | ||
FR ->> Traefik : return Error | ||
Traefik ->> Client : HTTP error response | ||
else Successful operation | ||
IFS ->> FR : return Promise<Project> | ||
deactivate IFS | ||
FR ->> Traefik : return Promise<Project> | ||
Traefik ->> Client : HTTP response | ||
end | ||
|
||
deactivate FR |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,102 @@ | ||
# System Overview | ||
|
||
## User Requirements | ||
|
||
The DTaaS software platform users expect a single platform | ||
to support the complete DT lifecycle. To be more precise, the platform users expect the following features: | ||
|
||
1. **Author** – create different assets of the DT on the | ||
platform itself. This step requires use of some software | ||
frameworks and tools whose sole purpose is to author | ||
DT assets. | ||
1. **Consolidate** – consolidate the list of available DT assets | ||
and authoring tools so that user can navigate the library | ||
of reusable assets. This functionality requires support | ||
for discovery of available assets. | ||
1. **Configure** – support selection and configuration of | ||
DTs. This functionality also requires support for validation of a given configuration. | ||
1. **Execute** – provision computing infrastructure on demand to support execution of a DT. | ||
1. **Explore** – interact with a DT and explore the results | ||
stored both inside and outside the platform. Exploration | ||
may lead to analytical insights. | ||
1. **Save** – save the state of a DT that’s already in the | ||
execution phase. This functionality is required for ondemand saving and re-spawning of DTs. | ||
1. **What-if analysis** – explore alternative scenarios to (i) | ||
plan for an optimal next step, (ii) recalibrate new DT | ||
assets, (iii) automated creation of new DTs or their | ||
assets; these newly created DT assets may be used to | ||
perform scientifically valid experiments. | ||
1. **Share** – share a DT with other users of their organisation. | ||
|
||
astitva1905 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
## System Architecture | ||
|
||
![System architecture](architecture.png) | ||
|
||
## System Components | ||
|
||
The figure shows the system architecture of the the DTaaS software platform. The main domains of this architecture are: | ||
|
||
1. [The Website](./client.md) | ||
|
||
1. [The Gateway](https://github.com/INTO-CPS-Association/DTaaS/tree/feature/distributed-demo/servers/config/gateway#the-gateway-server) - This is the single point of entry for direct access to the platform services. The gateway is responsible for controlling user access to the microservice components. | ||
|
||
1. [The Library Microservice (Reusable Assets)](./lib-ms.md) | ||
|
||
### Microservices | ||
|
||
1. **The security microservice** implements | ||
role-based access control (RBAC) in the platform. | ||
1. **The accounting microservice** is responsible for keeping track of the | ||
platform, DT asset and infrastructure usage. Any licensing, | ||
usage restrictions need to be enforced by the accounting | ||
microservice. Accounting is a pre-requisite to commercialisation of the platform. | ||
Due to significant use of external | ||
infrastructure and resources via the platform, the accounting | ||
microservice needs to interface with accounting systems of | ||
the external services. | ||
1. **The data microservice** is a frontend to all the databases | ||
integrated into the platform. A time-series database and a | ||
graph database are essential. These two databases store timeseries | ||
data from PT, events on PT/DT, commands sent by | ||
DT to PT. The PTs uses these databases even when their | ||
respective DTs are not in the execute phase. | ||
1. **The visualisation microservice** is again a frontend to | ||
visualisation software that are natively supported inside the platform. | ||
Any visualisation software running either on external | ||
systems or on client browsers do not need to interact with | ||
this microservice. They can directly use the data provided by | ||
the data microservice. | ||
|
||
## C4 Architectural Diagrams | ||
|
||
The C4 architectural diagrams of the DTaaS software are presented here. | ||
|
||
### Level 1 | ||
|
||
This Level 1 diagram only shows the users and the roles they play in the DTaaS software. | ||
|
||
![Alt text](c4l1.png) | ||
|
||
### Level 2 | ||
|
||
This Level 2 diagram shows the software containers of the DTaaS software. | ||
|
||
![Detailed C4 architecture](c4l3.png) | ||
|
||
A mapping of containers to system components is also available in the table. | ||
|
||
| System Component | Container(s) | | ||
|:---|:---| | ||
| Gateway | Service Router | | ||
| Unified Interface | React Webapplication | | ||
| Reusable Assets | Library Microservice | | ||
| Data | MQTT, InfluxDB, and RabbitMQ (not shown in the C4 Level 2 diagram) | | ||
| Visualization | InfluxDB (not shown in the C4 Level 2 diagram) | | ||
| DT Lifecycle | DT Lifecycle Manager and DT Configuration Validator | | ||
| Security | Gitlab | | ||
| Accounting | None | | ||
| Execution Manager | Execution Manager | | ||
|
||
## Disclaimer | ||
Only the web client and library microservice components are functional at present. | ||
Everything else is a work-in-progress. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
# Website | ||
|
||
The [Website](https://github.com/INTO-CPS-Association/DTaaS/tree/feature/distributed-demo/client#readme) is how the end-users interact with the software platform. | ||
|
||
This is the Client side (frontend) for Digital Twin as a Service (DTaaS) software. The software provides a React single page web application for the Digital Twin support platform. | ||
|
||
## Client Architecture | ||
|
||
astitva1905 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
![Client architecture](package-diagram.png) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bare URL used