Skip to content

Commit

Permalink
Add dev env scheme & extras (#13)
Browse files Browse the repository at this point in the history
* add dev env scheme

* extract csheme to separate file

* fix scheme

* fix image

* scale scheme

* add short description

* add keywords

* add folders

* underscores fix

* fix image

* fix image

* fix image

* fix image

* add abstracts

* add more to directory list

* dix underscore

* break page
  • Loading branch information
avan1235 authored Jun 5, 2021
1 parent acf123a commit f6ddaea
Show file tree
Hide file tree
Showing 9 changed files with 992 additions and 12 deletions.
8 changes: 8 additions & 0 deletions chapters/dev-env.tex
Original file line number Diff line number Diff line change
Expand Up @@ -107,3 +107,11 @@ \section{Development environment in integration tests}
environment. Additionally, the unit tests which don’t use the integrations’ environment just run scala
tests code without touching anything from the environment definition so there is an easy way to run them
and verify some specified parts of code without the process of working with any Docker containers.

\section{Development environment scheme}

The overview scheme of described development environment is presented below. It shows the scheme of
working with models via Nussknacker's frontend down to models listing and scoring models instances.

\newpage
\scalebox{0.9}{\input{chapters/extras/dev-env-scheme.tex}}
31 changes: 28 additions & 3 deletions chapters/disc-contents.tex
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ \chapter{Disc contents}

Attached CD contains snapshot of the repository.
Its directory structure is presented below with short descriptions of selected nodes.
\newpage

{
\newcommand{\node}[1]{
Expand All @@ -13,7 +14,6 @@ \chapter{Disc contents}
}

\vspace{1em}
\todo[inline]{Revisit in June 2021. Filler content for now.}
\begin{forest}
for tree={
grow'=0,
Expand All @@ -36,10 +36,35 @@ \chapter{Disc contents}
}
[{}
[\nodewithdesc{.github}{GitHub automation definitions}]
[\node{dev-environment}
[\nodewithdesc{docker-compose.yaml}{Definition of the development environment}]
[\nodewithdesc{dev-environment}{Development environment configuration and specifications}
[\node{README.md}]
[\node{clear\_environment.sh}]
[\node{create\_environment.sh}]
[\nodewithdesc{docker-compose-env.yaml}{Configuration of Nussknacker environment}]
[\nodewithdesc{docker-compose-h2o.yaml}{Configuration of H2O environment}]
[\nodewithdesc{docker-compose-mlflow.yaml}{Configuration of MLflow environment}]
[\nodewithdesc{docker-compose-pmml.yaml}{Configuration of PMML environment}]
]
[\nodewithdesc{docs}{Sources of documentation pages}]
[\nodewithdesc{prinz}{Prinz API specification}]
[\nodewithdesc{prinz\_h2o}{Implementation of H2O integration}]
[\nodewithdesc{prinz\_mlflow}{Implementation of MLflow integration}]
[\nodewithdesc{prinz\_pmml}{Implementation of PMML integration}]
[\nodewithdesc{prinz\_proxy}{Implementation of models proxying}]
[\nodewithdesc{prinz\_sample}{Example Prinz library usage}]
[\nodewithdesc{priz\_util}{Utility HTTP client}]
[\node{project}
[\node{build.properties}]
[\node{plugins.sbt}]
[\node{scalastyle\_config.xml}]
[\node{scalastyle\_test\_config.xml}]
]
[\nodewithdesc{.env}{Environment constants defitnition}]
[\node{.gitignore}]
[\node{LICENSE}]
[\node{README.md}]
[\nodewithdesc{build.sbt}{Scala project definition with dependencies}]
[\nodewithdesc{sbtwrapper}{sbt standalone compile script}]
]
\end{forest}
}
929 changes: 929 additions & 0 deletions chapters/extras/dev-env-scheme.tex

Large diffs are not rendered by default.

4 changes: 0 additions & 4 deletions chapters/introduction.tex

This file was deleted.

2 changes: 1 addition & 1 deletion thesis.tex
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@
\usepackage{forest}
\usepackage[unicode]{hyperref}
\usepackage{todonotes}
\usepackage{tikz}
% End of custom imports

% Custom definitions
Expand Down Expand Up @@ -61,7 +62,6 @@
%\listoftables

% Chapters
\input{chapters/introduction.tex}
\input{chapters/intro-nussknacker-prinz.tex}
\input{chapters/architecture-overview.tex}
\input{chapters/mlflow-integration.tex}
Expand Down
11 changes: 10 additions & 1 deletion title-page/abstract-en.tex
Original file line number Diff line number Diff line change
@@ -1 +1,10 @@
This is the abstract.
In the thesis we present a new implementation of Nussknacker's enricher named Prinz that is responsible for
Machine Learning models management and scoring. We describe the source of need for such implementation and
the background behind the Event Stream Processing approach. We show our approach to the process of developing
the open source library with API specification and our own API implementations including MLflow models
registry, PMML standard and H2O models registry. We describe the process towards managing open source project,
developers' environment management for the process of development and the decisions made to create our
implementation of Nussknacker’s enricher. Additionally, we present our thoughts on the topic of each integration,
its development process and the final results of API implementation which includes additional assumptions made
during the deployment of integration. As a final result of our work with the project, we provide working
library sources with readable examples of its deployment to the Nussknacker environment.
13 changes: 12 additions & 1 deletion title-page/abstract-pl.tex
Original file line number Diff line number Diff line change
@@ -1 +1,12 @@
To jest abstrakt.
Prezentujemy implementację Prinza - rozszerzenia do oprogramowania Nussknacker przeznaczonego
do zarządzania i ewaluacji modeli uczenia maszynowego w tymże środowisku. W pracy opisujemy
potrzebę powstania naszej biblioteki i tematykę związaną z przetwarzaniem dużej liczby danych
zdarzeniowych. Przedstawiamy nasz proces tworzenia biblioteki open source, włączając w to
specyfikację API projektu oraz jego implementacje w postaci integracji z MLflow, standardem
PMML oraz H2O. Ponadto opisujemy, jak przebiegał proces prowadzenia projektu open source,
przygotowania wygodnego środowiska dla programisty i decyzje podjęte podczas implementacji
poszczególnych integracji w ramach przyjętego interfejsu. Prezentujemy nasze konkluzje
dotyczące pracy z poszczególnymi integracjami i procesem projektowania ich wdrożenia do
naszego środowiska, jak i dodatkowe założenia powstałe podczas procesu implementacji.
Jako rezultat naszych prac dostarczamy źródła naszej biblioteki wraz z czytelnym przykładem
jej jej wykorzystania w środowisku Nussknackera.
3 changes: 2 additions & 1 deletion title-page/keywords-en.tex
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
keywords, go, here
Prinz, Nussknacker, Scala, Machine Learning, models scoring, ML integration, MLflow, PMML, JPMML, H2O, H2Oai,
Apache Flink, Event Stream Processing, ESP, ML models registry
3 changes: 2 additions & 1 deletion title-page/keywords-pl.tex
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
tu, będą, słowa, kluczowe
Prinz, Nussknacker, Scala, Machine Learning, ewaluacja modeli, integracja modeli ML, MLflow, PMML, JPMML, H2O, H2Oai,
Apache Flink, Event Stream Processing, ESP, rejestr modeli ML

0 comments on commit f6ddaea

Please sign in to comment.