refactor: architecture documentation

.github/pull_request_template.md

@@ -6,9 +6,9 @@ List any dependencies that are required for this change.
 
 Fixes # (issue)
 
-### Quick checks:
+### Quick checks
 
 - [ ] I have followed the [Code Guidelines](https://github.com/ConduitIO/conduit/blob/main/docs/code_guidelines.md).
 - [ ] There is no other [pull request](https://github.com/ConduitIO/conduit/pulls) for the same update/change.
 - [ ] I have written unit tests.
-- [ ] I have made sure that the PR is of reasonable size and can be easily reviewed.
+- [ ] I have made sure that the PR is of reasonable size and can be easily reviewed.

README.md

@@ -114,7 +114,7 @@ Download the right `.deb` file for your machine architecture from the
 [latest release](https://github.com/conduitio/conduit/releases/latest), then run:
 
 ```sh
-dpkg -i conduit_0.8.0_Linux_x86_64.deb
+dpkg -i conduit_0.10.0_Linux_x86_64.deb
 ```
 
 ### RPM
@@ -123,7 +123,7 @@ Download the right `.rpm` file for your machine architecture from the
 [latest release](https://github.com/conduitio/conduit/releases/latest), then run:
 
 ```sh
-rpm -i conduit_0.8.0_Linux_x86_64.rpm
+rpm -i conduit_0.10.0_Linux_x86_64.rpm
 ```
 
 ### Build from source
@@ -279,7 +279,7 @@ For more information about the UI refer to the [Readme](ui/README.md) in `/ui`.
 ## Documentation
 
 To learn more about how to use Conduit
-visit [docs.Conduit.io](https://docs.conduit.io).
+visit [Conduit.io/docs](https://conduit.io/docs).
 
 If you are interested in internals of Conduit we have prepared some technical
 documentation:
@@ -290,10 +290,12 @@ documentation:
   explains how you can define pipelines using YAML files.
 - [Processors](https://conduit.io/docs/processors/getting-started) contains examples and more information about
   Conduit processors.
-- [Conduit Architecture](docs/architecture.md)
+- [Conduit Architecture](https://conduit.io/docs/getting-started/architecture)
   will give you a high-level overview of Conduit.
 - [Conduit Metrics](docs/metrics.md)
   provides more information about how Conduit exposes metrics.
+- [Conduit Package structure](docs/package_structure.md)
+  provides more information about the different packages in Conduit.
 
 ## Contributing
 
@@ -314,8 +316,7 @@ We also value contributions in form of pull requests. When opening a PR please
 ensure:
 
 - You have followed
-  the [Code Guidelines](https://github.com/ConduitIO/conduit/blob/main/docs/code_guidelines.md)
-  .
+  the [Code Guidelines](https://github.com/ConduitIO/conduit/blob/main/docs/code_guidelines.md).
 - There is no other [pull request](https://github.com/ConduitIO/conduit/pulls)
   for the same update/change.
 - You have written unit tests.

docs/package_structure.md

@@ -0,0 +1,38 @@
+# Package structure
+
+- `cmd` - Contains main applications. The directory name for each application should match the name of the executable
+  (e.g. `cmd/conduit` produces an executable called `conduit`). It is the responsibility of main applications to do 3
+  things, it should not include anything else:
+  1. Read the configuration (from a file, the environment or arguments).
+  2. Instantiate, wire up and run internal services.
+  3. Listen for signals (i.e. SIGTERM, SIGINT) and forward them to internal services to ensure a graceful shutdown
+(e.g. via a closed context).
+  - `conduit` - The entrypoint for the main Conduit executable.
+- `pkg` - The internal libraries and services that Conduit runs.
+  - `conduit` - Defines the main runtime that ties all Conduit layers together.
+  - `connector` - Code regarding connectors, including connector store, connector service, connector configurations
+      and running instances.
+  - `foundation` - Foundation contains reusable code. Should not contain any business logic. A few honorable mentions:
+    - `assert` - Exposes common assertions for testing.
+    - `cerrors` - Exposes error creation and wrapping functionality. This is the only package for errors used in Conduit.
+    - `database` - Exposes functionality for storing values.
+    - `log` - Exposes a logger. This is the logger used throughout Conduit.
+    - `metrics` - Exposes functionality for gathering and exposing metrics.
+  - `orchestrator` - Code regarding the orchestration layer.
+  - `pipeline` - Code regarding pipelines, including pipeline store, pipeline service, running pipeline instances.
+  - `plugin` - Currently contains all logic related to plugins as well as the plugins themselves. In the future a lot of
+      this code will be extracted into separate repositories, what will be left is a plugin service that manages built-in
+      and external plugins.
+  - `processor` - Provides the types for processing a `Record`. A common abbreviation for transforms is `txf`.
+    - `transform/txfbuiltin` - Contains built-in transforms.
+    - `transform/txfjs` - Provides the functionality for implementing a transform in JavaScript.
+  - `record` - Everything regarding a `Record`, that is the central entity that is pushed through a Conduit pipeline.
+      This includes a record `Schema`.
+  - `web` - Everything related to Conduit APIs or hosted pages like the UI or Swagger.
+
+Other folders that don't contain Go code:
+
+- `docs` - Documentation regarding Conduit.
+- `proto` - Protobuf files (e.g. gRPC API definition).
+- `test` - Contains configurations needed for integration tests.
+- `ui` - A subproject containing the web UI for Conduit.

pkg/web/openapi/README.md

@@ -2,9 +2,9 @@
 
 This directory contains an abbreviated copy of the following repository:
 
-* `swagger-ui` - https://github.com/swagger-api/swagger-ui 8e6824cb452ae4b268f45f5203575194217c6653 (LICENSE, dist/)
+- `swagger-ui` - <https://github.com/swagger-api/swagger-ui> 8e6824cb452ae4b268f45f5203575194217c6653 (LICENSE, dist/)
 
-The `swagger-ui` directory contains HTML, Javascript, and CSS assets that dynamically generate Swagger documentation
+The `swagger-ui` directory contains HTML, JavaScript, and CSS assets that dynamically generate Swagger documentation
 from a Swagger-compliant API definition in [api.swagger.json](./swagger-ui/api/v1/api.swagger.json) file.
 That file is auto-generated by running `make proto` in the root of this repository. The static assets are copied
 from [this dist folder](https://github.com/swagger-api/swagger-ui/tree/master/dist) of the swagger-ui project. After