Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: architecture documentation #1732

Merged
merged 9 commits into from
Aug 1, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@ List any dependencies that are required for this change.

Fixes # (issue)

### Quick checks:
### Quick checks

- [ ] I have followed the [Code Guidelines](https://github.com/ConduitIO/conduit/blob/main/docs/code_guidelines.md).
- [ ] There is no other [pull request](https://github.com/ConduitIO/conduit/pulls) for the same update/change.
- [ ] I have written unit tests.
- [ ] I have made sure that the PR is of reasonable size and can be easily reviewed.
- [ ] I have made sure that the PR is of reasonable size and can be easily reviewed.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -279,7 +279,7 @@ For more information about the UI refer to the [Readme](ui/README.md) in `/ui`.
## Documentation

To learn more about how to use Conduit
visit [conduit.io/docs](https://conduit.io/docs).
visit [Conduit.io/docs](https://conduit.io/docs).

If you are interested in internals of Conduit we have prepared some technical
documentation:
Expand Down
35 changes: 18 additions & 17 deletions docs/package_structure.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,29 +5,30 @@
things, it should not include anything else:
1. Read the configuration (from a file, the environment or arguments).
2. Instantiate, wire up and run internal services.
3. Listen for signals (i.e. SIGTERM, SIGINT) and forward them to internal services to ensure a graceful shutdown (e.g. via a closed context).
3. Listen for signals (i.e. SIGTERM, SIGINT) and forward them to internal services to ensure a graceful shutdown
(e.g. via a closed context).
- `conduit` - The entrypoint for the main Conduit executable.
- `pkg` - The internal libraries and services that Conduit runs.
- `conduit` - Defines the main runtime that ties all Conduit layers together.
- `connector` - Code regarding connectors, including connector store, connector service, connector configurations
- `conduit` - Defines the main runtime that ties all Conduit layers together.
- `connector` - Code regarding connectors, including connector store, connector service, connector configurations
and running instances.
- `foundation` - Foundation contains reusable code. Should not contain any business logic. A few honorable mentions:
- `assert` - Exposes common assertions for testing.
- `cerrors` - Exposes error creation and wrapping functionality. This is the only package for errors used in Conduit.
- `database` - Exposes functionality for storing values.
- `log` - Exposes a logger. This is the logger used throughout Conduit.
- `metrics` - Exposes functionality for gathering and exposing metrics.
- `orchestrator` - Code regarding the orchestration layer.
- `pipeline` - Code regarding pipelines, including pipeline store, pipeline service, running pipeline instances.
- `plugin` - Currently contains all logic related to plugins as well as the plugins themselves. In the future a lot of
- `foundation` - Foundation contains reusable code. Should not contain any business logic. A few honorable mentions:
- `assert` - Exposes common assertions for testing.
- `cerrors` - Exposes error creation and wrapping functionality. This is the only package for errors used in Conduit.
- `database` - Exposes functionality for storing values.
- `log` - Exposes a logger. This is the logger used throughout Conduit.
- `metrics` - Exposes functionality for gathering and exposing metrics.
- `orchestrator` - Code regarding the orchestration layer.
- `pipeline` - Code regarding pipelines, including pipeline store, pipeline service, running pipeline instances.
- `plugin` - Currently contains all logic related to plugins as well as the plugins themselves. In the future a lot of
this code will be extracted into separate repositories, what will be left is a plugin service that manages built-in
and external plugins.
- `processor` - Provides the types for processing a `Record`. A common abbreviation for transforms is `txf`.
- `transform/txfbuiltin` - Contains built-in transforms.
- `transform/txfjs` - Provides the functionality for implementing a transform in JavaScript.
- `record` - Everything regarding a `Record`, that is the central entity that is pushed through a Conduit pipeline.
- `processor` - Provides the types for processing a `Record`. A common abbreviation for transforms is `txf`.
- `transform/txfbuiltin` - Contains built-in transforms.
- `transform/txfjs` - Provides the functionality for implementing a transform in JavaScript.
- `record` - Everything regarding a `Record`, that is the central entity that is pushed through a Conduit pipeline.
hariso marked this conversation as resolved.
Show resolved Hide resolved
This includes a record `Schema`.
- `web` - Everything related to Conduit APIs or hosted pages like the UI or Swagger.
- `web` - Everything related to Conduit APIs or hosted pages like the UI or Swagger.

Other folders that don't contain Go code:

Expand Down
4 changes: 2 additions & 2 deletions pkg/web/openapi/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@

This directory contains an abbreviated copy of the following repository:

* `swagger-ui` - https://github.com/swagger-api/swagger-ui 8e6824cb452ae4b268f45f5203575194217c6653 (LICENSE, dist/)
- `swagger-ui` - <https://github.com/swagger-api/swagger-ui> 8e6824cb452ae4b268f45f5203575194217c6653 (LICENSE, dist/)

The `swagger-ui` directory contains HTML, Javascript, and CSS assets that dynamically generate Swagger documentation
The `swagger-ui` directory contains HTML, JavaScript, and CSS assets that dynamically generate Swagger documentation
from a Swagger-compliant API definition in [api.swagger.json](./swagger-ui/api/v1/api.swagger.json) file.
That file is auto-generated by running `make proto` in the root of this repository. The static assets are copied
from [this dist folder](https://github.com/swagger-api/swagger-ui/tree/master/dist) of the swagger-ui project. After
Expand Down