[refactor] #4161: rewrite config, _minimally_ (#4239)

* [refactor]: wip Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: update structure - exclude genesis block loading from config - construct `KeyPair` on `iroha` completion - use full field names in `complete()`s Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: apply lints Signed-off-by: Dmitry Balashov <[email protected]> * [feat]: include more ENV vars, refactor Signed-off-by: Dmitry Balashov <[email protected]> * [feat]: impl merging - `UserField` wrap instead of `Option` - move trusted peers uniqueness check Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: update Kagami - Remove `config` subcommand - Update default `genesis` building Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: restructure code - move generic tools to `iroha_config_base` - remove `iroha_client_config` crate - define client config in `iroha_client::config` - add client config sample TOML Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: update `iroha_logger` Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: update `iroha_telemetry` - update usage of `iroha_config` - use `Duration` in `RetryPeriod` Signed-off-by: Dmitry Balashov <[email protected]> * [fix]: fix util macro Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: re-struct config, update `iroha_core` - split "user-layer" and "actual" config modules - update logger, telemetry, and kagami (genesis) Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: update more crates - `iroha_client` - `iroha_torii` - move Torii `uri` to `iroha_torii_const` Signed-off-by: Dmitry Balashov <[email protected]> * [feat]: compile `iroha`! Signed-off-by: Dmitry Balashov <[email protected]> * [feat]: compile `iroha_client`! Signed-off-by: Dmitry Balashov <[email protected]> * [feat]: compile `iroha_client_cli`! Signed-off-by: Dmitry Balashov <[email protected]> * [feat]: compile everything *_* Signed-off-by: Dmitry Balashov <[email protected]> * [fix]: chores Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: pass tests - update signature of `PeerId::new` (avoid extra clone) - fix `UserField::set` - fix deps of `iroha_config_base` - resolve runtime todos in `iroha_config` - update `iroha_swarm` generation output - some other refactoring Signed-off-by: Dmitry Balashov <[email protected]> * [feat]: implement `extends` Signed-off-by: Dmitry Balashov <[email protected]> * [chore]: update default snapshot storage Signed-off-by: Dmitry Balashov <[email protected]> * [fix]: update after rebase Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: update just everything - `test_env.py`: use toml configs - remove other channel configs - put example configs into `config_samples` dir - put docker setup into `config_samples/swarm` dir - pytests: split settings for CLI and config paths - make test env runnable Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: refactored config boilerplate into dedicated modules Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: apply some lints Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: update config naming in `[queue]` also run tests with all features enabled Signed-off-by: Dmitry Balashov <[email protected]> * [feat]: implement `Config::load` shorthand Signed-off-by: Dmitry Balashov <[email protected]> * [docs]: add comment Signed-off-by: Dmitry Balashov <[email protected]> * [ci]: fix typo Signed-off-by: Dmitry Balashov <[email protected]> * [ci]: install `tomli_w` via pacman Signed-off-by: Dmitry Balashov <[email protected]> * [chore]: fix whitespace Signed-off-by: Dmitry Balashov <[email protected]> * [revert]: update the snapshot store path Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: fix pytests - do not mutate client config from tests, override via env instead - add `TORII_URL` env var to client config Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: apply suggestions from code review Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: curl up `SumeragiStartArgs` Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: refine telemetry - simplify `regular_telemetry` to just `telemetry` - clearer config passing Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: use `Infallible` Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: simplify client config - `nonce` instead of `add_nonce` - move `[api]` to root Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: update `--config` arg - remove default value - remove `IROHA_CONFIG` env Signed-off-by: Dmitry Balashov <[email protected]> * [docs]: update `read_config_and_genesis` docs Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: chore Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: just `idle_time`, without `query_` Signed-off-by: Dmitry Balashov <[email protected]> * [revert]: use `ident_length_limits` in _actual_ config Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: use `Config`, not vague `Root` Signed-off-by: Dmitry Balashov <[email protected]> * [chore]: remove comment Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: use `capacity` term in Queue Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: `set_creation_time_ms`, use `*` instead of `mul` Signed-off-by: Dmitry Balashov <[email protected]> * [fix]: remove dead code Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: update `--config` args - use `PathBuf` again - remove default value at Client CLI Signed-off-by: Dmitry Balashov <[email protected]> * [chore]: cleaning Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: use `strum` in place of `parse_display` Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: no unsafe code any more Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: docs & refinements - document code in: - `iroha_config_base` - `iroha_config` - `iroha_client::config` - refactor `iroha_config_base` APIs - move `ExtendsPaths` into `iroha_config_base` - remove `[iroha]` user config section - move `chain_id` and key pair to the root - move `p2p_address` to `network.address` - rename `user_layer` modules to `user` - add `_bytes` suffix for relevant fields, put a TODO to add a newtype for it Signed-off-by: Dmitry Balashov <[email protected]> * [docs]: fill `peer.example.toml` Signed-off-by: Dmitry Balashov <[email protected]> * [chore]: re-export `ConfigurationDTO` from client Signed-off-by: Dmitry Balashov <[email protected]> * [test]: fix them Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: apply lints Signed-off-by: Dmitry Balashov <[email protected]> * [test]: fix pytests Signed-off-by: Dmitry Balashov <[email protected]> * [misc]: re-arrange sample configurations - use `configs` dir again - name example configs as _templates_ Signed-off-by: Dmitry Balashov <[email protected]> * [docs]: update README Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: remove `parse-display` from deps Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: remove extra `iroha_config_base` exposure from the client Signed-off-by: Dmitry Balashov <[email protected]> * [fix]: import `Deserialize` in the macro Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: use `PrivateKey::into_raw` instead Signed-off-by: Dmitry Balashov <[email protected]> * [ci]: use `--break-system-packages` `pip` flag Signed-off-by: Dmitry Balashov <[email protected]> * [fix]: regenerate swarms Signed-off-by: Dmitry Balashov <[email protected]> * [ci]: fix pytests workflow Signed-off-by: Dmitry Balashov <[email protected]> * [fix]: remove `PrivateKey::payload()` access Signed-off-by: Dmitry Balashov <[email protected]> * [docs]: fix "unable to validate" doc Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: accept `Duration` for `set_creation_time` Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: `PrivateKey::to_raw` Signed-off-by: Dmitry Balashov <[email protected]> * [fix]: do not extend trusted peers in config Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: hide user view from `iroha_client::config` Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: rename `*configuration` to `config` everywhere﹡ ﹡except crypto Signed-off-by: Dmitry Balashov <[email protected]> * [fix]: update default wasm fuel limit Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: change docs and methods of `WebLogin` Signed-off-by: Dmitry Balashov <[email protected]> * [fix]: trusted peers and config tests Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: rename `telemetry.dev.out_file` Signed-off-by: Dmitry Balashov <[email protected]> * [docs]: update `GenesisNetwork::new` errors Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: use `serde_with` Signed-off-by: Dmitry Balashov <[email protected]> * [misc]: move `nonzero_ext` to workspace level Signed-off-by: Dmitry Balashov <[email protected]> * [fix]: make `--config` optional It is still possible to set full config via env (e.g. `iroha_swarm` case) Signed-off-by: Dmitry Balashov <[email protected]> * [chore]: remove "regular" from telemetry re-export Signed-off-by: Dmitry Balashov <[email protected]> * [chore]: rename imports in `wasm.rs` Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: lints Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: use `serde_with` in swarm Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: rename parameters, cover full config in tests, fix bugs Signed-off-by: Dmitry Balashov <[email protected]> * [test]: cover absolute paths Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: update client configs, cover full in tests Signed-off-by: Dmitry Balashov <[email protected]> * [refactor]: lints Signed-off-by: Dmitry Balashov <[email protected]> * [fix]: also rename ENVs Signed-off-by: Dmitry Balashov <[email protected]> * [build]: add notes to Dockerfile Signed-off-by: Dmitry Balashov <[email protected]> * [chore]: update style in `iroha_test_config.toml` Signed-off-by: Dmitry Balashov <[email protected]> * [test]: fix old params in `test_env.py` Signed-off-by: Dmitry Balashov <[email protected]> * [test]: add a note to `panic_on_invalid_genesis.sh` Signed-off-by: Dmitry Balashov <[email protected]> --------- Signed-off-by: Dmitry Balashov <[email protected]>
hyperledger-iroha · Feb 16, 2024 · 9644767 · 9644767
1 parent 0f83d26
commit 9644767
Show file tree

Hide file tree

Showing 175 changed files with 5,189 additions and 5,785 deletions.
diff --git a/.github/workflows/iroha2-dev-pr.yml b/.github/workflows/iroha2-dev-pr.yml
@@ -28,12 +28,6 @@ jobs:
       - name: Check genesis.json
         if: always()
         run: ./scripts/tests/consistency.sh genesis
-      - name: Check client/config.json
-        if: always()
-        run: ./scripts/tests/consistency.sh client
-      - name: Check peer/config.json
-        if: always()
-        run: ./scripts/tests/consistency.sh peer
       - name: Check schema.json
         if: always()
         run: ./scripts/tests/consistency.sh schema
@@ -144,11 +138,10 @@ jobs:
       - uses: Swatinem/rust-cache@v2
       - name: Build binaries
         run: |
-          cargo build --bin iroha_client_cli
-          cargo build --bin kagami
-          cargo build --bin iroha
+          cargo build -p iroha_client_cli -p kagami -p iroha
       - name: Setup test Iroha 2 environment on the bare metal
         run: |
+          pip3 install -r scripts/requirements.txt --no-input --break-system-packages
           ./scripts/test_env.py setup
       - name: Mark binaries as executable
         run: |
@@ -159,6 +152,10 @@ jobs:
           poetry install
       - name: Run client cli tests
         working-directory: client_cli/pytests
+        env:
+          # prepared by `test_env.py`
+          CLIENT_CLI_BINARY: ../../test/iroha_client_cli
+          CLIENT_CLI_CONFIG: ../../test/client.toml
         run: |
           poetry run pytest
       - name: Cleanup test environment

diff --git a/.github/workflows/iroha2-release-pr.yml b/.github/workflows/iroha2-release-pr.yml
@@ -36,6 +36,7 @@ jobs:
           cargo build --bin iroha
       - name: Setup test Iroha 2 environment on bare metal
         run: |
+          pip3 install -r scripts/requirements.txt --no-input --break-system-packages
           ./scripts/test_env.py setup
       - name: Mark binaries as executable
         run: |

diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
@@ -222,7 +222,7 @@ Follow these commit guidelines:
 - To run the source-code based tests, execute [`cargo test`](https://doc.rust-lang.org/cargo/commands/cargo-test.html) in the Iroha root. Note that this is a long process.
 - To run benchmarks, execute [`cargo bench`](https://doc.rust-lang.org/cargo/commands/cargo-bench.html) from the Iroha root. To help debug benchmark outputs, set the `debug_assertions` environment variable like so: `RUSTFLAGS="--cfg debug_assertions" cargo bench`.
 - If you are working on a particular component, be mindful that when you run `cargo test` in a [workspace](https://doc.rust-lang.org/cargo/reference/workspaces.html), it will only run the tests for that workspace, which usually doesn't include any [integration tests](https://www.testingxperts.com/blog/what-is-integration-testing).
-- If you want to test your changes on a minimal network, the provided [`docker-compose.yml`](docker-compose.yml) creates a network of 4 Iroha peers in docker containers that can be used to test consensus and asset propagation-related logic. We recommend interacting with that network using either [`iroha-python`](https://github.com/hyperledger/iroha-python), or the included `iroha_client_cli`.
+- If you want to test your changes on a minimal network, the provided [`docker-compose.yml`](configs/swarm/docker-compose.yml) creates a network of 4 Iroha peers in docker containers that can be used to test consensus and asset propagation-related logic. We recommend interacting with that network using either [`iroha-python`](https://github.com/hyperledger/iroha-python), or the included `iroha_client_cli`.
 - Do not remove failing tests. Even tests that are ignored will be run in our pipeline eventually.
 - If possible, please benchmark your code both before and after making your changes, as a significant performance regression can break existing users' installations.
 

diff --git a/Cargo.lock b/Cargo.lock