Skip to content

Commit

Permalink
[reconfigurator] ClickHouse XML generation types (#6412)
Browse files Browse the repository at this point in the history
## Overview

This commit adds a library to generate ClickHouse replica server and
keeper configuration files. A lot of the code in the
`clickhouse-admin/types` directory has been copied over from
[clickward](https://github.com/oxidecomputer/clickward), but there are
several additions and modifications:

- New `new()` and `default()` methods that set default Oxide values.
- File generation is per node, as opposed to all files generated in a
single directory like clickward.

## Usage

To generate a replica server configuration file:

```rust
let keepers = vec![
    KeeperNodeConfig::new("ff::01".to_string()),
    KeeperNodeConfig::new("ff::02".to_string()),
    KeeperNodeConfig::new("ff::03".to_string()),
];

let servers = vec![
    ServerNodeConfig::new("ff::08".to_string()),
    ServerNodeConfig::new("ff::09".to_string()),
];

let config = ClickhouseServerConfig::new(
    Utf8PathBuf::from(config_dir.path()),
    ServerId(1),
    Utf8PathBuf::from_str("./").unwrap(),
    Ipv6Addr::from_str("ff::08").unwrap(),
    keepers,
    servers,
);

config.generate_xml_file().unwrap();
```

To generate a keeper configuration file:

```rust
let keepers = vec![
    RaftServerConfig::new(KeeperId(1), "ff::01".to_string()),
    RaftServerConfig::new(KeeperId(2), "ff::02".to_string()),
    RaftServerConfig::new(KeeperId(3), "ff::03".to_string()),
];

let config = ClickhouseKeeperConfig::new(
    Utf8PathBuf::from(config_dir.path()),
    KeeperId(1),
    keepers,
    Utf8PathBuf::from_str("./").unwrap(),
    Ipv6Addr::from_str("ff::08").unwrap(),
);

config.generate_xml_file().unwrap();
```

## Purpose

As part of the work to roll out replicated ClickHouse, we'll need to
dynamically generate the node configuration files via the
`clickhouse-admin` service. This commit is part of the work necessary to
do so.

Related: #5999 ,
#3824
  • Loading branch information
karencfv authored Aug 28, 2024
1 parent f89e1c3 commit 57b0900
Show file tree
Hide file tree
Showing 13 changed files with 982 additions and 23 deletions.
17 changes: 17 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,7 @@ default-members = [
"certificates",
"clickhouse-admin",
"clickhouse-admin/api",
"clickhouse-admin/types",
"clients/bootstrap-agent-client",
"clients/cockroach-admin-client",
"clients/ddm-admin-client",
Expand Down Expand Up @@ -311,6 +312,7 @@ chrono = { version = "0.4", features = [ "serde" ] }
ciborium = "0.2.2"
clap = { version = "4.5", features = ["cargo", "derive", "env", "wrap_help"] }
clickhouse-admin-api = { path = "clickhouse-admin/api" }
clickhouse-admin-types = { path = "clickhouse-admin/types" }
clickward = { git = "https://github.com/oxidecomputer/clickward", rev = "ceec762e6a87d2a22bf56792a3025e145caa095e" }
cockroach-admin-api = { path = "cockroach-admin/api" }
cockroach-admin-client = { path = "clients/cockroach-admin-client" }
Expand Down
1 change: 1 addition & 0 deletions clickhouse-admin/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ camino.workspace = true
chrono.workspace = true
clap.workspace = true
clickhouse-admin-api.workspace = true
clickhouse-admin-types.workspace = true
dropshot.workspace = true
http.workspace = true
illumos-utils.workspace = true
Expand Down
20 changes: 20 additions & 0 deletions clickhouse-admin/types/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
[package]
name = "clickhouse-admin-types"
version = "0.1.0"
edition = "2021"
license = "MPL-2.0"

[lints]
workspace = true

[dependencies]
anyhow.workspace = true
camino.workspace = true
camino-tempfile.workspace = true
derive_more.workspace = true
omicron-common.workspace = true
omicron-workspace-hack.workspace = true
schemars.workspace = true
serde.workspace = true
serde_json.workspace = true
expectorate.workspace = true
Loading

0 comments on commit 57b0900

Please sign in to comment.