diff --git a/config/config.md b/config/config.md index 0f70a8cb126e..6a500a5b4a34 100644 --- a/config/config.md +++ b/config/config.md @@ -13,11 +13,11 @@ | Key | Type | Default | Descriptions | | --- | -----| ------- | ----------- | | `mode` | String | `standalone` | The running mode of the datanode. It can be `standalone` or `distributed`. | -| `enable_telemetry` | Bool | `true` | Enable telemetry to collect anonymous usage data. | | `default_timezone` | String | Unset | The default timezone of the server. | | `init_regions_in_background` | Bool | `false` | Initialize all regions in the background during the startup.
By default, it provides services after all regions have been initialized. | | `init_regions_parallelism` | Integer | `16` | Parallelism of initializing regions. | | `max_concurrent_queries` | Integer | `0` | The maximum current queries allowed to be executed. Zero means unlimited. | +| `enable_telemetry` | Bool | `true` | Enable telemetry to collect anonymous usage data. Enabled by default. | | `runtime` | -- | -- | The runtime options. | | `runtime.global_rt_size` | Integer | `8` | The number of threads to execute the runtime for global read operations. | | `runtime.compact_rt_size` | Integer | `4` | The number of threads to execute the runtime for global write operations. | @@ -61,9 +61,9 @@ | `wal` | -- | -- | The WAL options. | | `wal.provider` | String | `raft_engine` | The provider of the WAL.
- `raft_engine`: the wal is stored in the local file system by raft-engine.
- `kafka`: it's remote wal that data is stored in Kafka. | | `wal.dir` | String | Unset | The directory to store the WAL files.
**It's only used when the provider is `raft_engine`**. | -| `wal.file_size` | String | `256MB` | The size of the WAL segment file.
**It's only used when the provider is `raft_engine`**. | -| `wal.purge_threshold` | String | `4GB` | The threshold of the WAL size to trigger a flush.
**It's only used when the provider is `raft_engine`**. | -| `wal.purge_interval` | String | `10m` | The interval to trigger a flush.
**It's only used when the provider is `raft_engine`**. | +| `wal.file_size` | String | `128MB` | The size of the WAL segment file.
**It's only used when the provider is `raft_engine`**. | +| `wal.purge_threshold` | String | `1GB` | The threshold of the WAL size to trigger a flush.
**It's only used when the provider is `raft_engine`**. | +| `wal.purge_interval` | String | `1m` | The interval to trigger a flush.
**It's only used when the provider is `raft_engine`**. | | `wal.read_batch_size` | Integer | `128` | The read batch size.
**It's only used when the provider is `raft_engine`**. | | `wal.sync_write` | Bool | `false` | Whether to use sync write.
**It's only used when the provider is `raft_engine`**. | | `wal.enable_log_recycle` | Bool | `true` | Whether to reuse logically truncated log files.
**It's only used when the provider is `raft_engine`**. | @@ -286,12 +286,12 @@ | `bind_addr` | String | `127.0.0.1:3002` | The bind address of metasrv. | | `server_addr` | String | `127.0.0.1:3002` | The communication server address for frontend and datanode to connect to metasrv, "127.0.0.1:3002" by default for localhost. | | `store_addrs` | Array | -- | Store server address default to etcd store. | +| `store_key_prefix` | String | `""` | If it's not empty, the metasrv will store all data with this key prefix. | +| `backend` | String | `EtcdStore` | The datastore for meta server. | | `selector` | String | `round_robin` | Datanode selector type.
- `round_robin` (default value)
- `lease_based`
- `load_based`
For details, please see "https://docs.greptime.com/developer-guide/metasrv/selector". | | `use_memory_store` | Bool | `false` | Store data in memory. | -| `enable_telemetry` | Bool | `true` | Whether to enable greptimedb telemetry. | -| `store_key_prefix` | String | `""` | If it's not empty, the metasrv will store all data with this key prefix. | | `enable_region_failover` | Bool | `false` | Whether to enable region failover.
This feature is only available on GreptimeDB running on cluster mode and
- Using Remote WAL
- Using shared storage (e.g., s3). | -| `backend` | String | `EtcdStore` | The datastore for meta server. | +| `enable_telemetry` | Bool | `true` | Whether to enable greptimedb telemetry. Enabled by default. | | `runtime` | -- | -- | The runtime options. | | `runtime.global_rt_size` | Integer | `8` | The number of threads to execute the runtime for global read operations. | | `runtime.compact_rt_size` | Integer | `4` | The number of threads to execute the runtime for global write operations. | @@ -356,7 +356,6 @@ | `node_id` | Integer | Unset | The datanode identifier and should be unique in the cluster. | | `require_lease_before_startup` | Bool | `false` | Start services after regions have obtained leases.
It will block the datanode start if it can't receive leases in the heartbeat from metasrv. | | `init_regions_in_background` | Bool | `false` | Initialize all regions in the background during the startup.
By default, it provides services after all regions have been initialized. | -| `enable_telemetry` | Bool | `true` | Enable telemetry to collect anonymous usage data. | | `init_regions_parallelism` | Integer | `16` | Parallelism of initializing regions. | | `max_concurrent_queries` | Integer | `0` | The maximum current queries allowed to be executed. Zero means unlimited. | | `rpc_addr` | String | Unset | Deprecated, use `grpc.addr` instead. | @@ -364,6 +363,7 @@ | `rpc_runtime_size` | Integer | Unset | Deprecated, use `grpc.runtime_size` instead. | | `rpc_max_recv_message_size` | String | Unset | Deprecated, use `grpc.rpc_max_recv_message_size` instead. | | `rpc_max_send_message_size` | String | Unset | Deprecated, use `grpc.rpc_max_send_message_size` instead. | +| `enable_telemetry` | Bool | `true` | Enable telemetry to collect anonymous usage data. Enabled by default. | | `http` | -- | -- | The HTTP server options. | | `http.addr` | String | `127.0.0.1:4000` | The address to bind the HTTP server. | | `http.timeout` | String | `30s` | HTTP request timeout. Set to 0 to disable timeout. | @@ -398,9 +398,9 @@ | `wal` | -- | -- | The WAL options. | | `wal.provider` | String | `raft_engine` | The provider of the WAL.
- `raft_engine`: the wal is stored in the local file system by raft-engine.
- `kafka`: it's remote wal that data is stored in Kafka. | | `wal.dir` | String | Unset | The directory to store the WAL files.
**It's only used when the provider is `raft_engine`**. | -| `wal.file_size` | String | `256MB` | The size of the WAL segment file.
**It's only used when the provider is `raft_engine`**. | -| `wal.purge_threshold` | String | `4GB` | The threshold of the WAL size to trigger a flush.
**It's only used when the provider is `raft_engine`**. | -| `wal.purge_interval` | String | `10m` | The interval to trigger a flush.
**It's only used when the provider is `raft_engine`**. | +| `wal.file_size` | String | `128MB` | The size of the WAL segment file.
**It's only used when the provider is `raft_engine`**. | +| `wal.purge_threshold` | String | `1GB` | The threshold of the WAL size to trigger a flush.
**It's only used when the provider is `raft_engine`**. | +| `wal.purge_interval` | String | `1m` | The interval to trigger a flush.
**It's only used when the provider is `raft_engine`**. | | `wal.read_batch_size` | Integer | `128` | The read batch size.
**It's only used when the provider is `raft_engine`**. | | `wal.sync_write` | Bool | `false` | Whether to use sync write.
**It's only used when the provider is `raft_engine`**. | | `wal.enable_log_recycle` | Bool | `true` | Whether to reuse logically truncated log files.
**It's only used when the provider is `raft_engine`**. | diff --git a/config/datanode.example.toml b/config/datanode.example.toml index 8bfa8732cc41..0ba80a9f7d92 100644 --- a/config/datanode.example.toml +++ b/config/datanode.example.toml @@ -13,9 +13,6 @@ require_lease_before_startup = false ## By default, it provides services after all regions have been initialized. init_regions_in_background = false -## Enable telemetry to collect anonymous usage data. -enable_telemetry = true - ## Parallelism of initializing regions. init_regions_parallelism = 16 @@ -42,6 +39,8 @@ rpc_max_recv_message_size = "512MB" ## @toml2docs:none-default rpc_max_send_message_size = "512MB" +## Enable telemetry to collect anonymous usage data. Enabled by default. +#+ enable_telemetry = true ## The HTTP server options. [http] @@ -143,15 +142,15 @@ dir = "/tmp/greptimedb/wal" ## The size of the WAL segment file. ## **It's only used when the provider is `raft_engine`**. -file_size = "256MB" +file_size = "128MB" ## The threshold of the WAL size to trigger a flush. ## **It's only used when the provider is `raft_engine`**. -purge_threshold = "4GB" +purge_threshold = "1GB" ## The interval to trigger a flush. ## **It's only used when the provider is `raft_engine`**. -purge_interval = "10m" +purge_interval = "1m" ## The read batch size. ## **It's only used when the provider is `raft_engine`**. diff --git a/config/metasrv.example.toml b/config/metasrv.example.toml index bcd7ee41412b..27716b5aa37b 100644 --- a/config/metasrv.example.toml +++ b/config/metasrv.example.toml @@ -10,6 +10,12 @@ server_addr = "127.0.0.1:3002" ## Store server address default to etcd store. store_addrs = ["127.0.0.1:2379"] +## If it's not empty, the metasrv will store all data with this key prefix. +store_key_prefix = "" + +## The datastore for meta server. +backend = "EtcdStore" + ## Datanode selector type. ## - `round_robin` (default value) ## - `lease_based` @@ -20,20 +26,14 @@ selector = "round_robin" ## Store data in memory. use_memory_store = false -## Whether to enable greptimedb telemetry. -enable_telemetry = true - -## If it's not empty, the metasrv will store all data with this key prefix. -store_key_prefix = "" - ## Whether to enable region failover. ## This feature is only available on GreptimeDB running on cluster mode and ## - Using Remote WAL ## - Using shared storage (e.g., s3). enable_region_failover = false -## The datastore for meta server. -backend = "EtcdStore" +## Whether to enable greptimedb telemetry. Enabled by default. +#+ enable_telemetry = true ## The runtime options. #+ [runtime] diff --git a/config/standalone.example.toml b/config/standalone.example.toml index 56cbeaddb9d5..8eae532d6166 100644 --- a/config/standalone.example.toml +++ b/config/standalone.example.toml @@ -1,9 +1,6 @@ ## The running mode of the datanode. It can be `standalone` or `distributed`. mode = "standalone" -## Enable telemetry to collect anonymous usage data. -enable_telemetry = true - ## The default timezone of the server. ## @toml2docs:none-default default_timezone = "UTC" @@ -18,6 +15,9 @@ init_regions_parallelism = 16 ## The maximum current queries allowed to be executed. Zero means unlimited. max_concurrent_queries = 0 +## Enable telemetry to collect anonymous usage data. Enabled by default. +#+ enable_telemetry = true + ## The runtime options. #+ [runtime] ## The number of threads to execute the runtime for global read operations. @@ -147,15 +147,15 @@ dir = "/tmp/greptimedb/wal" ## The size of the WAL segment file. ## **It's only used when the provider is `raft_engine`**. -file_size = "256MB" +file_size = "128MB" ## The threshold of the WAL size to trigger a flush. ## **It's only used when the provider is `raft_engine`**. -purge_threshold = "4GB" +purge_threshold = "1GB" ## The interval to trigger a flush. ## **It's only used when the provider is `raft_engine`**. -purge_interval = "10m" +purge_interval = "1m" ## The read batch size. ## **It's only used when the provider is `raft_engine`**. diff --git a/src/common/wal/src/config/raft_engine.rs b/src/common/wal/src/config/raft_engine.rs index af5daa9d386d..cfefd0c758b7 100644 --- a/src/common/wal/src/config/raft_engine.rs +++ b/src/common/wal/src/config/raft_engine.rs @@ -49,9 +49,9 @@ impl Default for RaftEngineConfig { fn default() -> Self { Self { dir: None, - file_size: ReadableSize::mb(256), - purge_threshold: ReadableSize::gb(4), - purge_interval: Duration::from_secs(600), + file_size: ReadableSize::mb(128), + purge_threshold: ReadableSize::gb(1), + purge_interval: Duration::from_secs(60), read_batch_size: 128, sync_write: false, enable_log_recycle: true, diff --git a/tests-integration/tests/http.rs b/tests-integration/tests/http.rs index 9d7b81f3919b..4da65f0b21f5 100644 --- a/tests-integration/tests/http.rs +++ b/tests-integration/tests/http.rs @@ -886,9 +886,9 @@ with_metric_engine = true [wal] provider = "raft_engine" -file_size = "256MiB" -purge_threshold = "4GiB" -purge_interval = "10m" +file_size = "128MiB" +purge_threshold = "1GiB" +purge_interval = "1m" read_batch_size = 128 sync_write = false enable_log_recycle = true