Skip to content

Commit

Permalink
doc: add missing parameters and reorganize existing parameters in com…
Browse files Browse the repository at this point in the history
…pact settings (#4355)

### Motivation
The current configuration documentation for BookKeeper lacks some important parameters. Additionally, existing parameters are not optimally organized. This update:
- Introduces missing parameters like `isThrottleByBytes`, `compactionRateByEntries`, and `compactionRateByBytes`.
- Reorganizes the configuration parameters for better clarity.

### Changes
- **Added:** New parameters that were missing in the configuration:
  - `compactionRateByEntries`: Set the rate at which compaction will read entries, measured in entries per second.
  - `compactionRateByBytes`: Set the rate at which compaction will read entries, measured in bytes per second.
  
- **Reorganized:** Adjusted the placement of existing parameters for better logical grouping and readability:
  - `minorCompactionThreshold`: Adjusted position for clarity.
  - `minorCompactionInterval`: Moved for logical grouping.

Signed-off-by: ZhangJian He <[email protected]>
Co-authored-by: ZhangJian He <[email protected]>
  • Loading branch information
thetumbled and shoothzj authored May 10, 2024
1 parent bd3ad50 commit 4ed58ab
Show file tree
Hide file tree
Showing 14 changed files with 84 additions and 74 deletions.
12 changes: 7 additions & 5 deletions site3/website/docs/reference/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,15 +172,17 @@ The table below lists parameters that you can set to configure bookies. All conf

| Parameter | Description | Default
| --------- | ----------- | ------- |
| isThrottleByBytes | Throttle compaction by bytes or by entries. | false |
| compactionRate | The rate at which compaction will read entries. The unit is adds per second. | 1000 |
| minorCompactionThreshold | Threshold of minor compaction. For those entry log files whose remaining size percentage reaches below this threshold will be compacted in a minor compaction. If it is set to less than zero, the minor compaction is disabled. | 0.2 |
| minorCompactionInterval | Interval to run minor compaction, in seconds. If it is set to less than zero, the minor compaction is disabled. | 3600 |
| compactionRateByEntries | Set the rate at which compaction will read entries. The unit is adds per second. | 1000 |
| compactionRateByBytes | Set the rate at which compaction will read entries. The unit is bytes added per second. | 1000000 |
| compactionMaxOutstandingRequests | Set the maximum number of entries which can be compacted without flushing. When compacting, the entries are written to the entrylog and the new offsets are cached in memory. Once the entrylog is flushed the index is updated with the new offsets. This parameter controls the number of entries added to the entrylog before a flush is forced. A higher value for this parameter means more memory will be used for offsets. Each offset consists of 3 longs. This parameter should *not* be modified unless you know what you're doing. | 100000 |
| minorCompactionThreshold | Threshold of minor compaction. For those entry log files whose remaining size percentage reaches below this threshold will be compacted in a minor compaction. If it is set to less than zero, the minor compaction is disabled. | 0.2 |
| majorCompactionThreshold | Threshold of major compaction. For those entry log files whose remaining size percentage reaches below this threshold will be compacted in a major compaction. Those entry log files whose remaining size percentage is still higher than the threshold will never be compacted. If it is set to less than zero, the minor compaction is disabled. | 0.8 |
| minorCompactionInterval | Interval to run minor compaction, in seconds. If it is set to less than zero, the minor compaction is disabled. | 3600 |
| majorCompactionInterval | Interval to run major compaction, in seconds. If it is set to less than zero, the major compaction is disabled. | 86400 |
| isThrottleByBytes | Throttle compaction by bytes or by entries. | false |
| compactionRateByEntries | Set the rate at which compaction will read entries. The unit is adds per second. | 1000 |
| compactionRateByBytes | Set the rate at which compaction will read entries. The unit is bytes added per second. | 1000000 |
| minorCompactionMaxTimeMillis | Maximum milliseconds to run minor Compaction. | -1 to run indefinitely. |
| majorCompactionMaxTimeMillis | Maximum milliseconds to run major Compaction. | -1 to run indefinitely. |
| useTransactionalCompaction | Flag to enable/disable transactional compaction. If it is set to true, it will use transactional compaction, which uses<br />new entry log files to store entries after compaction; otherwise, it will use normal compaction, which shares same entry<br />log file with normal add operations.<br /> | false |


Expand Down
10 changes: 5 additions & 5 deletions site3/website/versioned_docs/version-4.10.0/reference/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,15 +161,15 @@ The table below lists parameters that you can set to configure bookies. All conf

| Parameter | Description | Default
| --------- | ----------- | ------- |
| isThrottleByBytes | Throttle compaction by bytes or by entries. | false |
| compactionRate | The rate at which compaction will read entries. The unit is adds per second. | 1000 |
| minorCompactionThreshold | Threshold of minor compaction. For those entry log files whose remaining size percentage reaches below this threshold will be compacted in a minor compaction. If it is set to less than zero, the minor compaction is disabled. | 0.2 |
| minorCompactionInterval | Interval to run minor compaction, in seconds. If it is set to less than zero, the minor compaction is disabled. | 3600 |
| compactionRateByEntries | Set the rate at which compaction will read entries. The unit is adds per second. | 1000 |
| compactionRateByBytes | Set the rate at which compaction will read entries. The unit is bytes added per second. | 1000000 |
| compactionMaxOutstandingRequests | Set the maximum number of entries which can be compacted without flushing. When compacting, the entries are written to the entrylog and the new offsets are cached in memory. Once the entrylog is flushed the index is updated with the new offsets. This parameter controls the number of entries added to the entrylog before a flush is forced. A higher value for this parameter means more memory will be used for offsets. Each offset consists of 3 longs. This parameter should *not* be modified unless you know what you're doing. | 100000 |
| minorCompactionThreshold | Threshold of minor compaction. For those entry log files whose remaining size percentage reaches below this threshold will be compacted in a minor compaction. If it is set to less than zero, the minor compaction is disabled. | 0.2 |
| majorCompactionThreshold | Threshold of major compaction. For those entry log files whose remaining size percentage reaches below this threshold will be compacted in a major compaction. Those entry log files whose remaining size percentage is still higher than the threshold will never be compacted. If it is set to less than zero, the minor compaction is disabled. | 0.8 |
| minorCompactionInterval | Interval to run minor compaction, in seconds. If it is set to less than zero, the minor compaction is disabled. | 3600 |
| majorCompactionInterval | Interval to run major compaction, in seconds. If it is set to less than zero, the major compaction is disabled. | 86400 |
| isThrottleByBytes | Throttle compaction by bytes or by entries. | false |
| compactionRateByEntries | Set the rate at which compaction will read entries. The unit is adds per second. | 1000 |
| compactionRateByBytes | Set the rate at which compaction will read entries. The unit is bytes added per second. | 1000000 |
| useTransactionalCompaction | Flag to enable/disable transactional compaction. If it is set to true, it will use transactional compaction, which uses<br />new entry log files to store entries after compaction; otherwise, it will use normal compaction, which shares same entry<br />log file with normal add operations.<br /> | false |


Expand Down
10 changes: 5 additions & 5 deletions site3/website/versioned_docs/version-4.11.1/reference/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,15 +161,15 @@ The table below lists parameters that you can set to configure bookies. All conf

| Parameter | Description | Default
| --------- | ----------- | ------- |
| isThrottleByBytes | Throttle compaction by bytes or by entries. | false |
| compactionRate | The rate at which compaction will read entries. The unit is adds per second. | 1000 |
| minorCompactionThreshold | Threshold of minor compaction. For those entry log files whose remaining size percentage reaches below this threshold will be compacted in a minor compaction. If it is set to less than zero, the minor compaction is disabled. | 0.2 |
| minorCompactionInterval | Interval to run minor compaction, in seconds. If it is set to less than zero, the minor compaction is disabled. | 3600 |
| compactionRateByEntries | Set the rate at which compaction will read entries. The unit is adds per second. | 1000 |
| compactionRateByBytes | Set the rate at which compaction will read entries. The unit is bytes added per second. | 1000000 |
| compactionMaxOutstandingRequests | Set the maximum number of entries which can be compacted without flushing. When compacting, the entries are written to the entrylog and the new offsets are cached in memory. Once the entrylog is flushed the index is updated with the new offsets. This parameter controls the number of entries added to the entrylog before a flush is forced. A higher value for this parameter means more memory will be used for offsets. Each offset consists of 3 longs. This parameter should *not* be modified unless you know what you're doing. | 100000 |
| minorCompactionThreshold | Threshold of minor compaction. For those entry log files whose remaining size percentage reaches below this threshold will be compacted in a minor compaction. If it is set to less than zero, the minor compaction is disabled. | 0.2 |
| majorCompactionThreshold | Threshold of major compaction. For those entry log files whose remaining size percentage reaches below this threshold will be compacted in a major compaction. Those entry log files whose remaining size percentage is still higher than the threshold will never be compacted. If it is set to less than zero, the minor compaction is disabled. | 0.8 |
| minorCompactionInterval | Interval to run minor compaction, in seconds. If it is set to less than zero, the minor compaction is disabled. | 3600 |
| majorCompactionInterval | Interval to run major compaction, in seconds. If it is set to less than zero, the major compaction is disabled. | 86400 |
| isThrottleByBytes | Throttle compaction by bytes or by entries. | false |
| compactionRateByEntries | Set the rate at which compaction will read entries. The unit is adds per second. | 1000 |
| compactionRateByBytes | Set the rate at which compaction will read entries. The unit is bytes added per second. | 1000000 |
| useTransactionalCompaction | Flag to enable/disable transactional compaction. If it is set to true, it will use transactional compaction, which uses<br />new entry log files to store entries after compaction; otherwise, it will use normal compaction, which shares same entry<br />log file with normal add operations.<br /> | false |


Expand Down
10 changes: 5 additions & 5 deletions site3/website/versioned_docs/version-4.12.1/reference/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,15 +161,15 @@ The table below lists parameters that you can set to configure bookies. All conf

| Parameter | Description | Default
| --------- | ----------- | ------- |
| isThrottleByBytes | Throttle compaction by bytes or by entries. | false |
| compactionRate | The rate at which compaction will read entries. The unit is adds per second. | 1000 |
| minorCompactionThreshold | Threshold of minor compaction. For those entry log files whose remaining size percentage reaches below this threshold will be compacted in a minor compaction. If it is set to less than zero, the minor compaction is disabled. | 0.2 |
| minorCompactionInterval | Interval to run minor compaction, in seconds. If it is set to less than zero, the minor compaction is disabled. | 3600 |
| compactionRateByEntries | Set the rate at which compaction will read entries. The unit is adds per second. | 1000 |
| compactionRateByBytes | Set the rate at which compaction will read entries. The unit is bytes added per second. | 1000000 |
| compactionMaxOutstandingRequests | Set the maximum number of entries which can be compacted without flushing. When compacting, the entries are written to the entrylog and the new offsets are cached in memory. Once the entrylog is flushed the index is updated with the new offsets. This parameter controls the number of entries added to the entrylog before a flush is forced. A higher value for this parameter means more memory will be used for offsets. Each offset consists of 3 longs. This parameter should *not* be modified unless you know what you're doing. | 100000 |
| minorCompactionThreshold | Threshold of minor compaction. For those entry log files whose remaining size percentage reaches below this threshold will be compacted in a minor compaction. If it is set to less than zero, the minor compaction is disabled. | 0.2 |
| majorCompactionThreshold | Threshold of major compaction. For those entry log files whose remaining size percentage reaches below this threshold will be compacted in a major compaction. Those entry log files whose remaining size percentage is still higher than the threshold will never be compacted. If it is set to less than zero, the minor compaction is disabled. | 0.8 |
| minorCompactionInterval | Interval to run minor compaction, in seconds. If it is set to less than zero, the minor compaction is disabled. | 3600 |
| majorCompactionInterval | Interval to run major compaction, in seconds. If it is set to less than zero, the major compaction is disabled. | 86400 |
| isThrottleByBytes | Throttle compaction by bytes or by entries. | false |
| compactionRateByEntries | Set the rate at which compaction will read entries. The unit is adds per second. | 1000 |
| compactionRateByBytes | Set the rate at which compaction will read entries. The unit is bytes added per second. | 1000000 |
| useTransactionalCompaction | Flag to enable/disable transactional compaction. If it is set to true, it will use transactional compaction, which uses<br />new entry log files to store entries after compaction; otherwise, it will use normal compaction, which shares same entry<br />log file with normal add operations.<br /> | false |


Expand Down
10 changes: 5 additions & 5 deletions site3/website/versioned_docs/version-4.13.0/reference/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,15 +161,15 @@ The table below lists parameters that you can set to configure bookies. All conf

| Parameter | Description | Default
| --------- | ----------- | ------- |
| isThrottleByBytes | Throttle compaction by bytes or by entries. | false |
| compactionRate | The rate at which compaction will read entries. The unit is adds per second. | 1000 |
| minorCompactionThreshold | Threshold of minor compaction. For those entry log files whose remaining size percentage reaches below this threshold will be compacted in a minor compaction. If it is set to less than zero, the minor compaction is disabled. | 0.2 |
| minorCompactionInterval | Interval to run minor compaction, in seconds. If it is set to less than zero, the minor compaction is disabled. | 3600 |
| compactionRateByEntries | Set the rate at which compaction will read entries. The unit is adds per second. | 1000 |
| compactionRateByBytes | Set the rate at which compaction will read entries. The unit is bytes added per second. | 1000000 |
| compactionMaxOutstandingRequests | Set the maximum number of entries which can be compacted without flushing. When compacting, the entries are written to the entrylog and the new offsets are cached in memory. Once the entrylog is flushed the index is updated with the new offsets. This parameter controls the number of entries added to the entrylog before a flush is forced. A higher value for this parameter means more memory will be used for offsets. Each offset consists of 3 longs. This parameter should *not* be modified unless you know what you're doing. | 100000 |
| minorCompactionThreshold | Threshold of minor compaction. For those entry log files whose remaining size percentage reaches below this threshold will be compacted in a minor compaction. If it is set to less than zero, the minor compaction is disabled. | 0.2 |
| majorCompactionThreshold | Threshold of major compaction. For those entry log files whose remaining size percentage reaches below this threshold will be compacted in a major compaction. Those entry log files whose remaining size percentage is still higher than the threshold will never be compacted. If it is set to less than zero, the minor compaction is disabled. | 0.8 |
| minorCompactionInterval | Interval to run minor compaction, in seconds. If it is set to less than zero, the minor compaction is disabled. | 3600 |
| majorCompactionInterval | Interval to run major compaction, in seconds. If it is set to less than zero, the major compaction is disabled. | 86400 |
| isThrottleByBytes | Throttle compaction by bytes or by entries. | false |
| compactionRateByEntries | Set the rate at which compaction will read entries. The unit is adds per second. | 1000 |
| compactionRateByBytes | Set the rate at which compaction will read entries. The unit is bytes added per second. | 1000000 |
| useTransactionalCompaction | Flag to enable/disable transactional compaction. If it is set to true, it will use transactional compaction, which uses<br />new entry log files to store entries after compaction; otherwise, it will use normal compaction, which shares same entry<br />log file with normal add operations.<br /> | false |


Expand Down
12 changes: 7 additions & 5 deletions site3/website/versioned_docs/version-4.14.8/reference/config.md
Original file line number Diff line number Diff line change
Expand Up @@ -161,15 +161,17 @@ The table below lists parameters that you can set to configure bookies. All conf

| Parameter | Description | Default
| --------- | ----------- | ------- |
| isThrottleByBytes | Throttle compaction by bytes or by entries. | false |
| compactionRate | The rate at which compaction will read entries. The unit is adds per second. | 1000 |
| minorCompactionThreshold | Threshold of minor compaction. For those entry log files whose remaining size percentage reaches below this threshold will be compacted in a minor compaction. If it is set to less than zero, the minor compaction is disabled. | 0.2 |
| minorCompactionInterval | Interval to run minor compaction, in seconds. If it is set to less than zero, the minor compaction is disabled. | 3600 |
| compactionRateByEntries | Set the rate at which compaction will read entries. The unit is adds per second. | 1000 |
| compactionRateByBytes | Set the rate at which compaction will read entries. The unit is bytes added per second. | 1000000 |
| compactionMaxOutstandingRequests | Set the maximum number of entries which can be compacted without flushing. When compacting, the entries are written to the entrylog and the new offsets are cached in memory. Once the entrylog is flushed the index is updated with the new offsets. This parameter controls the number of entries added to the entrylog before a flush is forced. A higher value for this parameter means more memory will be used for offsets. Each offset consists of 3 longs. This parameter should *not* be modified unless you know what you're doing. | 100000 |
| minorCompactionThreshold | Threshold of minor compaction. For those entry log files whose remaining size percentage reaches below this threshold will be compacted in a minor compaction. If it is set to less than zero, the minor compaction is disabled. | 0.2 |
| majorCompactionThreshold | Threshold of major compaction. For those entry log files whose remaining size percentage reaches below this threshold will be compacted in a major compaction. Those entry log files whose remaining size percentage is still higher than the threshold will never be compacted. If it is set to less than zero, the minor compaction is disabled. | 0.8 |
| minorCompactionInterval | Interval to run minor compaction, in seconds. If it is set to less than zero, the minor compaction is disabled. | 3600 |
| majorCompactionInterval | Interval to run major compaction, in seconds. If it is set to less than zero, the major compaction is disabled. | 86400 |
| isThrottleByBytes | Throttle compaction by bytes or by entries. | false |
| compactionRateByEntries | Set the rate at which compaction will read entries. The unit is adds per second. | 1000 |
| compactionRateByBytes | Set the rate at which compaction will read entries. The unit is bytes added per second. | 1000000 |
| minorCompactionMaxTimeMillis | Maximum milliseconds to run minor Compaction. | -1 to run indefinitely. |
| majorCompactionMaxTimeMillis | Maximum milliseconds to run major Compaction. | -1 to run indefinitely. |
| useTransactionalCompaction | Flag to enable/disable transactional compaction. If it is set to true, it will use transactional compaction, which uses<br />new entry log files to store entries after compaction; otherwise, it will use normal compaction, which shares same entry<br />log file with normal add operations.<br /> | false |


Expand Down
Loading

0 comments on commit 4ed58ab

Please sign in to comment.