-
Notifications
You must be signed in to change notification settings - Fork 9
Benchmaker
Mikhail Kalinin edited this page May 16, 2019
·
1 revision
Benchmaker is a tool for benchmarking Beacon chain consensus algorithms, i.e. a part of Beacon chain spec which operates with attestations including slot, block and epoch processing; operations like deposits, slashings and the others are not counted. Benchmaker is based on Beacon chain simulator with one major difference, all validators are handled by a single beacon chain instance which significantly reduces memory and CPU consumption.
To run a simulator you need to install Java 8 or later. There are many guidelines in Internet, even the official one, from Oracle.
Clone the repo and execute build command:
git clone --branch develop --recurse-submodules -j4 https://github.com/harmony-dev/beacon-chain-java.git
cd beacon-chain-java
./gradlew clean build -x testUnpack the distribution:
unzip start/benchmaker/build/distributions/benchmaker-0.1.0.zipUse following commands to check it out:
cd benchmaker-0.1.0/bin
./benchmaker --no-bls./benchmaker --help
Usage: benchmaker [-hV] [--no-bls] [--no-cache] [--no-increment]
[--epochs=epochs] [--registry-size=size]
Benchmark tool for beacon chain spec
--epochs=epochs Length of benchmark session in epochs.
Note: session starts from a 3rd epoch.
--no-bls Turns off BLS.
--no-cache Turns off caches used in the spec execution.
--no-increment Turns off incremental hashing.
--registry-size=size Validator registry size.
-h, --help Show this help message and exit.
-V, --version Print version information and exit.
Use following command to run session with 10,000 validators:
./benchmaker --registry-size 10000 --no-bls --epochs 1
15:01:08.623 INFO - Benchmark session: 1 epochs, 10000 validators, BLS is disabled, caches are enabled
PROCESSING OVERVIEW min, ms avg, ms 95%, ms
slot 0.470 0.616 0.687
block 0.926 1.515 2.221
epoch 27420.555 27420.555 27420.555
SLOT PROCESSING
TOP METHODS min, ms avg, ms 95%, ms count total, ms
cache_state 0.047 0.067 0.060 1 0.067
advance_slot 0.001 0.002 0.002 1 0.002
HELPERS min, ms avg, ms 95%, ms count total, ms
signing_root 0.018 0.022 0.024 1 0.022
hash_tree_root 0.017 0.021 0.022 1 0.021
SUMMARY min, ms avg, ms % of total
TOP METHODS 0.048 0.068 100.00%
HELPERS
signing_root 0.018 0.022 32.75%
hash_tree_root 0.017 0.021 31.15%
BLOCK PROCESSING
TOP METHODS min, ms avg, ms 95%, ms count total, ms
process_block_header 0.488 0.796 0.693 1 0.796
process_attestation 0.176 0.265 1.322 1 0.265
process_randao 0.019 0.029 0.027 1 0.029
process_eth1_data 0.016 0.019 0.020 1 0.019
BLS min, ms avg, ms 95%, ms count total, ms
bls_aggregate_pubkeys 0.002 0.013 0.020 2 0.027
bls_verify 0.000 0.002 0.002 2 0.003
bls_verify_multiple 0.001 0.001 0.001 1 0.001
HELPERS min, ms avg, ms 95%, ms count total, ms
get_beacon_proposer_index 0.003 0.177 0.012 3 0.530
get_crosslink_committee 0.000 0.096 0.000 5 0.481
hash_tree_root 0.000 0.034 0.030 5 0.171
signing_root 0.009 0.083 0.037 2 0.166
get_attesting_indices 0.001 0.016 0.028 2 0.033
generate_seed 0.001 0.005 0.004 4 0.022
get_active_validator_indices 0.000 0.000 0.000 21 0.010
verify_bitfield 0.000 0.000 0.000 2 0.001
SUMMARY min, ms avg, ms % of total
TOP METHODS 0.700 1.109 100.00%
HELPERS
get_beacon_proposer_index 0.008 0.530 47.77%
get_crosslink_committee 0.000 0.481 43.40%
hash_tree_root 0.000 0.171 15.44%
signing_root 0.017 0.166 14.98%
get_attesting_indices 0.002 0.033 2.96%
generate_seed 0.003 0.022 1.97%
get_active_validator_indices 0.000 0.010 0.92%
verify_bitfield 0.000 0.001 0.06%
BLS
bls_aggregate_pubkeys 0.005 0.027 2.42%
bls_verify 0.001 0.003 0.29%
bls_verify_multiple 0.001 0.001 0.11%
EPOCH PROCESSING
TOP METHODS min, ms avg, ms 95%, ms count total, ms
process_rewards_and_penalties 27366.349 27366.349 27366.349 1 27366.349
process_justification_and_finalization 31.099 31.099 31.099 1 31.099
process_crosslinks 9.268 9.268 9.268 1 9.268
process_registry_updates 5.665 5.665 5.665 1 5.665
process_final_updates 4.152 4.152 4.152 1 4.152
process_slashings 1.984 1.984 1.984 1 1.984
HELPERS min, ms avg, ms 95%, ms count total, ms
get_base_reward 0.000 0.267 0.217 60000 16038.778
get_total_balance 0.000 0.264 0.216 60393 15949.137
get_attesting_indices 0.000 0.017 0.014 640758 10618.926
get_crosslink_committee 0.000 0.000 0.000 640950 209.611
get_unslashed_attesting_indices 0.000 0.318 0.049 196 62.331
verify_bitfield 0.000 0.000 0.000 640758 62.145
get_active_validator_indices 0.000 0.000 0.000 61034 15.284
get_winning_crosslink_and_attesting_indices 0.000 0.060 0.091 192 11.507
hash_tree_root 0.000 0.002 0.005 311 0.769
generate_seed 0.000 0.003 0.010 64 0.200
SUMMARY min, ms avg, ms % of total
TOP METHODS 27418.517 27418.517 100.00%
HELPERS
get_base_reward 0.213 16038.778 58.50%
get_total_balance 0.000 15949.137 58.17%
get_attesting_indices 0.013 10618.926 38.73%
get_crosslink_committee 0.000 209.611 0.76%
get_unslashed_attesting_indices 0.033 62.331 0.23%
verify_bitfield 0.000 62.145 0.23%
get_active_validator_indices 0.000 15.284 0.06%
get_winning_crosslink_and_attesting_indices 0.003 11.507 0.04%
hash_tree_root 0.001 0.769 0.00%
generate_seed 0.002 0.200 0.00%
TBD