Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: sync cached balance when adding new validator to registry #7255

Merged
merged 3 commits into from
Nov 28, 2024
Merged

fix: sync cached balance when adding new validator to registry #7255

merged 3 commits into from
Nov 28, 2024

Conversation

twoeths
Copy link
Contributor

@twoeths twoeths commented Nov 28, 2024

Motivation

Got "invalid state root" issue in mekong

debug: Block error slot=143681, code=BLOCK_ERROR_INVALID_STATE_ROOT, slot=143681, root=0xbffe8157d89ac64e41c6e856b91f2a26229d06183cbeff804df82ef460f3971e, expectedRoot=0x1b43fb4c8e07039aa3c28255899f385671ad499759b047465f144005f6a7a1f8

Description

  • we did not sync cached balance when adding a new validator to registry. When top up in 2nd deposit, the cached balance is undefined hence the processEffectiveBalanceUpdates is not correct

Closes #issue_number

Testtomg

  • I was able to reproduce the issue in unit test here
  • however it does not work in my node, still got same issue

@codecov Codecov
Copy link

codecov bot commented Nov 28, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 0% with 3 lines in your changes missing coverage. Please review.

Project coverage is 48.50%. Comparing base (0c1ad93) to head (6469a69).
Report is 1 commits behind head on unstable.

Additional details and impacted files 
@@             Coverage Diff              @@
##           unstable    #7255      +/-   ##
============================================
- Coverage     48.50%   48.50%   -0.01%     
============================================
  Files           600      600              
  Lines         40132    40135       +3     
  Branches       2054     2054              
============================================
  Hits          19468    19468              
- Misses        20626    20629       +3     
  Partials         38       38

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 28, 2024 •
edited
Loading

Performance Report

✔️ no performance regression detected

Full benchmark results
Benchmark suite Current: adf2f2c Previous: 0c1ad93 Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 2.1900 ms/op 3.6183 ms/op 0.61
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 53.012 us/op 87.809 us/op 0.60
BLS verify - blst 954.19 us/op 1.1979 ms/op 0.80
BLS verifyMultipleSignatures 3 - blst 1.3279 ms/op 1.6410 ms/op 0.81
BLS verifyMultipleSignatures 8 - blst 1.9237 ms/op 2.5485 ms/op 0.75
BLS verifyMultipleSignatures 32 - blst 6.7400 ms/op 7.4320 ms/op 0.91
BLS verifyMultipleSignatures 64 - blst 10.703 ms/op 12.373 ms/op 0.87
BLS verifyMultipleSignatures 128 - blst 17.054 ms/op 22.508 ms/op 0.76
BLS deserializing 10000 signatures 689.64 ms/op 821.79 ms/op 0.84
BLS deserializing 100000 signatures 6.8342 s/op 8.0818 s/op 0.85
BLS verifyMultipleSignatures - same message - 3 - blst 884.07 us/op 1.2256 ms/op 0.72
BLS verifyMultipleSignatures - same message - 8 - blst 1.0608 ms/op 1.2790 ms/op 0.83
BLS verifyMultipleSignatures - same message - 32 - blst 1.6672 ms/op 2.2014 ms/op 0.76
BLS verifyMultipleSignatures - same message - 64 - blst 2.5595 ms/op 3.0331 ms/op 0.84
BLS verifyMultipleSignatures - same message - 128 - blst 4.4211 ms/op 4.9384 ms/op 0.90
BLS aggregatePubkeys 32 - blst 19.433 us/op 24.396 us/op 0.80
BLS aggregatePubkeys 128 - blst 68.917 us/op 82.119 us/op 0.84
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 62.203 ms/op 126.99 ms/op 0.49
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 61.419 ms/op 70.251 ms/op 0.87
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 47.378 ms/op 54.917 ms/op 0.86
getSlashingsAndExits - default max 88.573 us/op 203.86 us/op 0.43
getSlashingsAndExits - 2k 306.52 us/op 458.29 us/op 0.67
proposeBlockBody type=full, size=empty 5.8235 ms/op 8.5661 ms/op 0.68
isKnown best case - 1 super set check 282.00 ns/op 771.00 ns/op 0.37
isKnown normal case - 2 super set checks 276.00 ns/op 827.00 ns/op 0.33
isKnown worse case - 16 super set checks 271.00 ns/op 734.00 ns/op 0.37
InMemoryCheckpointStateCache - add get delete 3.1640 us/op 4.0620 us/op 0.78
validate api signedAggregateAndProof - struct 2.1787 ms/op 1.8395 ms/op 1.18
validate gossip signedAggregateAndProof - struct 1.7466 ms/op 1.8476 ms/op 0.95
batch validate gossip attestation - vc 640000 - chunk 32 134.11 us/op 181.21 us/op 0.74
batch validate gossip attestation - vc 640000 - chunk 64 116.22 us/op 161.88 us/op 0.72
batch validate gossip attestation - vc 640000 - chunk 128 110.74 us/op 159.41 us/op 0.69
batch validate gossip attestation - vc 640000 - chunk 256 112.54 us/op 140.65 us/op 0.80
pickEth1Vote - no votes 1.2571 ms/op 1.8826 ms/op 0.67
pickEth1Vote - max votes 6.8549 ms/op 12.014 ms/op 0.57
pickEth1Vote - Eth1Data hashTreeRoot value x2048 17.962 ms/op 21.251 ms/op 0.85
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 27.448 ms/op 33.383 ms/op 0.82
pickEth1Vote - Eth1Data fastSerialize value x2048 547.35 us/op 717.82 us/op 0.76
pickEth1Vote - Eth1Data fastSerialize tree x2048 3.0662 ms/op 4.2292 ms/op 0.73
bytes32 toHexString 483.00 ns/op 791.00 ns/op 0.61
bytes32 Buffer.toString(hex) 279.00 ns/op 315.00 ns/op 0.89
bytes32 Buffer.toString(hex) from Uint8Array 489.00 ns/op 612.00 ns/op 0.80
bytes32 Buffer.toString(hex) + 0x 274.00 ns/op 312.00 ns/op 0.88
Object access 1 prop 0.15000 ns/op 0.22700 ns/op 0.66
Map access 1 prop 0.13600 ns/op 0.17300 ns/op 0.79
Object get x1000 5.9920 ns/op 7.8290 ns/op 0.77
Map get x1000 6.8970 ns/op 7.9820 ns/op 0.86
Object set x1000 38.024 ns/op 77.847 ns/op 0.49
Map set x1000 27.373 ns/op 50.518 ns/op 0.54
Return object 10000 times 0.30420 ns/op 0.35600 ns/op 0.85
Throw Error 10000 times 3.4889 us/op 4.0008 us/op 0.87
toHex 163.61 ns/op 197.74 ns/op 0.83
Buffer.from 153.62 ns/op 191.67 ns/op 0.80
shared Buffer 94.865 ns/op 123.00 ns/op 0.77
fastMsgIdFn sha256 / 200 bytes 2.3520 us/op 3.1290 us/op 0.75
fastMsgIdFn h32 xxhash / 200 bytes 239.00 ns/op 393.00 ns/op 0.61
fastMsgIdFn h64 xxhash / 200 bytes 266.00 ns/op 396.00 ns/op 0.67
fastMsgIdFn sha256 / 1000 bytes 7.3370 us/op 10.619 us/op 0.69
fastMsgIdFn h32 xxhash / 1000 bytes 364.00 ns/op 590.00 ns/op 0.62
fastMsgIdFn h64 xxhash / 1000 bytes 343.00 ns/op 427.00 ns/op 0.80
fastMsgIdFn sha256 / 10000 bytes 64.136 us/op 93.475 us/op 0.69
fastMsgIdFn h32 xxhash / 10000 bytes 1.8920 us/op 2.4900 us/op 0.76
fastMsgIdFn h64 xxhash / 10000 bytes 1.2480 us/op 1.5250 us/op 0.82
send data - 1000 256B messages 14.335 ms/op 19.869 ms/op 0.72
send data - 1000 512B messages 18.094 ms/op 24.480 ms/op 0.74
send data - 1000 1024B messages 27.472 ms/op 41.180 ms/op 0.67
send data - 1000 1200B messages 21.667 ms/op 44.156 ms/op 0.49
send data - 1000 2048B messages 37.283 ms/op 47.562 ms/op 0.78
send data - 1000 4096B messages 36.964 ms/op 52.602 ms/op 0.70
send data - 1000 16384B messages 83.595 ms/op 119.52 ms/op 0.70
send data - 1000 65536B messages 235.72 ms/op 334.39 ms/op 0.70
enrSubnets - fastDeserialize 64 bits 1.1270 us/op 1.5360 us/op 0.73
enrSubnets - ssz BitVector 64 bits 425.00 ns/op 488.00 ns/op 0.87
enrSubnets - fastDeserialize 4 bits 195.00 ns/op 213.00 ns/op 0.92
enrSubnets - ssz BitVector 4 bits 413.00 ns/op 453.00 ns/op 0.91
prioritizePeers score -10:0 att 32-0.1 sync 2-0 193.46 us/op 202.78 us/op 0.95
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 190.74 us/op 193.23 us/op 0.99
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 330.61 us/op 320.03 us/op 1.03
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 493.70 us/op 553.48 us/op 0.89
prioritizePeers score 0:0 att 64-1 sync 4-1 914.74 us/op 1.1593 ms/op 0.79
array of 16000 items push then shift 1.7211 us/op 2.0863 us/op 0.82
LinkedList of 16000 items push then shift 7.8450 ns/op 14.538 ns/op 0.54
array of 16000 items push then pop 142.86 ns/op 162.44 ns/op 0.88
LinkedList of 16000 items push then pop 7.6760 ns/op 8.7090 ns/op 0.88
array of 24000 items push then shift 2.5146 us/op 3.1043 us/op 0.81
LinkedList of 24000 items push then shift 8.2140 ns/op 11.036 ns/op 0.74
array of 24000 items push then pop 201.03 ns/op 230.78 ns/op 0.87
LinkedList of 24000 items push then pop 7.8670 ns/op 9.2940 ns/op 0.85
intersect bitArray bitLen 8 6.7750 ns/op 7.4970 ns/op 0.90
intersect array and set length 8 53.962 ns/op 97.565 ns/op 0.55
intersect bitArray bitLen 128 30.555 ns/op 33.183 ns/op 0.92
intersect array and set length 128 761.56 ns/op 989.36 ns/op 0.77
bitArray.getTrueBitIndexes() bitLen 128 2.3000 us/op 2.6290 us/op 0.87
bitArray.getTrueBitIndexes() bitLen 248 4.1620 us/op 4.8200 us/op 0.86
bitArray.getTrueBitIndexes() bitLen 512 8.4060 us/op 10.326 us/op 0.81
Buffer.concat 32 items 998.00 ns/op 1.1870 us/op 0.84
Uint8Array.set 32 items 1.7970 us/op 1.7070 us/op 1.05
Buffer.copy 2.2000 us/op 1.9830 us/op 1.11
Uint8Array.set - with subarray 2.9460 us/op 2.7930 us/op 1.05
Uint8Array.set - without subarray 1.4980 us/op 2.0460 us/op 0.73
getUint32 - dataview 245.00 ns/op 316.00 ns/op 0.78
getUint32 - manual 205.00 ns/op 192.00 ns/op 1.07
Set add up to 64 items then delete first 2.5133 us/op 2.5967 us/op 0.97
OrderedSet add up to 64 items then delete first 4.0790 us/op 3.5762 us/op 1.14
Set add up to 64 items then delete last 3.3042 us/op 2.7908 us/op 1.18
OrderedSet add up to 64 items then delete last 5.0311 us/op 4.5540 us/op 1.10
Set add up to 64 items then delete middle 2.8511 us/op 3.2518 us/op 0.88
OrderedSet add up to 64 items then delete middle 5.6211 us/op 6.2436 us/op 0.90
Set add up to 128 items then delete first 6.3424 us/op 6.0006 us/op 1.06
OrderedSet add up to 128 items then delete first 9.9961 us/op 9.1091 us/op 1.10
Set add up to 128 items then delete last 6.4746 us/op 6.4871 us/op 1.00
OrderedSet add up to 128 items then delete last 8.9055 us/op 10.612 us/op 0.84
Set add up to 128 items then delete middle 5.7484 us/op 6.2259 us/op 0.92
OrderedSet add up to 128 items then delete middle 16.055 us/op 16.992 us/op 0.94
Set add up to 256 items then delete first 12.134 us/op 12.301 us/op 0.99
OrderedSet add up to 256 items then delete first 19.223 us/op 18.065 us/op 1.06
Set add up to 256 items then delete last 10.668 us/op 10.982 us/op 0.97
OrderedSet add up to 256 items then delete last 15.142 us/op 15.639 us/op 0.97
Set add up to 256 items then delete middle 9.9125 us/op 10.659 us/op 0.93
OrderedSet add up to 256 items then delete middle 41.385 us/op 43.397 us/op 0.95
transfer serialized Status (84 B) 1.4320 us/op 1.5010 us/op 0.95
copy serialized Status (84 B) 1.1940 us/op 1.3340 us/op 0.90
transfer serialized SignedVoluntaryExit (112 B) 1.5960 us/op 1.6090 us/op 0.99
copy serialized SignedVoluntaryExit (112 B) 1.3140 us/op 1.3790 us/op 0.95
transfer serialized ProposerSlashing (416 B) 2.2400 us/op 2.6650 us/op 0.84
copy serialized ProposerSlashing (416 B) 2.6930 us/op 2.2190 us/op 1.21
transfer serialized Attestation (485 B) 2.7190 us/op 2.2090 us/op 1.23
copy serialized Attestation (485 B) 2.7510 us/op 1.7820 us/op 1.54
transfer serialized AttesterSlashing (33232 B) 2.8500 us/op 2.2130 us/op 1.29
copy serialized AttesterSlashing (33232 B) 6.1150 us/op 5.8300 us/op 1.05
transfer serialized Small SignedBeaconBlock (128000 B) 3.3580 us/op 2.9250 us/op 1.15
copy serialized Small SignedBeaconBlock (128000 B) 15.332 us/op 17.674 us/op 0.87
transfer serialized Avg SignedBeaconBlock (200000 B) 3.9570 us/op 3.7810 us/op 1.05
copy serialized Avg SignedBeaconBlock (200000 B) 22.123 us/op 27.377 us/op 0.81
transfer serialized BlobsSidecar (524380 B) 2.8340 us/op 3.0540 us/op 0.93
copy serialized BlobsSidecar (524380 B) 85.463 us/op 85.926 us/op 0.99
transfer serialized Big SignedBeaconBlock (1000000 B) 2.6940 us/op 3.1400 us/op 0.86
copy serialized Big SignedBeaconBlock (1000000 B) 165.41 us/op 175.55 us/op 0.94
pass gossip attestations to forkchoice per slot 2.8151 ms/op 2.9800 ms/op 0.94
forkChoice updateHead vc 100000 bc 64 eq 0 454.90 us/op 521.03 us/op 0.87
forkChoice updateHead vc 600000 bc 64 eq 0 2.8343 ms/op 3.0988 ms/op 0.91
forkChoice updateHead vc 1000000 bc 64 eq 0 4.8972 ms/op 5.4285 ms/op 0.90
forkChoice updateHead vc 600000 bc 320 eq 0 2.6612 ms/op 3.2512 ms/op 0.82
forkChoice updateHead vc 600000 bc 1200 eq 0 3.6570 ms/op 3.2064 ms/op 1.14
forkChoice updateHead vc 600000 bc 7200 eq 0 3.1497 ms/op 3.8474 ms/op 0.82
forkChoice updateHead vc 600000 bc 64 eq 1000 9.7704 ms/op 11.191 ms/op 0.87
forkChoice updateHead vc 600000 bc 64 eq 10000 9.9977 ms/op 10.959 ms/op 0.91
forkChoice updateHead vc 600000 bc 64 eq 300000 13.666 ms/op 17.729 ms/op 0.77
computeDeltas 500000 validators 300 proto nodes 3.9570 ms/op 4.5596 ms/op 0.87
computeDeltas 500000 validators 1200 proto nodes 3.8763 ms/op 4.7110 ms/op 0.82
computeDeltas 500000 validators 7200 proto nodes 3.9555 ms/op 4.4544 ms/op 0.89
computeDeltas 750000 validators 300 proto nodes 5.9667 ms/op 6.6282 ms/op 0.90
computeDeltas 750000 validators 1200 proto nodes 6.2779 ms/op 6.9695 ms/op 0.90
computeDeltas 750000 validators 7200 proto nodes 6.3065 ms/op 6.3573 ms/op 0.99
computeDeltas 1400000 validators 300 proto nodes 12.141 ms/op 12.188 ms/op 1.00
computeDeltas 1400000 validators 1200 proto nodes 11.791 ms/op 12.053 ms/op 0.98
computeDeltas 1400000 validators 7200 proto nodes 11.488 ms/op 11.957 ms/op 0.96
computeDeltas 2100000 validators 300 proto nodes 17.582 ms/op 18.514 ms/op 0.95
computeDeltas 2100000 validators 1200 proto nodes 18.173 ms/op 18.702 ms/op 0.97
computeDeltas 2100000 validators 7200 proto nodes 17.791 ms/op 18.084 ms/op 0.98
altair processAttestation - 250000 vs - 7PWei normalcase 1.8255 ms/op 1.9665 ms/op 0.93
altair processAttestation - 250000 vs - 7PWei worstcase 2.7116 ms/op 2.8666 ms/op 0.95
altair processAttestation - setStatus - 1/6 committees join 94.817 us/op 129.55 us/op 0.73
altair processAttestation - setStatus - 1/3 committees join 174.31 us/op 242.08 us/op 0.72
altair processAttestation - setStatus - 1/2 committees join 250.07 us/op 341.49 us/op 0.73
altair processAttestation - setStatus - 2/3 committees join 327.38 us/op 419.46 us/op 0.78
altair processAttestation - setStatus - 4/5 committees join 480.18 us/op 595.72 us/op 0.81
altair processAttestation - setStatus - 100% committees join 562.05 us/op 708.07 us/op 0.79
altair processBlock - 250000 vs - 7PWei normalcase 4.5763 ms/op 5.2169 ms/op 0.88
altair processBlock - 250000 vs - 7PWei normalcase hashState 23.394 ms/op 29.525 ms/op 0.79
altair processBlock - 250000 vs - 7PWei worstcase 41.176 ms/op 40.664 ms/op 1.01
altair processBlock - 250000 vs - 7PWei worstcase hashState 81.755 ms/op 91.045 ms/op 0.90
phase0 processBlock - 250000 vs - 7PWei normalcase 2.0569 ms/op 2.3635 ms/op 0.87
phase0 processBlock - 250000 vs - 7PWei worstcase 32.569 ms/op 26.114 ms/op 1.25
altair processEth1Data - 250000 vs - 7PWei normalcase 309.14 us/op 359.35 us/op 0.86
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 6.3590 us/op 7.4710 us/op 0.85
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 44.527 us/op 33.750 us/op 1.32
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 11.351 us/op 8.4740 us/op 1.34
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 5.8300 us/op 4.8900 us/op 1.19
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 179.50 us/op 120.27 us/op 1.49
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 1.2431 ms/op 1.6791 ms/op 0.74
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 2.2526 ms/op 1.5389 ms/op 1.46
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 2.3396 ms/op 2.3049 ms/op 1.02
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 3.9425 ms/op 3.8088 ms/op 1.04
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.6432 ms/op 1.6402 ms/op 1.00
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 3.6409 ms/op 3.7542 ms/op 0.97
Tree 40 250000 create 225.88 ms/op 211.67 ms/op 1.07
Tree 40 250000 get(125000) 143.89 ns/op 151.68 ns/op 0.95
Tree 40 250000 set(125000) 648.23 ns/op 677.53 ns/op 0.96
Tree 40 250000 toArray() 14.958 ms/op 17.916 ms/op 0.83
Tree 40 250000 iterate all - toArray() + loop 14.527 ms/op 19.267 ms/op 0.75
Tree 40 250000 iterate all - get(i) 48.004 ms/op 56.149 ms/op 0.85
Array 250000 create 2.7358 ms/op 3.2000 ms/op 0.85
Array 250000 clone - spread 1.3762 ms/op 1.4945 ms/op 0.92
Array 250000 get(125000) 0.40200 ns/op 0.42000 ns/op 0.96
Array 250000 set(125000) 0.42200 ns/op 0.43200 ns/op 0.98
Array 250000 iterate all - loop 103.88 us/op 84.259 us/op 1.23
phase0 afterProcessEpoch - 250000 vs - 7PWei 48.106 ms/op 51.599 ms/op 0.93
Array.fill - length 1000000 3.5579 ms/op 3.7092 ms/op 0.96
Array push - length 1000000 17.130 ms/op 17.273 ms/op 0.99
Array.get 0.26886 ns/op 0.27885 ns/op 0.96
Uint8Array.get 0.42598 ns/op 0.44267 ns/op 0.96
phase0 beforeProcessEpoch - 250000 vs - 7PWei 17.237 ms/op 18.546 ms/op 0.93
altair processEpoch - mainnet_e81889 269.53 ms/op 293.81 ms/op 0.92
mainnet_e81889 - altair beforeProcessEpoch 19.470 ms/op 20.111 ms/op 0.97
mainnet_e81889 - altair processJustificationAndFinalization 12.792 us/op 11.696 us/op 1.09
mainnet_e81889 - altair processInactivityUpdates 5.3583 ms/op 5.3652 ms/op 1.00
mainnet_e81889 - altair processRewardsAndPenalties 47.291 ms/op 62.002 ms/op 0.76
mainnet_e81889 - altair processRegistryUpdates 2.3880 us/op 2.5600 us/op 0.93
mainnet_e81889 - altair processSlashings 505.00 ns/op 754.00 ns/op 0.67
mainnet_e81889 - altair processEth1DataReset 484.00 ns/op 421.00 ns/op 1.15
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.9101 ms/op 2.7167 ms/op 0.70
mainnet_e81889 - altair processSlashingsReset 2.5580 us/op 4.8600 us/op 0.53
mainnet_e81889 - altair processRandaoMixesReset 6.4040 us/op 4.0430 us/op 1.58
mainnet_e81889 - altair processHistoricalRootsUpdate 500.00 ns/op 530.00 ns/op 0.94
mainnet_e81889 - altair processParticipationFlagUpdates 2.2590 us/op 2.4840 us/op 0.91
mainnet_e81889 - altair processSyncCommitteeUpdates 474.00 ns/op 405.00 ns/op 1.17
mainnet_e81889 - altair afterProcessEpoch 51.873 ms/op 52.309 ms/op 0.99
capella processEpoch - mainnet_e217614 1.1403 s/op 1.0940 s/op 1.04
mainnet_e217614 - capella beforeProcessEpoch 75.352 ms/op 79.480 ms/op 0.95
mainnet_e217614 - capella processJustificationAndFinalization 21.031 us/op 22.978 us/op 0.92
mainnet_e217614 - capella processInactivityUpdates 17.445 ms/op 19.241 ms/op 0.91
mainnet_e217614 - capella processRewardsAndPenalties 234.92 ms/op 259.77 ms/op 0.90
mainnet_e217614 - capella processRegistryUpdates 14.169 us/op 20.378 us/op 0.70
mainnet_e217614 - capella processSlashings 617.00 ns/op 671.00 ns/op 0.92
mainnet_e217614 - capella processEth1DataReset 1.7570 us/op 931.00 ns/op 1.89
mainnet_e217614 - capella processEffectiveBalanceUpdates 5.7871 ms/op 17.739 ms/op 0.33
mainnet_e217614 - capella processSlashingsReset 6.5520 us/op 4.4410 us/op 1.48
mainnet_e217614 - capella processRandaoMixesReset 7.2460 us/op 6.0780 us/op 1.19
mainnet_e217614 - capella processHistoricalRootsUpdate 877.00 ns/op 1.5580 us/op 0.56
mainnet_e217614 - capella processParticipationFlagUpdates 3.3910 us/op 3.4700 us/op 0.98
mainnet_e217614 - capella afterProcessEpoch 122.42 ms/op 134.52 ms/op 0.91
phase0 processEpoch - mainnet_e58758 358.73 ms/op 424.50 ms/op 0.85
mainnet_e58758 - phase0 beforeProcessEpoch 77.903 ms/op 101.42 ms/op 0.77
mainnet_e58758 - phase0 processJustificationAndFinalization 14.069 us/op 23.959 us/op 0.59
mainnet_e58758 - phase0 processRewardsAndPenalties 40.191 ms/op 40.836 ms/op 0.98
mainnet_e58758 - phase0 processRegistryUpdates 12.329 us/op 8.4590 us/op 1.46
mainnet_e58758 - phase0 processSlashings 288.00 ns/op 311.00 ns/op 0.93
mainnet_e58758 - phase0 processEth1DataReset 306.00 ns/op 333.00 ns/op 0.92
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 2.2275 ms/op 1.9738 ms/op 1.13
mainnet_e58758 - phase0 processSlashingsReset 4.2120 us/op 2.7130 us/op 1.55
mainnet_e58758 - phase0 processRandaoMixesReset 6.1150 us/op 3.7130 us/op 1.65
mainnet_e58758 - phase0 processHistoricalRootsUpdate 310.00 ns/op 534.00 ns/op 0.58
mainnet_e58758 - phase0 processParticipationRecordUpdates 2.8770 us/op 3.0450 us/op 0.94
mainnet_e58758 - phase0 afterProcessEpoch 44.053 ms/op 46.716 ms/op 0.94
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.6824 ms/op 2.2951 ms/op 0.73
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 2.2578 ms/op 2.5077 ms/op 0.90
altair processInactivityUpdates - 250000 normalcase 20.275 ms/op 18.300 ms/op 1.11
altair processInactivityUpdates - 250000 worstcase 20.967 ms/op 18.948 ms/op 1.11
phase0 processRegistryUpdates - 250000 normalcase 9.3250 us/op 9.2800 us/op 1.00
phase0 processRegistryUpdates - 250000 badcase_full_deposits 312.24 us/op 438.47 us/op 0.71
phase0 processRegistryUpdates - 250000 worstcase 0.5 137.06 ms/op 127.24 ms/op 1.08
altair processRewardsAndPenalties - 250000 normalcase 59.094 ms/op 47.304 ms/op 1.25
altair processRewardsAndPenalties - 250000 worstcase 45.360 ms/op 46.887 ms/op 0.97
phase0 getAttestationDeltas - 250000 normalcase 7.8787 ms/op 8.5917 ms/op 0.92
phase0 getAttestationDeltas - 250000 worstcase 7.7920 ms/op 7.3618 ms/op 1.06
phase0 processSlashings - 250000 worstcase 101.72 us/op 79.029 us/op 1.29
altair processSyncCommitteeUpdates - 250000 127.67 ms/op 128.35 ms/op 0.99
BeaconState.hashTreeRoot - No change 227.00 ns/op 232.00 ns/op 0.98
BeaconState.hashTreeRoot - 1 full validator 90.486 us/op 98.088 us/op 0.92
BeaconState.hashTreeRoot - 32 full validator 853.38 us/op 908.87 us/op 0.94
BeaconState.hashTreeRoot - 512 full validator 8.7167 ms/op 8.9625 ms/op 0.97
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 107.99 us/op 117.59 us/op 0.92
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.5347 ms/op 1.6150 ms/op 0.95
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 20.664 ms/op 24.140 ms/op 0.86
BeaconState.hashTreeRoot - 1 balances 79.617 us/op 108.91 us/op 0.73
BeaconState.hashTreeRoot - 32 balances 765.53 us/op 857.51 us/op 0.89
BeaconState.hashTreeRoot - 512 balances 8.4418 ms/op 6.9453 ms/op 1.22
BeaconState.hashTreeRoot - 250000 balances 197.21 ms/op 162.73 ms/op 1.21
aggregationBits - 2048 els - zipIndexesInBitList 25.105 us/op 23.960 us/op 1.05
byteArrayEquals 32 54.682 ns/op 54.208 ns/op 1.01
Buffer.compare 32 17.620 ns/op 17.229 ns/op 1.02
byteArrayEquals 1024 1.6205 us/op 1.6013 us/op 1.01
Buffer.compare 1024 26.751 ns/op 24.729 ns/op 1.08
byteArrayEquals 16384 25.865 us/op 25.550 us/op 1.01
Buffer.compare 16384 202.00 ns/op 188.22 ns/op 1.07
byteArrayEquals 123687377 195.83 ms/op 196.69 ms/op 1.00
Buffer.compare 123687377 12.928 ms/op 6.6194 ms/op 1.95
byteArrayEquals 32 - diff last byte 58.649 ns/op 53.964 ns/op 1.09
Buffer.compare 32 - diff last byte 20.668 ns/op 17.474 ns/op 1.18
byteArrayEquals 1024 - diff last byte 1.7731 us/op 1.6272 us/op 1.09
Buffer.compare 1024 - diff last byte 28.855 ns/op 25.494 ns/op 1.13
byteArrayEquals 16384 - diff last byte 27.922 us/op 25.885 us/op 1.08
Buffer.compare 16384 - diff last byte 226.74 ns/op 202.91 ns/op 1.12
byteArrayEquals 123687377 - diff last byte 195.85 ms/op 195.04 ms/op 1.00
Buffer.compare 123687377 - diff last byte 6.2932 ms/op 6.7661 ms/op 0.93
byteArrayEquals 32 - random bytes 5.0150 ns/op 5.2790 ns/op 0.95
Buffer.compare 32 - random bytes 16.724 ns/op 17.438 ns/op 0.96
byteArrayEquals 1024 - random bytes 5.0140 ns/op 5.2350 ns/op 0.96
Buffer.compare 1024 - random bytes 16.677 ns/op 17.379 ns/op 0.96
byteArrayEquals 16384 - random bytes 5.0220 ns/op 5.2480 ns/op 0.96
Buffer.compare 16384 - random bytes 16.690 ns/op 17.364 ns/op 0.96
byteArrayEquals 123687377 - random bytes 6.3400 ns/op 6.6700 ns/op 0.95
Buffer.compare 123687377 - random bytes 18.000 ns/op 18.820 ns/op 0.96
regular array get 100000 times 32.043 us/op 43.826 us/op 0.73
wrappedArray get 100000 times 32.066 us/op 34.678 us/op 0.92
arrayWithProxy get 100000 times 12.877 ms/op 14.267 ms/op 0.90
ssz.Root.equals 44.825 ns/op 46.968 ns/op 0.95
byteArrayEquals 44.232 ns/op 46.432 ns/op 0.95
Buffer.compare 10.093 ns/op 10.524 ns/op 0.96
processSlot - 1 slots 16.475 us/op 14.736 us/op 1.12
processSlot - 32 slots 3.3899 ms/op 2.8613 ms/op 1.18
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 42.527 ms/op 39.278 ms/op 1.08
getCommitteeAssignments - req 1 vs - 250000 vc 2.1195 ms/op 2.1948 ms/op 0.97
getCommitteeAssignments - req 100 vs - 250000 vc 4.0852 ms/op 4.3904 ms/op 0.93
getCommitteeAssignments - req 1000 vs - 250000 vc 4.3987 ms/op 4.6571 ms/op 0.94
findModifiedValidators - 10000 modified validators 260.01 ms/op 340.85 ms/op 0.76
findModifiedValidators - 1000 modified validators 168.59 ms/op 360.77 ms/op 0.47
findModifiedValidators - 100 modified validators 186.93 ms/op 340.60 ms/op 0.55
findModifiedValidators - 10 modified validators 152.00 ms/op 324.93 ms/op 0.47
findModifiedValidators - 1 modified validators 142.83 ms/op 308.33 ms/op 0.46
findModifiedValidators - no difference 157.14 ms/op 307.99 ms/op 0.51
compare ViewDUs 3.2038 s/op 4.1486 s/op 0.77
compare each validator Uint8Array 1.8653 s/op 1.3591 s/op 1.37
compare ViewDU to Uint8Array 970.75 ms/op 1.1488 s/op 0.85
migrate state 1000000 validators, 24 modified, 0 new 768.90 ms/op 774.22 ms/op 0.99
migrate state 1000000 validators, 1700 modified, 1000 new 1.0059 s/op 990.78 ms/op 1.02
migrate state 1000000 validators, 3400 modified, 2000 new 1.2027 s/op 1.1855 s/op 1.01
migrate state 1500000 validators, 24 modified, 0 new 800.39 ms/op 751.13 ms/op 1.07
migrate state 1500000 validators, 1700 modified, 1000 new 1.0285 s/op 988.22 ms/op 1.04
migrate state 1500000 validators, 3400 modified, 2000 new 1.2774 s/op 1.2078 s/op 1.06
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 4.3000 ns/op 5.0200 ns/op 0.86
state getBlockRootAtSlot - 250000 vs - 7PWei 877.54 ns/op 951.79 ns/op 0.92
computeProposers - vc 250000 7.0093 ms/op 7.3379 ms/op 0.96
computeEpochShuffling - vc 250000 42.498 ms/op 43.055 ms/op 0.99
getNextSyncCommittee - vc 250000 118.43 ms/op 126.95 ms/op 0.93
computeSigningRoot for AttestationData 21.400 us/op 21.816 us/op 0.98
hash AttestationData serialized data then Buffer.toString(base64) 1.5172 us/op 1.6002 us/op 0.95
toHexString serialized data 881.25 ns/op 894.20 ns/op 0.99
Buffer.toString(base64) 178.59 ns/op 206.26 ns/op 0.87
nodejs block root to RootHex using toHex 154.12 ns/op 158.86 ns/op 0.97
nodejs block root to RootHex using toRootHex 92.996 ns/op 97.418 ns/op 0.95
browser block root to RootHex using the deprecated toHexString 229.71 ns/op 233.06 ns/op 0.99
browser block root to RootHex using toHex 183.12 ns/op 188.33 ns/op 0.97
browser block root to RootHex using toRootHex 163.09 ns/op 164.80 ns/op 0.99

by benchmarkbot/action

@twoeths
Copy link
Contributor Author

twoeths commented Nov 28, 2024 •
edited
Loading

the latest commit should fix the issue for a node with existing database

@philknows or @nflaig to confirm 🙏

@twoeths twoeths marked this pull request as ready for review November 28, 2024 06:38
@twoeths twoeths requested a review from a team as a code owner November 28, 2024 06:38
nflaig
nflaig approved these changes Nov 28, 2024
View reviewed changes
Copy link
Member

@nflaig nflaig left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - I am able to sync my node to head from existing db state

@twoeths
Copy link
Contributor Author

twoeths commented Nov 28, 2024

was able to reproduce the issue and get the node synced using this branch too

@twoeths twoeths merged commit 48dea55 into unstable Nov 28, 2024
20 checks passed
@twoeths twoeths deleted the te/mekong_double_deposits branch November 28, 2024 07:20
@twoeths
Copy link
Contributor Author

twoeths commented Nov 28, 2024

found this in the log Nov-28 07:16:22.083[chain] info: Removed persisted checkpoint states from the last run count=33, maxEpochsInMemory=3

nflaig pushed a commit that referenced this pull request Nov 28, 2024
@twoeths @nflaig
fix: sync cached balance when adding new validator to registry (#7255)
7f96e70 
* fix: sync cached balance when adding new validator to registry

* chore: add more comments

* fix: remove persisted checkpoint states from the previous run at startup
@philknows philknows mentioned this pull request Nov 28, 2024
Merged
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.23.1 🎉

wemeetagain added a commit that referenced this pull request Dec 20, 2024
@wemeetagain @nflaig
@nazarhussain @g11tech @matthewkeil @ensi321 @twoeths @philknows @dependabot @acolytec3 @varunguleriaCodes @ClockworkYuzu
chore: release v1.24.0 (#7325)
11f35a8 
* feat: add keymanager endpoint to retrieve proposer config (#7210)

* feat: add keymanager endpoint to retrieve proposer config

* Do not return empty builder config

* Check all builder proposer config values

* Fix settings builder config if undefined

* Fix builder config parsing

* Use ssz type to handle json serialization

Default parsing can't handle BigInt

* Revert "Use ssz type to handle json serialization"

This reverts commit 01fcea7.

* Fix boost factor json serialization

* Remove unused import

* Update test data

* Update proposer config test

* feat: add mekong network option (#7212)

* chore: fix import order with biome syntax (#7211)

Fix import order

* fix: consistently validate pubkey and throw 404 if not found (#7214)

* Throw error if pubkey is unknown when getting graffiti

* Consistently validate pubkey and throw 404 if not found

* fix: only return local keys from /eth/v1/keystores (#7215)

* fix: only return local keys from /eth/v1/keystores

* Fix fetching remote keys in node assertion

* feat: add and use getBlobsV1 to expedite gossip import (#7134)

* hookup the getblobs api to get bob and proof data from el

remove unused

fix import

metrics overhault, test, debugging testing, some feeback

fix

add nethermind bug dicussion link

fix

resolve conflicts

* deblobs timeout

* fix metric

* chore: revert async aggregate with randomness (#7218)

Revert "feat: asyncAggregateWithRandomness (#7204)"

This reverts commit e31d535.

* fix: update config for relaunched mekong network (#7220)

* fix: light client generating `LightClientUpdate` with wrong length of branches (#7187)

* initial commit

* Rewrite SyncCommitteeWitnessRepository

* Fix finality branch

* Update unit test

* fix e2e

* Review PR

---------

Co-authored-by: Nico Flaig <[email protected]>

* fix: archive finalized state when shutting down beacon node (#7221)

* Fix typo

* feat: remove unfinalized pubkey cache (#7230)

* Remove unfinalized pubkey cache

* lint

* Fix unit test

* chore: skip web3_provider unit tests (#7252)

* fix: prune checkpoint states at syncing time (#7241)

* fix: prune checkpoint states at syncing time

* fix: lint

* fix: check-types in test

* fix: sync cached isCompoundingValidatorArr at epoch transition (#7247)

* fix: handle outOfRangeData when range sync Deneb (#7249)

* fix: handle outOfRangeData for beaconBlocksMaybeBlobsByRange()

* fix: lint

* fix: archiveBlocks - handle deneb outOfRangeData block

* fix: sync cached balance when adding new validator to registry (#7255)

* fix: sync cached balance when adding new validator to registry

* chore: add more comments

* fix: remove persisted checkpoint states from the previous run at startup

* fix: do not throw error when trying to prune missing directory (#7257)

* docs: update documentation Oct 2024 (#7178)

* docs update oct 2024 init

* Reconfig quickstart nav and minor fixes

* fix lint

* spelling fixes

* minor fixes and add to wordlist

* prettier fix

* add to wordlist

* sort wordlist

* modify dominance to include lighthouse

* fix typescript casing and add recommendation

* add selection and boost_factor with keymanager notice

* update wordlist

* remove builder enabled and add keymanager api

* spelling

---------

Co-authored-by: Nico Flaig <[email protected]>

* chore(deps): bump cross-spawn from 7.0.3 to 7.0.6 in /docs (#7268)

Bumps [cross-spawn](https://github.com/moxystudio/node-cross-spawn) from 7.0.3 to 7.0.6.
- [Changelog](https://github.com/moxystudio/node-cross-spawn/blob/master/CHANGELOG.md)
- [Commits](moxystudio/node-cross-spawn@v7.0.3...v7.0.6)

---
updated-dependencies:
- dependency-name: cross-spawn
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* feat: add error log to notifier if execution client auth failed (#7239)

* feat: add error log to notifier if execution client auth failed

* Update packages/beacon-node/src/node/notifier.ts

---------

Co-authored-by: NC <[email protected]>

* docs: display rcConfig flag on CLI reference page (#7270)

* docs: display rcConfig flag on CLI reference page

* Update word list

* chore: remove prettier as default formatter for all file types (#7275)

* chore: unhide flags relevant for devnets / testing (#7271)

* feat: debug too many shuffling promises (#7251)

* feat: add asyncShufflingCalculation to StateTransitionOpts

* feat: add asyncShufflingCalculation to all regen / processSlots consumers

* fix: default to false for async shuffling and remove unnecessary props

* fix: remove unnecessary flags from stateTransition

* feat: implement conditional build of shuffling for prepareNextSlot

* fix: spec test bug where shufflingCache is present from BeaconChain constructor

* feat: sync build next shuffling if not queued async

* fix: use getSync to pull next shuffling correctly

* docs: add comment to prepareNextSlot

* refactor: rename StateCloneOpts to StateRegenerationOpts

* feat: pass asyncShufflingCalculation through to afterProcessEpoch and refactor conditional to run purely sync

* docs: add issue number to comment

* chore: lint

* chore: unpin nodejs version from 22.4 (#6982)

* Revert "chore: pin nodejs version to 22.4 (#6964)"

This reverts commit f20484b.

* Don't revert formatting changes

---------

Co-authored-by: Nico Flaig <[email protected]>
Co-authored-by: Cayman <[email protected]>

* chore: update bootnodes file url for holesky and sepolia (#7276)

* feat: add `debug_getHistoricalSummaries` endpoint (#7245)

* feat: add new getHistoricalSummaries endpoint to debug namespace

* Add JSON response

* Restructure to use stateId and add proof to response

* add test scaffolding

* Address feedback

* Move getHistoricalSummaries to lodestar namespace

* add lodestar namespace unit test

* update route name to lodestar namespace

* cast state object as Capella state

* Lint

* json properties need to be lower case

* Make it v1 since it's now part of lodestar namespace

* Group with other /lodestar endpoints

* Simplify beacon node impl

* Rename return type

* Update test description

* Fix variable name

---------

Co-authored-by: Nico Flaig <[email protected]>

* chore: log sync committee signature errors as `error` (#7283)

* fix: update engine_getClientVersionV1 commit encoding (#7282)

* fix: check pubkey or validator index known to a state (#7284)

* fix: check pubkey or validator index known to a state

* chore: add more comments

* feat: lodestar script setup (#7254)

* feat: lodestar_setup

* feat: script_updates + docs

* feat: script_addition_in_docs + command_update

* Remove duplicate script from docs folder

* Minor script updates

* Update script to prepare docs and ignore copied file

* Update installation page

* Wording

---------

Co-authored-by: Nico Flaig <[email protected]>

* feat: add terminal-sized Electra giraffe banner (#7286)

* Create giraffeBanners.ts

* Wire in banner

* Fix file name

* lint

* Address @nflaig's comment

---------

Co-authored-by: NC <[email protected]>

* chore: pin nodejs version to 22.4 (#7291)

Revert "chore: unpin nodejs version from 22.4 (#6982)"

This reverts commit 69ae688.

* feat: expose `DOMAIN_APPLICATION_MASK` in config/spec api (#7296)

* feat: expose DOMAIN_APPLICATION_MASK in config/spec api

* Lint

* feat: make `MAX_REQUEST_BLOB_SIDECARS` and `MAX_BLOBS_PER_BLOCK` configurable (#7294)

* Init commit

* Fix check-types

* Add comment on how MAX_REQUEST_BLOB_SIDECARS is calculated

* Ensure proper config object is passed

* Address comment

---------

Co-authored-by: Nico Flaig <[email protected]>

* feat: use `BLOB_SIDECAR_SUBNET_COUNT` to configure blob subnets (#7297)

feat: use BLOB_SIDECAR_SUBNET_COUNT to configure blob subnets

* chore: log sync aggregate participants when producing beacon block body (#7300)

* chore: log sync aggregate participants when producing beacon block body

* Use isForkLightClient instead of ForkSeq

* Fix produce block unit tests

* chore: print graffiti when producing beacon block body (#7303)

* fix: warn if engine / builder failed to produce block within cutoff time (#7305)

* feat: add kzg commitment length check when validating gossip blocks (#7302)

* feat: add blob sidecar index check (#7313)

Validate blobSidecar index

* fix: fix blob sidecar index check (#7315)

Fix index check

* chore: fix format of printed graffiti from hex to utf-8 (#7306)

* chore: fix format of printed graffiti from hex to utf-8

* Use Buffer.from no copy with offset

* docs: batch commit typos and update contributor readme (#7312)

* batch commit typos and update contributor readme

* update donation text

Co-authored-by: Nico Flaig <[email protected]>

* correct spelling

Co-authored-by: Nico Flaig <[email protected]>

---------

Co-authored-by: Nico Flaig <[email protected]>

* chore: remove trailing null bytes from printed graffiti (#7320)

* chore: remove trailing null bytes from printed graffiti

* Use replaceAll instead of regex

* chore: unpin nodejs version from 22.4 (#7324)

Revert "chore: pin nodejs version to 22.4 (#7291)"

This reverts commit 99794d3.

* chore: bump package versions to 1.24.0

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Nico Flaig <[email protected]>
Co-authored-by: Nazar Hussain <[email protected]>
Co-authored-by: g11tech <[email protected]>
Co-authored-by: Matthew Keil <[email protected]>
Co-authored-by: NC <[email protected]>
Co-authored-by: twoeths <[email protected]>
Co-authored-by: Phil Ngo <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: acolytec3 <[email protected]>
Co-authored-by: Varun Guleria <[email protected]>
Co-authored-by: ClockworkYuzu <[email protected]>
@wemeetagain
Copy link
Member

🎉 This PR is included in v1.24.0 🎉

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nflaig nflaig nflaig approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@twoeths @wemeetagain @nflaig