Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: commit pending partial withdrawals after adding new entry #7375

Draft
wants to merge 1 commit into
base: unstable
Choose a base branch
from
Draft

fix: commit pending partial withdrawals after adding new entry #7375

wants to merge 1 commit into from
+1 −0

Conversation

nflaig
Copy link
Member

@nflaig nflaig commented Jan 19, 2025

Fixes devnet-5 issue

Jan-19 05:30:54.001[]                 info: Synced - slot: 19199 - head: 0xb612…f518 - exec-block: valid(16077 0x5a23…) - finalized: 0x2839…bb9d:597 - peers: 41
Jan-19 05:31:06.001[]                 info: Synced - slot: 19200 - head: 0x24e9…9b84 - exec-block: valid(16078 0x89e6…) - finalized: 0x5142…f40c:598 - peers: 41
Jan-19 05:31:12.288[network]          warn: Error receiving block slot=19201, peer=16Uiu2HAkxHheop9frWVri98J4bnncEbXH8R23y1wRGmCkvwHBzBA, code=BLOCK_ERROR_BEACON_CHAIN_ERROR, error=Must commit changes before reading all nodes
Error: Must commit changes before reading all nodes
    at ListCompositeTreeViewDU.populateAllNodes (/usr/app/node_modules/@chainsafe/ssz/src/viewDU/arrayComposite.ts:251:13)
    at ListCompositeTreeViewDU.getAllReadonly (/usr/app/node_modules/@chainsafe/ssz/src/viewDU/arrayComposite.ts:151:10)
    at getPendingBalanceToWithdraw (file:///usr/app/packages/state-transition/src/util/validator.ts:87:6)
    at processWithdrawalRequest (file:///usr/app/packages/state-transition/src/block/processWithdrawalRequest.ts:46:36)
    at processOperations (file:///usr/app/packages/state-transition/src/block/processOperations.ts:75:7)
    at processBlock (file:///usr/app/packages/state-transition/src/block/index.ts:61:3)
    at stateTransition (file:///usr/app/packages/state-transition/src/stateTransition.ts:106:3)
    at verifyBlocksStateTransitionOnly (file:///usr/app/packages/beacon-node/src/chain/blocks/verifyBlocksStateTransitionOnly.ts:46:23)
    at BeaconChain.verifyBlocksInEpoch (file:///usr/app/packages/beacon-node/src/chain/blocks/verifyBlock.ts:114:7)
    at processTicksAndRejections (node:internal/process/task_queues:105:5)
Jan-19 05:31:18.001[]                 info: Synced - slot: 19201 - head: (slot -1) 0x24e9…9b84 - exec-block: valid(16078 0x89e6…) - finalized: 0x5142…f40c:598 - peers: 41

which happened when processing block with two withdrawal requests.

See discord for more context.

@nflaig nflaig marked this pull request as ready for review January 19, 2025 21:15
@nflaig nflaig requested a review from a team as a code owner January 19, 2025 21:15
@codecov Codecov
Copy link

codecov bot commented Jan 19, 2025

Codecov Report

Attention: Patch coverage is 0% with 1 line in your changes missing coverage. Please review.

Project coverage is 48.62%. Comparing base (06831cf) to head (6658d29).

Additional details and impacted files 
@@             Coverage Diff              @@
##           unstable    #7375      +/-   ##
============================================
- Coverage     48.62%   48.62%   -0.01%     
============================================
  Files           603      603              
  Lines         40516    40517       +1     
  Branches       2071     2071              
============================================
  Hits          19700    19700              
- Misses        20778    20779       +1     
  Partials         38       38

@github-actions GitHub Actions
Copy link
Contributor

Performance Report

✔️ no performance regression detected

🚀🚀 Significant benchmark improvement detected

Benchmark suite Current: 0771fb2 Previous: 06831cf Ratio
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 3.2070 us/op 11.277 us/op 0.28
Full benchmark results
Benchmark suite Current: 0771fb2 Previous: 06831cf Ratio
getPubkeys - index2pubkey - req 1000 vs - 250000 vc 1.6348 ms/op 1.7507 ms/op 0.93
getPubkeys - validatorsArr - req 1000 vs - 250000 vc 46.982 us/op 46.950 us/op 1.00
BLS verify - blst 777.37 us/op 983.98 us/op 0.79
BLS verifyMultipleSignatures 3 - blst 1.2931 ms/op 1.2295 ms/op 1.05
BLS verifyMultipleSignatures 8 - blst 2.0842 ms/op 1.8946 ms/op 1.10
BLS verifyMultipleSignatures 32 - blst 4.4764 ms/op 5.6728 ms/op 0.79
BLS verifyMultipleSignatures 64 - blst 8.3233 ms/op 11.067 ms/op 0.75
BLS verifyMultipleSignatures 128 - blst 16.016 ms/op 17.534 ms/op 0.91
BLS deserializing 10000 signatures 629.37 ms/op 696.53 ms/op 0.90
BLS deserializing 100000 signatures 6.2508 s/op 6.9720 s/op 0.90
BLS verifyMultipleSignatures - same message - 3 - blst 940.93 us/op 923.99 us/op 1.02
BLS verifyMultipleSignatures - same message - 8 - blst 1.0838 ms/op 1.1069 ms/op 0.98
BLS verifyMultipleSignatures - same message - 32 - blst 1.6728 ms/op 1.8766 ms/op 0.89
BLS verifyMultipleSignatures - same message - 64 - blst 2.4913 ms/op 2.6411 ms/op 0.94
BLS verifyMultipleSignatures - same message - 128 - blst 4.0106 ms/op 4.4469 ms/op 0.90
BLS aggregatePubkeys 32 - blst 17.210 us/op 19.903 us/op 0.86
BLS aggregatePubkeys 128 - blst 60.554 us/op 71.477 us/op 0.85
notSeenSlots=1 numMissedVotes=1 numBadVotes=10 89.389 ms/op 61.814 ms/op 1.45
notSeenSlots=1 numMissedVotes=0 numBadVotes=4 42.610 ms/op 59.550 ms/op 0.72
notSeenSlots=2 numMissedVotes=1 numBadVotes=10 42.432 ms/op 49.640 ms/op 0.85
getSlashingsAndExits - default max 108.11 us/op 131.79 us/op 0.82
getSlashingsAndExits - 2k 433.02 us/op 434.87 us/op 1.00
proposeBlockBody type=full, size=empty 4.4495 ms/op 5.7538 ms/op 0.77
isKnown best case - 1 super set check 451.00 ns/op 269.00 ns/op 1.68
isKnown normal case - 2 super set checks 447.00 ns/op 258.00 ns/op 1.73
isKnown worse case - 16 super set checks 448.00 ns/op 259.00 ns/op 1.73
InMemoryCheckpointStateCache - add get delete 2.5280 us/op 2.5510 us/op 0.99
validate api signedAggregateAndProof - struct 1.4282 ms/op 1.4694 ms/op 0.97
validate gossip signedAggregateAndProof - struct 1.4758 ms/op 1.4483 ms/op 1.02
batch validate gossip attestation - vc 640000 - chunk 32 119.44 us/op 127.67 us/op 0.94
batch validate gossip attestation - vc 640000 - chunk 64 105.54 us/op 112.44 us/op 0.94
batch validate gossip attestation - vc 640000 - chunk 128 95.425 us/op 103.46 us/op 0.92
batch validate gossip attestation - vc 640000 - chunk 256 93.733 us/op 110.75 us/op 0.85
pickEth1Vote - no votes 840.13 us/op 1.0688 ms/op 0.79
pickEth1Vote - max votes 5.4154 ms/op 6.9442 ms/op 0.78
pickEth1Vote - Eth1Data hashTreeRoot value x2048 13.914 ms/op 20.736 ms/op 0.67
pickEth1Vote - Eth1Data hashTreeRoot tree x2048 20.610 ms/op 25.418 ms/op 0.81
pickEth1Vote - Eth1Data fastSerialize value x2048 353.92 us/op 515.37 us/op 0.69
pickEth1Vote - Eth1Data fastSerialize tree x2048 4.3575 ms/op 3.3694 ms/op 1.29
bytes32 toHexString 518.00 ns/op 440.00 ns/op 1.18
bytes32 Buffer.toString(hex) 399.00 ns/op 234.00 ns/op 1.71
bytes32 Buffer.toString(hex) from Uint8Array 490.00 ns/op 364.00 ns/op 1.35
bytes32 Buffer.toString(hex) + 0x 396.00 ns/op 235.00 ns/op 1.69
Object access 1 prop 0.30100 ns/op 0.12700 ns/op 2.37
Map access 1 prop 0.33200 ns/op 0.13000 ns/op 2.55
Object get x1000 4.8070 ns/op 5.7080 ns/op 0.84
Map get x1000 5.6150 ns/op 6.1850 ns/op 0.91
Object set x1000 22.205 ns/op 33.875 ns/op 0.66
Map set x1000 18.394 ns/op 23.656 ns/op 0.78
Return object 10000 times 0.28410 ns/op 0.29340 ns/op 0.97
Throw Error 10000 times 2.6610 us/op 3.3922 us/op 0.78
toHex 99.501 ns/op 174.14 ns/op 0.57
Buffer.from 90.755 ns/op 148.31 ns/op 0.61
shared Buffer 60.611 ns/op 101.62 ns/op 0.60
fastMsgIdFn sha256 / 200 bytes 1.8780 us/op 2.2980 us/op 0.82
fastMsgIdFn h32 xxhash / 200 bytes 388.00 ns/op 259.00 ns/op 1.50
fastMsgIdFn h64 xxhash / 200 bytes 438.00 ns/op 271.00 ns/op 1.62
fastMsgIdFn sha256 / 1000 bytes 5.5490 us/op 7.5540 us/op 0.73
fastMsgIdFn h32 xxhash / 1000 bytes 510.00 ns/op 396.00 ns/op 1.29
fastMsgIdFn h64 xxhash / 1000 bytes 500.00 ns/op 346.00 ns/op 1.45
fastMsgIdFn sha256 / 10000 bytes 46.697 us/op 64.768 us/op 0.72
fastMsgIdFn h32 xxhash / 10000 bytes 1.8410 us/op 1.8410 us/op 1.00
fastMsgIdFn h64 xxhash / 10000 bytes 1.2930 us/op 1.1980 us/op 1.08
send data - 1000 256B messages 9.5036 ms/op 11.654 ms/op 0.82
send data - 1000 512B messages 13.271 ms/op 18.581 ms/op 0.71
send data - 1000 1024B messages 20.846 ms/op 25.561 ms/op 0.82
send data - 1000 1200B messages 22.923 ms/op 26.575 ms/op 0.86
send data - 1000 2048B messages 28.689 ms/op 33.108 ms/op 0.87
send data - 1000 4096B messages 26.152 ms/op 31.326 ms/op 0.83
send data - 1000 16384B messages 58.961 ms/op 73.747 ms/op 0.80
send data - 1000 65536B messages 228.99 ms/op 201.74 ms/op 1.14
enrSubnets - fastDeserialize 64 bits 1.1660 us/op 1.0680 us/op 1.09
enrSubnets - ssz BitVector 64 bits 499.00 ns/op 354.00 ns/op 1.41
enrSubnets - fastDeserialize 4 bits 334.00 ns/op 148.00 ns/op 2.26
enrSubnets - ssz BitVector 4 bits 507.00 ns/op 342.00 ns/op 1.48
prioritizePeers score -10:0 att 32-0.1 sync 2-0 109.56 us/op 143.77 us/op 0.76
prioritizePeers score 0:0 att 32-0.25 sync 2-0.25 132.16 us/op 180.15 us/op 0.73
prioritizePeers score 0:0 att 32-0.5 sync 2-0.5 192.52 us/op 239.65 us/op 0.80
prioritizePeers score 0:0 att 64-0.75 sync 4-0.75 345.12 us/op 395.31 us/op 0.87
prioritizePeers score 0:0 att 64-1 sync 4-1 411.48 us/op 523.13 us/op 0.79
array of 16000 items push then shift 1.3085 us/op 1.5830 us/op 0.83
LinkedList of 16000 items push then shift 6.5010 ns/op 6.9440 ns/op 0.94
array of 16000 items push then pop 77.086 ns/op 90.495 ns/op 0.85
LinkedList of 16000 items push then pop 6.3320 ns/op 6.8460 ns/op 0.92
array of 24000 items push then shift 1.8998 us/op 2.3390 us/op 0.81
LinkedList of 24000 items push then shift 7.0100 ns/op 6.9440 ns/op 1.01
array of 24000 items push then pop 103.11 ns/op 130.04 ns/op 0.79
LinkedList of 24000 items push then pop 6.5370 ns/op 7.1690 ns/op 0.91
intersect bitArray bitLen 8 5.6810 ns/op 6.5370 ns/op 0.87
intersect array and set length 8 37.940 ns/op 44.962 ns/op 0.84
intersect bitArray bitLen 128 26.834 ns/op 29.862 ns/op 0.90
intersect array and set length 128 579.84 ns/op 661.62 ns/op 0.88
bitArray.getTrueBitIndexes() bitLen 128 1.4320 us/op 1.4280 us/op 1.00
bitArray.getTrueBitIndexes() bitLen 248 2.0430 us/op 2.3110 us/op 0.88
bitArray.getTrueBitIndexes() bitLen 512 3.8470 us/op 4.5870 us/op 0.84
Buffer.concat 32 items 915.00 ns/op 855.00 ns/op 1.07
Uint8Array.set 32 items 1.6800 us/op 1.6460 us/op 1.02
Buffer.copy 2.6480 us/op 2.7200 us/op 0.97
Uint8Array.set - with subarray 2.3380 us/op 2.2170 us/op 1.05
Uint8Array.set - without subarray 1.6230 us/op 1.4700 us/op 1.10
getUint32 - dataview 390.00 ns/op 229.00 ns/op 1.70
getUint32 - manual 330.00 ns/op 153.00 ns/op 2.16
Set add up to 64 items then delete first 1.7774 us/op 2.1612 us/op 0.82
OrderedSet add up to 64 items then delete first 2.7630 us/op 3.2004 us/op 0.86
Set add up to 64 items then delete last 2.0210 us/op 2.4304 us/op 0.83
OrderedSet add up to 64 items then delete last 3.0293 us/op 3.5876 us/op 0.84
Set add up to 64 items then delete middle 2.0231 us/op 2.4918 us/op 0.81
OrderedSet add up to 64 items then delete middle 4.4281 us/op 5.1155 us/op 0.87
Set add up to 128 items then delete first 4.0164 us/op 5.1838 us/op 0.77
OrderedSet add up to 128 items then delete first 6.2458 us/op 7.5986 us/op 0.82
Set add up to 128 items then delete last 3.8581 us/op 4.8289 us/op 0.80
OrderedSet add up to 128 items then delete last 5.8472 us/op 7.2762 us/op 0.80
Set add up to 128 items then delete middle 3.8448 us/op 4.8385 us/op 0.79
OrderedSet add up to 128 items then delete middle 11.710 us/op 13.616 us/op 0.86
Set add up to 256 items then delete first 7.7939 us/op 10.098 us/op 0.77
OrderedSet add up to 256 items then delete first 12.270 us/op 15.869 us/op 0.77
Set add up to 256 items then delete last 7.5437 us/op 9.6884 us/op 0.78
OrderedSet add up to 256 items then delete last 11.541 us/op 14.538 us/op 0.79
Set add up to 256 items then delete middle 7.5476 us/op 9.5623 us/op 0.79
OrderedSet add up to 256 items then delete middle 34.006 us/op 40.974 us/op 0.83
transfer serialized Status (84 B) 2.2210 us/op 2.5380 us/op 0.88
copy serialized Status (84 B) 1.3750 us/op 1.2440 us/op 1.11
transfer serialized SignedVoluntaryExit (112 B) 2.3830 us/op 2.5400 us/op 0.94
copy serialized SignedVoluntaryExit (112 B) 1.4520 us/op 1.2680 us/op 1.15
transfer serialized ProposerSlashing (416 B) 2.7350 us/op 2.6400 us/op 1.04
copy serialized ProposerSlashing (416 B) 2.1390 us/op 1.4000 us/op 1.53
transfer serialized Attestation (485 B) 2.8070 us/op 2.8490 us/op 0.99
copy serialized Attestation (485 B) 2.1610 us/op 1.9690 us/op 1.10
transfer serialized AttesterSlashing (33232 B) 3.5830 us/op 3.2910 us/op 1.09
copy serialized AttesterSlashing (33232 B) 4.7460 us/op 5.0310 us/op 0.94
transfer serialized Small SignedBeaconBlock (128000 B) 2.8380 us/op 4.4830 us/op 0.63
copy serialized Small SignedBeaconBlock (128000 B) 12.348 us/op 14.137 us/op 0.87
transfer serialized Avg SignedBeaconBlock (200000 B) 3.4730 us/op 4.8160 us/op 0.72
copy serialized Avg SignedBeaconBlock (200000 B) 11.946 us/op 19.553 us/op 0.61
transfer serialized BlobsSidecar (524380 B) 3.5140 us/op 4.0750 us/op 0.86
copy serialized BlobsSidecar (524380 B) 82.139 us/op 177.96 us/op 0.46
transfer serialized Big SignedBeaconBlock (1000000 B) 4.3940 us/op 4.3430 us/op 1.01
copy serialized Big SignedBeaconBlock (1000000 B) 128.47 us/op 150.86 us/op 0.85
pass gossip attestations to forkchoice per slot 2.3162 ms/op 2.7567 ms/op 0.84
forkChoice updateHead vc 100000 bc 64 eq 0 388.11 us/op 456.67 us/op 0.85
forkChoice updateHead vc 600000 bc 64 eq 0 2.5151 ms/op 3.3429 ms/op 0.75
forkChoice updateHead vc 1000000 bc 64 eq 0 4.1058 ms/op 4.8417 ms/op 0.85
forkChoice updateHead vc 600000 bc 320 eq 0 2.3330 ms/op 2.8573 ms/op 0.82
forkChoice updateHead vc 600000 bc 1200 eq 0 2.4170 ms/op 2.8911 ms/op 0.84
forkChoice updateHead vc 600000 bc 7200 eq 0 2.6408 ms/op 3.6141 ms/op 0.73
forkChoice updateHead vc 600000 bc 64 eq 1000 9.4318 ms/op 10.534 ms/op 0.90
forkChoice updateHead vc 600000 bc 64 eq 10000 9.3473 ms/op 10.486 ms/op 0.89
forkChoice updateHead vc 600000 bc 64 eq 300000 11.344 ms/op 14.028 ms/op 0.81
computeDeltas 500000 validators 300 proto nodes 3.3570 ms/op 4.1342 ms/op 0.81
computeDeltas 500000 validators 1200 proto nodes 3.3718 ms/op 4.2797 ms/op 0.79
computeDeltas 500000 validators 7200 proto nodes 3.3294 ms/op 5.0443 ms/op 0.66
computeDeltas 750000 validators 300 proto nodes 5.0374 ms/op 6.4580 ms/op 0.78
computeDeltas 750000 validators 1200 proto nodes 4.8982 ms/op 6.4997 ms/op 0.75
computeDeltas 750000 validators 7200 proto nodes 5.0763 ms/op 6.6634 ms/op 0.76
computeDeltas 1400000 validators 300 proto nodes 9.2338 ms/op 12.145 ms/op 0.76
computeDeltas 1400000 validators 1200 proto nodes 9.1995 ms/op 12.664 ms/op 0.73
computeDeltas 1400000 validators 7200 proto nodes 9.1898 ms/op 12.881 ms/op 0.71
computeDeltas 2100000 validators 300 proto nodes 13.688 ms/op 20.489 ms/op 0.67
computeDeltas 2100000 validators 1200 proto nodes 13.729 ms/op 19.227 ms/op 0.71
computeDeltas 2100000 validators 7200 proto nodes 13.692 ms/op 21.764 ms/op 0.63
altair processAttestation - 250000 vs - 7PWei normalcase 1.5424 ms/op 3.1560 ms/op 0.49
altair processAttestation - 250000 vs - 7PWei worstcase 2.3094 ms/op 3.9778 ms/op 0.58
altair processAttestation - setStatus - 1/6 committees join 84.174 us/op 152.67 us/op 0.55
altair processAttestation - setStatus - 1/3 committees join 173.96 us/op 285.59 us/op 0.61
altair processAttestation - setStatus - 1/2 committees join 267.33 us/op 387.68 us/op 0.69
altair processAttestation - setStatus - 2/3 committees join 356.19 us/op 517.32 us/op 0.69
altair processAttestation - setStatus - 4/5 committees join 493.23 us/op 691.87 us/op 0.71
altair processAttestation - setStatus - 100% committees join 583.02 us/op 764.08 us/op 0.76
altair processBlock - 250000 vs - 7PWei normalcase 3.3254 ms/op 5.2380 ms/op 0.63
altair processBlock - 250000 vs - 7PWei normalcase hashState 23.467 ms/op 29.182 ms/op 0.80
altair processBlock - 250000 vs - 7PWei worstcase 34.303 ms/op 41.337 ms/op 0.83
altair processBlock - 250000 vs - 7PWei worstcase hashState 73.527 ms/op 87.230 ms/op 0.84
phase0 processBlock - 250000 vs - 7PWei normalcase 1.7201 ms/op 2.0467 ms/op 0.84
phase0 processBlock - 250000 vs - 7PWei worstcase 23.697 ms/op 26.573 ms/op 0.89
altair processEth1Data - 250000 vs - 7PWei normalcase 262.10 us/op 448.01 us/op 0.59
getExpectedWithdrawals 250000 eb:1,eth1:1,we:0,wn:0,smpl:15 3.2070 us/op 11.277 us/op 0.28
getExpectedWithdrawals 250000 eb:0.95,eth1:0.1,we:0.05,wn:0,smpl:219 36.125 us/op 58.652 us/op 0.62
getExpectedWithdrawals 250000 eb:0.95,eth1:0.3,we:0.05,wn:0,smpl:42 12.343 us/op 23.448 us/op 0.53
getExpectedWithdrawals 250000 eb:0.95,eth1:0.7,we:0.05,wn:0,smpl:18 8.4910 us/op 11.999 us/op 0.71
getExpectedWithdrawals 250000 eb:0.1,eth1:0.1,we:0,wn:0,smpl:1020 184.37 us/op 229.13 us/op 0.80
getExpectedWithdrawals 250000 eb:0.03,eth1:0.03,we:0,wn:0,smpl:11777 970.73 us/op 1.9461 ms/op 0.50
getExpectedWithdrawals 250000 eb:0.01,eth1:0.01,we:0,wn:0,smpl:16384 1.2665 ms/op 2.1222 ms/op 0.60
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,smpl:16384 1.4219 ms/op 1.9983 ms/op 0.71
getExpectedWithdrawals 250000 eb:0,eth1:0,we:0,wn:0,nocache,smpl:16384 3.4832 ms/op 6.2819 ms/op 0.55
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,smpl:16384 1.4647 ms/op 2.0028 ms/op 0.73
getExpectedWithdrawals 250000 eb:0,eth1:1,we:0,wn:0,nocache,smpl:16384 3.5233 ms/op 6.0817 ms/op 0.58
Tree 40 250000 create 183.04 ms/op 280.07 ms/op 0.65
Tree 40 250000 get(125000) 112.63 ns/op 160.03 ns/op 0.70
Tree 40 250000 set(125000) 542.23 ns/op 702.79 ns/op 0.77
Tree 40 250000 toArray() 12.642 ms/op 21.092 ms/op 0.60
Tree 40 250000 iterate all - toArray() + loop 12.461 ms/op 20.966 ms/op 0.59
Tree 40 250000 iterate all - get(i) 38.734 ms/op 58.315 ms/op 0.66
Array 250000 create 2.3701 ms/op 3.8893 ms/op 0.61
Array 250000 clone - spread 1.2506 ms/op 1.4673 ms/op 0.85
Array 250000 get(125000) 0.56000 ns/op 0.43300 ns/op 1.29
Array 250000 set(125000) 0.57200 ns/op 0.46100 ns/op 1.24
Array 250000 iterate all - loop 74.033 us/op 103.87 us/op 0.71
phase0 afterProcessEpoch - 250000 vs - 7PWei 40.258 ms/op 52.860 ms/op 0.76
Array.fill - length 1000000 2.4843 ms/op 3.9239 ms/op 0.63
Array push - length 1000000 14.237 ms/op 21.329 ms/op 0.67
Array.get 0.26361 ns/op 0.28924 ns/op 0.91
Uint8Array.get 0.34286 ns/op 0.45635 ns/op 0.75
phase0 beforeProcessEpoch - 250000 vs - 7PWei 13.097 ms/op 21.419 ms/op 0.61
altair processEpoch - mainnet_e81889 283.45 ms/op 286.89 ms/op 0.99
mainnet_e81889 - altair beforeProcessEpoch 18.513 ms/op 18.919 ms/op 0.98
mainnet_e81889 - altair processJustificationAndFinalization 11.001 us/op 23.766 us/op 0.46
mainnet_e81889 - altair processInactivityUpdates 3.8922 ms/op 5.4120 ms/op 0.72
mainnet_e81889 - altair processRewardsAndPenalties 51.670 ms/op 53.390 ms/op 0.97
mainnet_e81889 - altair processRegistryUpdates 2.4880 us/op 4.2250 us/op 0.59
mainnet_e81889 - altair processSlashings 828.00 ns/op 626.00 ns/op 1.32
mainnet_e81889 - altair processEth1DataReset 858.00 ns/op 885.00 ns/op 0.97
mainnet_e81889 - altair processEffectiveBalanceUpdates 1.6250 ms/op 1.2921 ms/op 1.26
mainnet_e81889 - altair processSlashingsReset 2.7060 us/op 7.5750 us/op 0.36
mainnet_e81889 - altair processRandaoMixesReset 3.5420 us/op 8.0250 us/op 0.44
mainnet_e81889 - altair processHistoricalRootsUpdate 1.1190 us/op 938.00 ns/op 1.19
mainnet_e81889 - altair processParticipationFlagUpdates 2.0870 us/op 3.5730 us/op 0.58
mainnet_e81889 - altair processSyncCommitteeUpdates 896.00 ns/op 1.1900 us/op 0.75
mainnet_e81889 - altair afterProcessEpoch 41.052 ms/op 52.185 ms/op 0.79
capella processEpoch - mainnet_e217614 1.0121 s/op 1.0619 s/op 0.95
mainnet_e217614 - capella beforeProcessEpoch 64.668 ms/op 71.217 ms/op 0.91
mainnet_e217614 - capella processJustificationAndFinalization 10.387 us/op 17.484 us/op 0.59
mainnet_e217614 - capella processInactivityUpdates 14.666 ms/op 17.756 ms/op 0.83
mainnet_e217614 - capella processRewardsAndPenalties 246.89 ms/op 245.55 ms/op 1.01
mainnet_e217614 - capella processRegistryUpdates 15.230 us/op 22.987 us/op 0.66
mainnet_e217614 - capella processSlashings 869.00 ns/op 716.00 ns/op 1.21
mainnet_e217614 - capella processEth1DataReset 809.00 ns/op 601.00 ns/op 1.35
mainnet_e217614 - capella processEffectiveBalanceUpdates 3.6121 ms/op 14.715 ms/op 0.25
mainnet_e217614 - capella processSlashingsReset 1.5020 us/op 5.6120 us/op 0.27
mainnet_e217614 - capella processRandaoMixesReset 3.4360 us/op 6.8050 us/op 0.50
mainnet_e217614 - capella processHistoricalRootsUpdate 753.00 ns/op 1.1240 us/op 0.67
mainnet_e217614 - capella processParticipationFlagUpdates 3.4930 us/op 1.9270 us/op 1.81
mainnet_e217614 - capella afterProcessEpoch 104.49 ms/op 123.12 ms/op 0.85
phase0 processEpoch - mainnet_e58758 316.27 ms/op 350.83 ms/op 0.90
mainnet_e58758 - phase0 beforeProcessEpoch 68.377 ms/op 81.401 ms/op 0.84
mainnet_e58758 - phase0 processJustificationAndFinalization 9.8790 us/op 21.000 us/op 0.47
mainnet_e58758 - phase0 processRewardsAndPenalties 30.999 ms/op 31.468 ms/op 0.99
mainnet_e58758 - phase0 processRegistryUpdates 7.1830 us/op 12.887 us/op 0.56
mainnet_e58758 - phase0 processSlashings 851.00 ns/op 648.00 ns/op 1.31
mainnet_e58758 - phase0 processEth1DataReset 793.00 ns/op 559.00 ns/op 1.42
mainnet_e58758 - phase0 processEffectiveBalanceUpdates 1.3373 ms/op 1.4568 ms/op 0.92
mainnet_e58758 - phase0 processSlashingsReset 1.3410 us/op 3.2670 us/op 0.41
mainnet_e58758 - phase0 processRandaoMixesReset 1.9850 us/op 5.4580 us/op 0.36
mainnet_e58758 - phase0 processHistoricalRootsUpdate 703.00 ns/op 642.00 ns/op 1.10
mainnet_e58758 - phase0 processParticipationRecordUpdates 3.4170 us/op 3.8330 us/op 0.89
mainnet_e58758 - phase0 afterProcessEpoch 36.535 ms/op 43.501 ms/op 0.84
phase0 processEffectiveBalanceUpdates - 250000 normalcase 1.6886 ms/op 2.1973 ms/op 0.77
phase0 processEffectiveBalanceUpdates - 250000 worstcase 0.5 1.2553 ms/op 2.7974 ms/op 0.45
altair processInactivityUpdates - 250000 normalcase 15.898 ms/op 20.705 ms/op 0.77
altair processInactivityUpdates - 250000 worstcase 16.776 ms/op 20.126 ms/op 0.83
phase0 processRegistryUpdates - 250000 normalcase 6.5270 us/op 11.566 us/op 0.56
phase0 processRegistryUpdates - 250000 badcase_full_deposits 342.09 us/op 485.30 us/op 0.70
phase0 processRegistryUpdates - 250000 worstcase 0.5 108.91 ms/op 108.23 ms/op 1.01
altair processRewardsAndPenalties - 250000 normalcase 51.030 ms/op 39.966 ms/op 1.28
altair processRewardsAndPenalties - 250000 worstcase 37.068 ms/op 49.152 ms/op 0.75
phase0 getAttestationDeltas - 250000 normalcase 5.7269 ms/op 7.4544 ms/op 0.77
phase0 getAttestationDeltas - 250000 worstcase 5.7150 ms/op 6.9910 ms/op 0.82
phase0 processSlashings - 250000 worstcase 87.217 us/op 112.31 us/op 0.78
altair processSyncCommitteeUpdates - 250000 98.728 ms/op 120.03 ms/op 0.82
BeaconState.hashTreeRoot - No change 456.00 ns/op 265.00 ns/op 1.72
BeaconState.hashTreeRoot - 1 full validator 116.40 us/op 114.36 us/op 1.02
BeaconState.hashTreeRoot - 32 full validator 1.4968 ms/op 1.1875 ms/op 1.26
BeaconState.hashTreeRoot - 512 full validator 11.185 ms/op 9.1215 ms/op 1.23
BeaconState.hashTreeRoot - 1 validator.effectiveBalance 126.63 us/op 111.50 us/op 1.14
BeaconState.hashTreeRoot - 32 validator.effectiveBalance 1.8148 ms/op 1.7457 ms/op 1.04
BeaconState.hashTreeRoot - 512 validator.effectiveBalance 19.561 ms/op 19.844 ms/op 0.99
BeaconState.hashTreeRoot - 1 balances 97.050 us/op 101.91 us/op 0.95
BeaconState.hashTreeRoot - 32 balances 945.23 us/op 752.61 us/op 1.26
BeaconState.hashTreeRoot - 512 balances 7.2217 ms/op 6.9835 ms/op 1.03
BeaconState.hashTreeRoot - 250000 balances 160.58 ms/op 177.07 ms/op 0.91
aggregationBits - 2048 els - zipIndexesInBitList 20.858 us/op 24.421 us/op 0.85
byteArrayEquals 32 46.819 ns/op 53.382 ns/op 0.88
Buffer.compare 32 15.034 ns/op 17.055 ns/op 0.88
byteArrayEquals 1024 1.2409 us/op 1.5949 us/op 0.78
Buffer.compare 1024 23.136 ns/op 25.418 ns/op 0.91
byteArrayEquals 16384 19.676 us/op 25.274 us/op 0.78
Buffer.compare 16384 200.45 ns/op 192.47 ns/op 1.04
byteArrayEquals 123687377 148.42 ms/op 189.85 ms/op 0.78
Buffer.compare 123687377 5.3796 ms/op 6.7574 ms/op 0.80
byteArrayEquals 32 - diff last byte 45.661 ns/op 53.723 ns/op 0.85
Buffer.compare 32 - diff last byte 15.140 ns/op 18.233 ns/op 0.83
byteArrayEquals 1024 - diff last byte 1.2019 us/op 1.6103 us/op 0.75
Buffer.compare 1024 - diff last byte 22.893 ns/op 25.523 ns/op 0.90
byteArrayEquals 16384 - diff last byte 19.037 us/op 25.601 us/op 0.74
Buffer.compare 16384 - diff last byte 194.34 ns/op 194.22 ns/op 1.00
byteArrayEquals 123687377 - diff last byte 147.05 ms/op 193.49 ms/op 0.76
Buffer.compare 123687377 - diff last byte 5.2987 ms/op 6.3706 ms/op 0.83
byteArrayEquals 32 - random bytes 4.6490 ns/op 5.1410 ns/op 0.90
Buffer.compare 32 - random bytes 14.814 ns/op 17.104 ns/op 0.87
byteArrayEquals 1024 - random bytes 4.7240 ns/op 5.1330 ns/op 0.92
Buffer.compare 1024 - random bytes 14.655 ns/op 17.096 ns/op 0.86
byteArrayEquals 16384 - random bytes 4.6870 ns/op 5.1160 ns/op 0.92
Buffer.compare 16384 - random bytes 14.674 ns/op 17.117 ns/op 0.86
byteArrayEquals 123687377 - random bytes 7.4000 ns/op 6.4300 ns/op 1.15
Buffer.compare 123687377 - random bytes 17.460 ns/op 18.340 ns/op 0.95
regular array get 100000 times 29.225 us/op 32.877 us/op 0.89
wrappedArray get 100000 times 29.215 us/op 32.854 us/op 0.89
arrayWithProxy get 100000 times 8.7894 ms/op 13.154 ms/op 0.67
ssz.Root.equals 41.155 ns/op 45.555 ns/op 0.90
byteArrayEquals 41.165 ns/op 45.045 ns/op 0.91
Buffer.compare 8.5830 ns/op 10.242 ns/op 0.84
processSlot - 1 slots 11.526 us/op 13.959 us/op 0.83
processSlot - 32 slots 3.2804 ms/op 3.0562 ms/op 1.07
getEffectiveBalanceIncrementsZeroInactive - 250000 vs - 7PWei 42.274 ms/op 39.411 ms/op 1.07
getCommitteeAssignments - req 1 vs - 250000 vc 1.7026 ms/op 2.1510 ms/op 0.79
getCommitteeAssignments - req 100 vs - 250000 vc 3.3977 ms/op 4.1608 ms/op 0.82
getCommitteeAssignments - req 1000 vs - 250000 vc 3.6510 ms/op 4.6719 ms/op 0.78
findModifiedValidators - 10000 modified validators 219.34 ms/op 305.03 ms/op 0.72
findModifiedValidators - 1000 modified validators 149.15 ms/op 174.47 ms/op 0.85
findModifiedValidators - 100 modified validators 130.95 ms/op 169.30 ms/op 0.77
findModifiedValidators - 10 modified validators 130.31 ms/op 176.44 ms/op 0.74
findModifiedValidators - 1 modified validators 131.99 ms/op 158.91 ms/op 0.83
findModifiedValidators - no difference 140.49 ms/op 169.76 ms/op 0.83
compare ViewDUs 3.1547 s/op 3.3481 s/op 0.94
compare each validator Uint8Array 1.5932 s/op 1.6522 s/op 0.96
compare ViewDU to Uint8Array 682.20 ms/op 1.0330 s/op 0.66
migrate state 1000000 validators, 24 modified, 0 new 706.13 ms/op 747.44 ms/op 0.94
migrate state 1000000 validators, 1700 modified, 1000 new 953.74 ms/op 998.72 ms/op 0.95
migrate state 1000000 validators, 3400 modified, 2000 new 1.1758 s/op 1.2204 s/op 0.96
migrate state 1500000 validators, 24 modified, 0 new 728.58 ms/op 807.33 ms/op 0.90
migrate state 1500000 validators, 1700 modified, 1000 new 965.94 ms/op 1.1234 s/op 0.86
migrate state 1500000 validators, 3400 modified, 2000 new 1.1513 s/op 1.2723 s/op 0.90
RootCache.getBlockRootAtSlot - 250000 vs - 7PWei 5.7900 ns/op 4.9800 ns/op 1.16
state getBlockRootAtSlot - 250000 vs - 7PWei 853.02 ns/op 609.66 ns/op 1.40
computeProposers - vc 250000 6.1036 ms/op 8.8333 ms/op 0.69
computeEpochShuffling - vc 250000 35.129 ms/op 44.316 ms/op 0.79
getNextSyncCommittee - vc 250000 104.18 ms/op 143.98 ms/op 0.72
computeSigningRoot for AttestationData 21.246 us/op 26.277 us/op 0.81
hash AttestationData serialized data then Buffer.toString(base64) 1.1027 us/op 1.6902 us/op 0.65
toHexString serialized data 729.39 ns/op 1.0382 us/op 0.70
Buffer.toString(base64) 125.84 ns/op 208.97 ns/op 0.60
nodejs block root to RootHex using toHex 117.02 ns/op 184.02 ns/op 0.64
nodejs block root to RootHex using toRootHex 75.660 ns/op 97.967 ns/op 0.77
browser block root to RootHex using the deprecated toHexString 209.22 ns/op 240.76 ns/op 0.87
browser block root to RootHex using toHex 168.68 ns/op 183.96 ns/op 0.92
browser block root to RootHex using toRootHex 149.78 ns/op 172.11 ns/op 0.87

by benchmarkbot/action

wemeetagain
wemeetagain reviewed Jan 19, 2025
View reviewed changes
packages/state-transition/src/block/processWithdrawalRequest.ts
@@ -76,6 +76,7 @@ export function processWithdrawalRequest(
withdrawableEpoch,
});
state.pendingPartialWithdrawals.push(pendingPartialWithdrawal);
state.pendingPartialWithdrawals.commit();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we commit once across multiple withdrawal requests?
eg:

for (...) {
  processWithdrawalRequest(...);
}
state.pendingPartialWithdrawals.commit();

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the problem is that we need to commit before we call getPendingBalanceToWithdraw

lodestar/packages/state-transition/src/block/processWithdrawalRequest.ts

Line 46 in 06831cf

const pendingBalanceToWithdraw = getPendingBalanceToWithdraw(state, validatorIndex);

since this calls getAllReadonly

lodestar/packages/state-transition/src/util/validator.ts

Line 87 in 06831cf

.getAllReadonly()

I am sure there is a better solution for this

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so if we don't commit after we push the first withdrawal request the issue happens

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nflaig calling commit() at the middle of state transition is not great but this is a limitation at ssz side, we should be able to get all items without calling commit()

call state.pendingPartialWithdrawals.getAllReadonly() at the start of processWithdrawalRequest

then modify getPendingBalanceToWithdraw

let total = 0;
for (let i = 0; i < state.pendingPartialWithdrawals.length; i++) {
  const item = state.pendingPartialWithdrawals.get(i);
  if (item.validatorIndex === validatorIndex) {
    total += Number(item.amount);
  }
  
  return total;
}

performance should be the same, a unit test should help

@nflaig
Copy link
Member Author

nflaig commented Jan 19, 2025

putting as draft for now until more testing is done

@nflaig nflaig marked this pull request as draft January 19, 2025 22:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@twoeths twoeths twoeths left review comments

@wemeetagain wemeetagain wemeetagain left review comments

At least 1 approving review is required to merge this pull request.

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@nflaig @wemeetagain @twoeths