Lodestar produces blocks with empty sync aggregate

[nflaig]

This behavior has been observed on the Mekong testnet.

Eg. https://dora.mekong.ethpandaops.io/slot/274010 contains an empty sync aggregate (0% participation)

There are no obvious error logs that could explain this, the only log (from lodestar-geth-3) I found was from our light client server which was complaining about it

Dec-13 17:22:00.136[chain]           debug: sync committee below required MIN_SYNC_COMMITTEE_PARTICIPANTS syncPeriod=33, attestedPeriod=33, syncAggregateParticipation=0

This seems to be isolated to electra / mekong / devnet-4 where it can be observed quite frequently, also has been reported by Luca on discord.

[nflaig]

This also happened on holesky, see https://holesky.beaconcha.in/slot/3255000

[prelude444]

same here. also with attestation only 1.

and it's happening on mainnet. Also, ever since the upgrade to the latest v1.24.0, my luck for finding block went straight down to less than 20%.

[nflaig]

same here. also with attestation only 1.

could please share which slot this was, the empty sync aggregate issue is likely due to the same issue as on holesky which was related to building a block on a really late parent block (> 4 seconds into the slot), this will be resolved once we enable proposer boost reorg by default as we would not build on top of such late blocks. I am a bit surprised that it only includes a single attestation though, this should be unrelated.

Also, ever since the upgrade to the latest v1.24.0, my luck for finding block went straight down to less than 20%.

I am guessing by finding a block you mean be selected to propose a block? This is not something related to the client but the ethereum protocol and is out of your control

[prelude444]

slot 10798404

It's my first time ever propose a block that has only 1 attestation. Yes it seems strange. My validator is not slow, 99% efficiency avg.

The luck in proposing a block also went to below 20%. Normally its around 60%-130%ish. never went this low. Maybe I was just extremely unlucky lately. If it's not due to the outlier in luck, i can only attribute the cause to maybe 2 things, 1, I switched my router to a new one, or 2. I upgraded to v1.24.0. Everything else was the same as before.

[nflaig]

slot 10798404

I was checking on my node when I received the block from previous slot and it was around ~3 seconds into the slot when it was fully validated and imported as head, late parent block does not seem to be the issue here. We are keeping an eye on this issue but so far it's not something that happens consistently, especially on mainnet. Other than late parent block I am not sure what else could be causing this, maybe someone from team has other ideas

The luck in proposing a block also went to below 20%. Normally its around 60%-130%ish. never went this low. Maybe I was just extremely unlucky lately. If it's not due to the outlier in luck, i can only attribute the cause to maybe 2 things, 1, I switched my router to a new one, or 2. I upgraded to v1.24.0. Everything else was the same as before.

This is completely out of your control and random, you could be offline and be selected as proposer as long as your validator is active on the beacon chain, you can read up on this here https://ethereum.org/en/developers/docs/consensus-mechanisms/pos/block-proposal/#random-selection. Something to keep in mind is that the amount of validators on mainnet increases over time which reduces the probability for you to be selected as proposer.