Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix post-verge sync #404

Merged
merged 2 commits into from
Mar 13, 2024
Merged

fix post-verge sync #404

merged 2 commits into from
Mar 13, 2024

Conversation

gballet
Copy link
Owner

@gballet gballet commented Mar 13, 2024

Sync was broken after the network had passed the verkle fork. This was because the EL will try to produce its own block when starting up, before the CL has had a chance to start inserting blocks.

The problem was that the timestamp being past the fork, verkle mode would activate for this side block. And then, when the first sync block was inserted (having been created before the fork), the started variable would not be reinitialized. This would cause an invalid merkle root.

The fix ensures that started is reinitialized to false when inserting a pre-verge block.

} else {
// If the verkle activation time hasn't started, declare it as "not started".
// This is so that
bc.stateCache.InitTransitionStatus(false, false)
Copy link
Collaborator

@jsign jsign Mar 13, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well for one thing the comment above is outdated.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TLDR no it doesn't. The reason for this is that this activation comes from the miner, AFTER the blockchain has been initialized. It's also true of a reorg: if for whatever reason the verkle fork time changes (let's say it's pushed back) the reorg will have to go through without the chain to be reinitialized, and so NewBlockchain will not be called.

@@ -1768,6 +1768,10 @@ func (bc *BlockChain) insertChain(chain types.Blocks, setHead bool) (int, error)
if !bc.stateCache.InTransition() && !bc.stateCache.Transitioned() {
bc.stateCache.StartVerkleTransition(parent.Root, emptyVerkleRoot, bc.Config(), bc.Config().PragueTime, parent.Root)
}
} else {
// If the verkle activation time hasn't started, declare it as "not started".
// This is so that
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just in case: this comment is truncated.

Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops, yeah

@gballet gballet merged commit ec5b79e into transition-post-genesis Mar 13, 2024
1 check passed
gballet added a commit that referenced this pull request Mar 26, 2024
* fix post-verge sync

* review: fix truncated comment
gballet added a commit that referenced this pull request Apr 15, 2024
* fix post-verge sync

* review: fix truncated comment
gballet added a commit that referenced this pull request Apr 29, 2024
quell linter issue

support Transition post tree in conversion

Refactor transition post genesis (#311)

* rewrite per-block conversion pointer management

* remove unused method

* fix: a branch that can verge at genesis or post genesis (#314)

* fix: import cycle in conversion refactor (#315)

* fix shadowfork panic in OpenStorageTrie

fix OpenStorageTrie: return an error instead of panicking if the account tree not a verkle tree (#321)

add a switch to force proof in blocks (#322)

* add a switch to force proof in blocks

* activate switch

* fix switch type

add switch to override the stride of the overlay conversion (#323)

* add switch to override the stride of the overlay conversion

* set a default stride of 10k

add a few traces for relaunch

add missing step in the chain of setting the override for overlay stride

fix: save and load transition state for block processing (#324)

* fix: save and load transition state for block processing

* log whether the tree is verkle in LoadTransitionState

* fix: ensure the transition is marked as started in insertChain

* dump saved address

* fix nil pointer panic

* remove stacktrace that is no longer useful

* fix a panic

* fix build

* check: copy current account address BEFORE it's saved

* mandatory panic fix

* Remove debug fmt.Println

* more cleanup + comments

fix: ensure StorageProcessed is properly recovered

add traces for gas pool overflow

cmd/{geth, utils}: add a command to clear verkle costs (#326)

* cmd/{geth, utils}: add a command to clear verkle costs

fix:  boolean issue

fix: load finalization state in FinalizeAndAssemble (#340)

* Conversion and TransitionTrie fixes (#346)

* fixes

Signed-off-by: Ignacio Hagopian <[email protected]>

* remove old comment

Signed-off-by: Ignacio Hagopian <[email protected]>

---------

Signed-off-by: Ignacio Hagopian <[email protected]>

* trace cleanup

---------

Signed-off-by: Ignacio Hagopian <[email protected]>
Co-authored-by: Ignacio Hagopian <[email protected]>

fix: check for nil override in NewBlockChain (#357)

fix rebase issue

fix a few rebase issues

add debug traces

fix: assume that having to create a new transaction state mean it hasn't happened.

This is a workaround, because it will always be false when restarting a node
that has started the transition but not necessarily completed it.

add logs to debug shadowfork

persist conversion state to db and use an LRU cache for active transition states (#375)

* persist conversion state to db

* fix: don't recreate LRU when writing state

* opt: only write state to db if not already present in LRU

* fix: rlp can't encode TransitionState

* fix: use gob because binary.Write does not support structs 🤦‍♂️

* fix: nil pointer panic

* add logs to debug shadowfork

* no such thing as not enough traces

* ditto

* fix stupid bug

* add a comment for readability

* add more traces

* Lock the state transition during conversion (#384)

* heavy handed approach: lock the state transition during conversion

* also lock transition state loading/unloading

* reduce logs verbosity

* add conversion test to workflow (#386)

* add conversion test to workflow

* mandatory -f switch fix in rm

* forgot & at the end of the geth command

* remove incorrect kill

* add debug traces

* add an overlay stride

* fix typo

* Apply suggestions from code review

eth: add debug_conversionStatus RPC call (#392)

* eth: add debug_conversionStatus RPC call

* add debug trace4s

* Apply suggestions from code review

* export started/ended fields

fix post-verge sync (#404)

* fix post-verge sync

* review: fix truncated comment
gballet added a commit that referenced this pull request Apr 29, 2024
quell linter issue

support Transition post tree in conversion

Refactor transition post genesis (#311)

* rewrite per-block conversion pointer management

* remove unused method

* fix: a branch that can verge at genesis or post genesis (#314)

* fix: import cycle in conversion refactor (#315)

* fix shadowfork panic in OpenStorageTrie

fix OpenStorageTrie: return an error instead of panicking if the account tree not a verkle tree (#321)

add a switch to force proof in blocks (#322)

* add a switch to force proof in blocks

* activate switch

* fix switch type

add switch to override the stride of the overlay conversion (#323)

* add switch to override the stride of the overlay conversion

* set a default stride of 10k

add a few traces for relaunch

add missing step in the chain of setting the override for overlay stride

fix: save and load transition state for block processing (#324)

* fix: save and load transition state for block processing

* log whether the tree is verkle in LoadTransitionState

* fix: ensure the transition is marked as started in insertChain

* dump saved address

* fix nil pointer panic

* remove stacktrace that is no longer useful

* fix a panic

* fix build

* check: copy current account address BEFORE it's saved

* mandatory panic fix

* Remove debug fmt.Println

* more cleanup + comments

fix: ensure StorageProcessed is properly recovered

add traces for gas pool overflow

cmd/{geth, utils}: add a command to clear verkle costs (#326)

* cmd/{geth, utils}: add a command to clear verkle costs

fix:  boolean issue

fix: load finalization state in FinalizeAndAssemble (#340)

* Conversion and TransitionTrie fixes (#346)

* fixes

Signed-off-by: Ignacio Hagopian <[email protected]>

* remove old comment

Signed-off-by: Ignacio Hagopian <[email protected]>

---------

Signed-off-by: Ignacio Hagopian <[email protected]>

* trace cleanup

---------

Signed-off-by: Ignacio Hagopian <[email protected]>
Co-authored-by: Ignacio Hagopian <[email protected]>

fix: check for nil override in NewBlockChain (#357)

fix rebase issue

fix a few rebase issues

add debug traces

fix: assume that having to create a new transaction state mean it hasn't happened.

This is a workaround, because it will always be false when restarting a node
that has started the transition but not necessarily completed it.

add logs to debug shadowfork

persist conversion state to db and use an LRU cache for active transition states (#375)

* persist conversion state to db

* fix: don't recreate LRU when writing state

* opt: only write state to db if not already present in LRU

* fix: rlp can't encode TransitionState

* fix: use gob because binary.Write does not support structs 🤦‍♂️

* fix: nil pointer panic

* add logs to debug shadowfork

* no such thing as not enough traces

* ditto

* fix stupid bug

* add a comment for readability

* add more traces

* Lock the state transition during conversion (#384)

* heavy handed approach: lock the state transition during conversion

* also lock transition state loading/unloading

* reduce logs verbosity

* add conversion test to workflow (#386)

* add conversion test to workflow

* mandatory -f switch fix in rm

* forgot & at the end of the geth command

* remove incorrect kill

* add debug traces

* add an overlay stride

* fix typo

* Apply suggestions from code review

eth: add debug_conversionStatus RPC call (#392)

* eth: add debug_conversionStatus RPC call

* add debug trace4s

* Apply suggestions from code review

* export started/ended fields

fix post-verge sync (#404)

* fix post-verge sync

* review: fix truncated comment
@gballet gballet mentioned this pull request Apr 29, 2024
gballet added a commit that referenced this pull request May 2, 2024
quell linter issue

support Transition post tree in conversion

Refactor transition post genesis (#311)

* rewrite per-block conversion pointer management

* remove unused method

* fix: a branch that can verge at genesis or post genesis (#314)

* fix: import cycle in conversion refactor (#315)

* fix shadowfork panic in OpenStorageTrie

fix OpenStorageTrie: return an error instead of panicking if the account tree not a verkle tree (#321)

add a switch to force proof in blocks (#322)

* add a switch to force proof in blocks

* activate switch

* fix switch type

add switch to override the stride of the overlay conversion (#323)

* add switch to override the stride of the overlay conversion

* set a default stride of 10k

add a few traces for relaunch

add missing step in the chain of setting the override for overlay stride

fix: save and load transition state for block processing (#324)

* fix: save and load transition state for block processing

* log whether the tree is verkle in LoadTransitionState

* fix: ensure the transition is marked as started in insertChain

* dump saved address

* fix nil pointer panic

* remove stacktrace that is no longer useful

* fix a panic

* fix build

* check: copy current account address BEFORE it's saved

* mandatory panic fix

* Remove debug fmt.Println

* more cleanup + comments

fix: ensure StorageProcessed is properly recovered

add traces for gas pool overflow

cmd/{geth, utils}: add a command to clear verkle costs (#326)

* cmd/{geth, utils}: add a command to clear verkle costs

fix:  boolean issue

fix: load finalization state in FinalizeAndAssemble (#340)

* Conversion and TransitionTrie fixes (#346)

* fixes

Signed-off-by: Ignacio Hagopian <[email protected]>

* remove old comment

Signed-off-by: Ignacio Hagopian <[email protected]>

---------

Signed-off-by: Ignacio Hagopian <[email protected]>

* trace cleanup

---------

Signed-off-by: Ignacio Hagopian <[email protected]>
Co-authored-by: Ignacio Hagopian <[email protected]>

fix: check for nil override in NewBlockChain (#357)

fix rebase issue

fix a few rebase issues

add debug traces

fix: assume that having to create a new transaction state mean it hasn't happened.

This is a workaround, because it will always be false when restarting a node
that has started the transition but not necessarily completed it.

add logs to debug shadowfork

persist conversion state to db and use an LRU cache for active transition states (#375)

* persist conversion state to db

* fix: don't recreate LRU when writing state

* opt: only write state to db if not already present in LRU

* fix: rlp can't encode TransitionState

* fix: use gob because binary.Write does not support structs 🤦‍♂️

* fix: nil pointer panic

* add logs to debug shadowfork

* no such thing as not enough traces

* ditto

* fix stupid bug

* add a comment for readability

* add more traces

* Lock the state transition during conversion (#384)

* heavy handed approach: lock the state transition during conversion

* also lock transition state loading/unloading

* reduce logs verbosity

* add conversion test to workflow (#386)

* add conversion test to workflow

* mandatory -f switch fix in rm

* forgot & at the end of the geth command

* remove incorrect kill

* add debug traces

* add an overlay stride

* fix typo

* Apply suggestions from code review

eth: add debug_conversionStatus RPC call (#392)

* eth: add debug_conversionStatus RPC call

* add debug trace4s

* Apply suggestions from code review

* export started/ended fields

fix post-verge sync (#404)

* fix post-verge sync

* review: fix truncated comment
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants