Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement logic to clone and rollback storage for engine switching. #334

Merged
merged 20 commits into from
Sep 20, 2023

Conversation

piotrm50
Copy link
Collaborator

@piotrm50 piotrm50 commented Sep 6, 2023

Fixes #313

pkg/protocol/engine/accounts/accountsledger/snapshot.go Outdated Show resolved Hide resolved
pkg/testsuite/mock/node.go Show resolved Hide resolved
pkg/protocol/enginemanager/enginemanager.go Outdated Show resolved Hide resolved
pkg/protocol/enginemanager/enginemanager.go Outdated Show resolved Hide resolved
pkg/protocol/enginemanager/enginemanager.go Outdated Show resolved Hide resolved
pkg/storage/storage.go Outdated Show resolved Hide resolved
pkg/storage/storage.go Outdated Show resolved Hide resolved
pkg/storage/prunable/prunable.go Outdated Show resolved Hide resolved

// Remove committee for the next epoch only if forking point is before point of no return and committee is reused.
// Always remove committees for epochs that are newer than targetSlotEpoch+1.
func (p *Prunable) shouldRollbackCommittee(epochIndex iotago.EpochIndex, targetSlotIndex iotago.SlotIndex) (bool, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic is very specific and it might lead to issues if the behavior of the committee rotation changes. Maybe we can put it in the Sybilprotection and hand it in as a callback?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. The only problem here is that the rollback also depends on the storage perception of the committee: if it was persisted and found or not. Using a callback would also force us to pass the prunable committee storage in the sybilprotection, which in turn will depend on the sybilprotection to determine if a rollback is due: creating a cycle.

Copy link
Contributor

@jonastheis jonastheis Sep 20, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should be able to put it there: PerformanceTracker has the committee store already:

committeeStore *epochstore.Store[*account.Accounts]

The only thing we'd need to do is expose this method on the SybilProtection interface and hand it to the prunable. Problem here is that the storage is initialized before the entire engine.

So maybe it's okay to leave it there...

pkg/storage/database/syncedkvstore.go Outdated Show resolved Hide resolved
@karimodm karimodm requested a review from jonastheis September 19, 2023 13:51
@karimodm karimodm marked this pull request as ready for review September 19, 2023 13:51
@karimodm karimodm force-pushed the feat/copy-forked-storage branch from cede9c9 to c237d8c Compare September 20, 2023 07:26
@karimodm karimodm merged commit 80d6b07 into develop Sep 20, 2023
7 checks passed
@karimodm karimodm deleted the feat/copy-forked-storage branch September 20, 2023 07:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement copying over of data (and deletion) when switching engines
3 participants