Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make blocks cache locked in Store when used + minor cleanup #7594

Merged
merged 14 commits into from
Nov 4, 2023

Conversation

zilm13
Copy link
Contributor

@zilm13 zilm13 commented Oct 13, 2023

PR Description

I've noticed that blocks are not actually locked if we go in block provider, after inMemory lookup in retrieveSignedBlock (it could actually changed between) and when directly using blockProvider, so tried to fix it.

Fixed Issue(s)

Documentation

  • I thought about documentation and added the doc-change-required label to this PR if updates are required.

Changelog

  • I thought about adding a changelog entry, and added one if I deemed necessary.

@zilm13 zilm13 changed the title Make blocks locked when used + minor cleanup Make blocks cache locked in Store when used + minor cleanup Oct 13, 2023
@@ -37,6 +38,22 @@ static BlockProvider fromDynamicMap(Supplier<Map<Bytes32, SignedBeaconBlock>> ma
return (roots) -> fromMap(mapSupplier.get()).getBlocks(roots);
}

static BlockProvider fromMapWithLock(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should probably add a test case

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks you asked for it, I made all stupidly wrong, fixed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did the wrong thing again, thinking how to do it correctly

Copy link
Contributor Author

@zilm13 zilm13 Oct 16, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So my understanding on this is:

  1. We should remove SafeFuture.supplyAsync() to avoid such errors, noone was using it, and we should use our own AsyncRunner whenever it's needed
  2. In this case we should use our AsyncRunner
  3. We should also use it in Store methods that use locks and return SafeFuture otherwise they are synchronously locked (retrieveFinalizedCheckpointAndState, createStateGenerationTask)
  4. It will require rework of many Store tests

What do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think we could raise a ticket for that cleanup, but probably not worth it now, it'd be a big distraction if its used a bunch in tests...
Using async here though would make sense so we dont add to the problem...

@zilm13 zilm13 requested a review from rolfyone October 16, 2023 13:46
@zilm13 zilm13 marked this pull request as draft October 16, 2023 16:24
}
})
.ifExceptionGetsHereRaiseABug();
assertThat(isNotBlocked.await(1, TimeUnit.SECONDS)).isFalse();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

even something small like 100ms should be plenty here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure

@zilm13 zilm13 marked this pull request as ready for review October 17, 2023 14:55
@zilm13
Copy link
Contributor Author

zilm13 commented Oct 17, 2023

@rolfyone finished, ready for review

  1. Somehow test failed on Windows (at least once, on notBlockedOnFuture) with 100ms delay. I don't want to add any flakiness, pushed it to 500ms.
  2. Final fixes for other methods are in separate PR, based on this one, need to merge this one at first

Copy link
Contributor

@rolfyone rolfyone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zilm13
Copy link
Contributor Author

zilm13 commented Nov 3, 2023

Disabled windows test for a while, will check on my local windows

@zilm13
Copy link
Contributor Author

zilm13 commented Nov 4, 2023

Fixed flakiness with 50a3be1

@zilm13 zilm13 merged commit be47d28 into Consensys:master Nov 4, 2023
4 checks passed
@zilm13 zilm13 deleted the fix-blobs-store branch November 4, 2023 11:38
@zilm13 zilm13 mentioned this pull request Nov 25, 2023
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants