Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid registration of non-active validators #7423

Merged
merged 31 commits into from
Oct 10, 2023

Conversation

zilm13
Copy link
Contributor

@zilm13 zilm13 commented Aug 15, 2023

PR Description

  • I don't like OwnedValidators.getActiveValidators() name because it's misleading. I'd rename to exclude word active, but it will lead to changes in several classes, so I skipped it to omit unnecessary changes in this PR and could do a separate PR with this change.
  • Cache looks like not an issue. We will not cache not yet active validator. For exited validator we will not receive any duty but once we try to renew registration, it will be filtered out and will not lead to sign attempt. Cache will not be pruned in this case but it doesn't look an issue too. As validator remains in owned, we will try to further attempts anyway.
  • What I don't like is that we do everything on the 3rd slot, so if validator becomes active ongoing in this epoch and luckily become a proposer in first 3 slots, it will not use builder registration for it

I appreciate any ideas how could I test it thoroughly, because it's a sensitive part of the code.

Fixed Issue(s)

Fixes #7382
Fixes #6290

Documentation

  • I thought about documentation and added the doc-change-required label to this PR if updates are required.

Changelog

  • I thought about adding a changelog entry, and added one if I deemed necessary.

@zilm13 zilm13 marked this pull request as draft August 16, 2023 07:55
@zilm13 zilm13 marked this pull request as ready for review August 18, 2023 19:34
@zilm13 zilm13 requested a review from StefanBratanov August 18, 2023 19:35
@zilm13
Copy link
Contributor Author

zilm13 commented Aug 18, 2023

Finished reworking it. Now everything in batches. Tried to keep all existing logic.

Copy link
Contributor

@StefanBratanov StefanBratanov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@rolfyone rolfyone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zilm13
Copy link
Contributor Author

zilm13 commented Aug 24, 2023

Still testing, don't merge

@zilm13 zilm13 self-assigned this Aug 28, 2023
@tbenr
Copy link
Contributor

tbenr commented Sep 6, 2023

as discussed on standup:

ValidatorTimingActions::onSlot calls statusLogger.checkValidatorStatusChanges(); at second slot of each epoch

it also happens on startup via statusLogger.printInitialValidatorStatuses();.

@StefanBratanov
Copy link
Contributor

Hmm good point, so we make duplicate calls.

@zilm13
Copy link
Contributor Author

zilm13 commented Sep 6, 2023

Yeah, nice catch, detaching it to ValidatorStatusProvider so both logger and registrator could use it

@zilm13 zilm13 marked this pull request as draft September 6, 2023 15:07
@zilm13
Copy link
Contributor Author

zilm13 commented Sep 7, 2023

Rework finished, some notes:

  • start from ValidatorStatusProvider
  • updated to register pending validators according to the spec [1], [2]
  • I think it's a cleaner approach that validator statuses are not queried in ValidatorApiChannel when we do registration, though it could be changed and we don't call for statuses twice when we run vc locally

@zilm13 zilm13 marked this pull request as ready for review September 7, 2023 15:47
@@ -300,7 +301,36 @@ public SafeFuture<Void> prepareBeaconProposer(

public SafeFuture<Void> registerValidators(
SszList<SignedValidatorRegistration> validatorRegistrations) {
return validatorApiChannel.registerValidators(validatorRegistrations);
return validatorApiChannel
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have 2 doubts here:

1- putting filtering here means that it will only be applied when registering from a remote VC, and it will not be applied in single process.
I see this ValidatorDataProvider as a very thin layer. We should not have business logic unless we really want to differentiate between in process and remote VC handling.

2- if we filter upfront it means that validators in unknown status will not go in our BN data structure. Which means that, if VC doesn't keep calling the registration endpoint, the validator could become known at some point in future. It is an edge case but in theory we loose this info.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sorry for 2-. It is totally irrelevant considering what we are trying to achieve with this PR :)

Copy link
Contributor

@tbenr tbenr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

releasing some pending comments while waiting for the fixed version

@@ -69,8 +69,11 @@ public void handleRequest(final RestApiRequest request) throws JsonProcessingExc
.handle(
(__, error) -> {
if (error != null) {
return AsyncApiResponse.respondWithError(
SC_INTERNAL_SERVER_ERROR, error.getMessage());
final String message =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about using ExceptionUtil::getRootCauseMessage or add another method there if there is nothing currently fitting the need

@Override
public void onNewValidatorStatuses(
final Map<BLSPublicKey, ValidatorStatus> newValidatorStatuses) {
Map<BLSPublicKey, ValidatorStatus> oldValidatorStatuses =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

final

Comment on lines 54 to 58
for (Map.Entry<BLSPublicKey, ValidatorStatus> entry : newValidatorStatuses.entrySet()) {
BLSPublicKey key = entry.getKey();
ValidatorStatus newStatus = entry.getValue();
ValidatorStatus oldStatus = oldValidatorStatuses.get(key);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

entry and other locals -> final

}

private void printValidatorStatusSummary(Map<BLSPublicKey, ValidatorStatus> validatorStatuses) {
Map<ValidatorStatus, AtomicInteger> validatorStatusCount = new HashMap<>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

final

}

private void printValidatorStatusesOneByOne(
Map<BLSPublicKey, ValidatorStatus> validatorStatuses) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

final


private void printValidatorStatusesOneByOne(
Map<BLSPublicKey, ValidatorStatus> validatorStatuses) {
for (BLSPublicKey publicKey : validators.getPublicKeys()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

finals

@zilm13 zilm13 marked this pull request as draft September 20, 2023 19:40
@zilm13
Copy link
Contributor Author

zilm13 commented Oct 2, 2023

Reworked to use subscribers model, so ValidatorStatusProvider actively checks and updating validator statuses, including new additions. One of subscriber is ValidatorRegistrator which checks if recent status update requires registration according to its rules and if it's required, registers validators without checking for statuses again. Also I've promoted log level of some events in ValidatorRegistrator, makes sense for me. If it's not, could revert it.

@zilm13 zilm13 marked this pull request as ready for review October 2, 2023 21:27
Copy link
Contributor

@tbenr tbenr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

some comments waiting onPossibleMissedEvents fix

import tech.pegasys.teku.spec.schemas.ApiSchemas;
import tech.pegasys.teku.spec.signatures.Signer;

public class ValidatorRegistrationSigningService {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems like not a real Service. How about ValidatorRegistrationCreator?

metricsSystem, validators, validatorApiChannel, asyncRunner);
final ValidatorStatusLogger validatorStatusLogger = new ValidatorStatusLogger(validators);
validatorStatusProvider.subscribeNewValidatorStatuses(
validatorStatusLogger::onNewValidatorStatuses);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what about calling the method onRefreshedValidatorStatuses? otherwise it could seem that it is called only when you have a new status. The same applies to the subscription interface

@@ -115,11 +128,44 @@ void shouldUpdateMetricsForValidatorStatuses() {
assertThat(gauge.getValue(ValidatorStatus.active_ongoing.name()))
.isEqualTo(OptionalDouble.of(0));

logger.checkValidatorStatusChanges();
provider.onSlot(spec.computeStartSlotAtEpoch(UInt64.ONE).plus(1));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe you should change the name of this test. It is testing that onSlot we do the job of updating statuses, and you could add the check that the subscriber is called.

Copy link
Contributor

@tbenr tbenr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zilm13 zilm13 merged commit 35f9747 into Consensys:master Oct 10, 2023
@zilm13 zilm13 deleted the fix/active-validator-registration branch October 10, 2023 19:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants