Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix IBFT2 chains using BONSAI DB #12

Closed
wants to merge 741 commits into from

Conversation

matthew1001
Copy link

PR description

This PR fixes a bug with IBFT2 chains where blocks are persisted at block-proposal time instead of block-import time. With Bonsai DB this causes issues updating the flat DB because if a round change happens the original proposal isn't actually the block that's accepted for that round.

Fixed Issue(s)

Fixes hyperledger#7629

krsh24 and others added 30 commits June 11, 2024 08:01
* Error out on permissions config accounts-allowlist validation errors.

Signed-off-by: krishnannarayanan <[email protected]>

* Fixing compilation errors

Signed-off-by: krishnannarayanan <[email protected]>

* Incorrect file check in

Signed-off-by: krishnannarayanan <[email protected]>

* Support for eth_maxPriorityFeePerGas

Signed-off-by: krishnannarayanan <[email protected]>

* Change log update for my PRs

Signed-off-by: krishnannarayanan <[email protected]>

---------

Signed-off-by: krishnannarayanan <[email protected]>
… to the method (hyperledger#7188)

Signed-off-by: Daniel Lehrner <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>
* Add new acceptance test to soak test BFT chains

Signed-off-by: Matthew Whitehead <[email protected]>

* Spotless

Signed-off-by: Matthew Whitehead <[email protected]>

* Tidy up a little with re-usable start and stop functions with built in delays

Signed-off-by: Matthew Whitehead <[email protected]>

* Add shanghai version of Simple Storage contract

Signed-off-by: Matthew Whitehead <[email protected]>

* Put commented gradle code back in. Fix the web3j example commands in .sol files

Signed-off-by: Matthew Whitehead <[email protected]>

* Spotless

Signed-off-by: Matthew Whitehead <[email protected]>

* Set publication artifacts to avoid clash

Signed-off-by: Matthew Whitehead <[email protected]>

* Exclude from regular acceptance tests

Signed-off-by: Matthew Whitehead <[email protected]>

* Add shanghai fork to the test. Stall the chain for less time to reduce the time taken to mine new blocks

Signed-off-by: Matthew Whitehead <[email protected]>

* Tidy up

Signed-off-by: Matthew Whitehead <[email protected]>

* Update acceptance-tests/tests/shanghai/build.gradle

Co-authored-by: Simon Dudley <[email protected]>
Signed-off-by: Matt Whitehead <[email protected]>

* Tidy up var names

Signed-off-by: Matthew Whitehead <[email protected]>

* Fix ports for IBFT2 as well as QBFT

Signed-off-by: Matthew Whitehead <[email protected]>

* Remove maven publish spec, disable jar building for shanghai contract project

Signed-off-by: Matthew Whitehead <[email protected]>

* web3j version

Signed-off-by: Matthew Whitehead <[email protected]>

* Make fixed port optional when creating a BFT node

Signed-off-by: Matthew Whitehead <[email protected]>

* Only check artifact coordinates for those starting 'org.*'

Signed-off-by: Matthew Whitehead <[email protected]>

---------

Signed-off-by: Matthew Whitehead <[email protected]>
Signed-off-by: Matt Whitehead <[email protected]>
Signed-off-by: Matt Whitehead <[email protected]>
Co-authored-by: Simon Dudley <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>
* Refactor genesis options file management

Signed-off-by: Fabio Di Fabio <[email protected]>

* Make loading allocation from genesis lazy

Signed-off-by: Fabio Di Fabio <[email protected]>

* Update tests

Signed-off-by: Fabio Di Fabio <[email protected]>

* Memory optimization with streaming

Signed-off-by: Fabio Di Fabio <[email protected]>

* Improve loading and storgin genesis state at startup

Signed-off-by: Fabio Di Fabio <[email protected]>

* Remove comments

Signed-off-by: Fabio Di Fabio <[email protected]>

* Avoid parsing genesis file allocations twice

Signed-off-by: Fabio Di Fabio <[email protected]>

* Update javadoc

Signed-off-by: Fabio Di Fabio <[email protected]>

* Fix

Signed-off-by: Fabio Di Fabio <[email protected]>

* Fix

Signed-off-by: Fabio Di Fabio <[email protected]>

* Ignore unknown objects in allocations

Signed-off-by: Fabio Di Fabio <[email protected]>

* avoid keeping genesis allocation data in memory

Signed-off-by: Fabio Di Fabio <[email protected]>

* Update CHANGELOG

Signed-off-by: Fabio Di Fabio <[email protected]>

---------

Signed-off-by: Fabio Di Fabio <[email protected]>
Signed-off-by: ahamlat <[email protected]>
Co-authored-by: ahamlat <[email protected]>
Enable JFR continuous profiling with default setting

Signed-off-by: Ameziane H <[email protected]>
* fix(test): Fix dns daemon periodic test

Increase second run to 3 seconds

Signed-off-by: Usman Saleem <[email protected]>

* fix(test): Fix dns daemon periodic test

change vertx testContext verify to failNow

Signed-off-by: Usman Saleem <[email protected]>

* fix(test): Fix dns daemon periodic test

Remove Disabled

Signed-off-by: Usman Saleem <[email protected]>

* fix(test): Code formatting

Signed-off-by: Usman Saleem <[email protected]>

---------

Signed-off-by: Usman Saleem <[email protected]>
* Creates a new issue template for releases

Signed-off-by: Justin Florentine <[email protected]>

* aligns release process with wiki docs

Signed-off-by: Justin Florentine <[email protected]>

---------

Signed-off-by: Justin Florentine <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>
Remove all EIP-3074 code from Besu. Since EIP-3074 has been replaced
with EIP-7702 in Pectra, and there is no intent to schedule it for a
future fork there is no need to retain the code.

Signed-off-by: Danno Ferrin <[email protected]>
A complete and up to date implementation of EIP-7692 EOF v.1. For genesis 
file activation use "PragueEOFTime", for references tests it activates as 
part of Prague.

Signed-off-by: Danno Ferrin <[email protected]>
* Update Dockerfile

reuse NO_PROXY_CACHE env

Signed-off-by: Enrico Del Fante <[email protected]>

* Update Dockerfile

Signed-off-by: Enrico Del Fante <[email protected]>

---------

Signed-off-by: Enrico Del Fante <[email protected]>
Co-authored-by: Usman Saleem <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>
)

* Don't persist BFT proposed blocks, only committed ones

Signed-off-by: Matthew Whitehead <[email protected]>

* Fix unit tests, update copyright

Signed-off-by: Matthew Whitehead <[email protected]>

* Update changelog

Signed-off-by: Matthew Whitehead <[email protected]>

---------

Signed-off-by: Matthew Whitehead <[email protected]>
Signed-off-by: Matt Whitehead <[email protected]>
Signed-off-by: cangqiaoyuzhuo <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>
…erledger#7180)

* Avoid keeping txpool lock during block creation

Signed-off-by: Fabio Di Fabio <[email protected]>

* Update CHANGELOG

Signed-off-by: Fabio Di Fabio <[email protected]>

* Remove unneeded synchronized

Signed-off-by: Fabio Di Fabio <[email protected]>

---------

Signed-off-by: Fabio Di Fabio <[email protected]>
…perledger#6734)

* Change eth trace from 'Message not expected' to 'Request message'

Signed-off-by: Matthew Whitehead <[email protected]>

* Add braces to log message

Signed-off-by: Matthew Whitehead <[email protected]>

* Spotless fix

Signed-off-by: Matthew Whitehead <[email protected]>

---------

Signed-off-by: Matthew Whitehead <[email protected]>
Signed-off-by: Matt Whitehead <[email protected]>
Container security scanning workflow added. This runs on schedule everyday. Also possible to run on-demand for a given image tag

Signed-off-by: Chaminda Divitotawela <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>
Co-authored-by: Justin Florentine <[email protected]>
Repository follow standard to use git hash to pin the GitHub actions. Updated the container security scan workflow actions with their git hashes

Signed-off-by: Chaminda Divitotawela <[email protected]>
Release workflow publish step was missing the depepndency of artifacts jobs. Due to this reason it could not collect the artifact hashes from the artifacts job. This was introduced in the release workflow consolidation

Signed-off-by: Chaminda Divitotawela <[email protected]>
Fail earlier with the initcode size check

Signed-off-by: Danno Ferrin <[email protected]>
Trivy scan result upload to GitHub fails due to permission issue. Added permission security-events=write to the workflow file as a fix. Since workflow permission explicitly defined, it requires contents=read explicity set as well

Signed-off-by: Chaminda Divitotawela <[email protected]>
Container verification step in release process automated with the container verify GitHub workflow. New workflow is triggered at the end of the release workflow which will check the release container images starts successfully. Verification test only checks container starts and reach the Ethereum main loop

Signed-off-by: Chaminda Divitotawela <[email protected]>
Fix some reasons for chain download halts when syncing

Signed-off-by: [email protected] <[email protected]>
Signed-off-by: Stefan Pingel <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>
Check and test for the unused container rule, and only returncontract
targets can have truncated data rule.
Also test the other subcontainer rules in unit tests.

Signed-off-by: Danno Ferrin <[email protected]>
…rty (hyperledger#7222)

* Add build version option to prefix git hash with custom version property
* Refactor to make appending the git hash a boolean property. Include a commented-out example of how to use the properties in the gradle file

Signed-off-by: Matthew Whitehead <[email protected]>
garyschulte and others added 26 commits September 4, 2024 18:31
…yperledger#7576)

* Remove tx from pool when its score is lower than a configured value

Signed-off-by: Fabio Di Fabio <[email protected]>

* Update besu/src/main/java/org/hyperledger/besu/cli/options/TransactionPoolOptions.java

Co-authored-by: Stefan Pingel <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>

* Check for below min score after the penalization

Signed-off-by: Fabio Di Fabio <[email protected]>

---------

Signed-off-by: Fabio Di Fabio <[email protected]>
Co-authored-by: Stefan Pingel <[email protected]>
* implement engine_getBlobsV1

Signed-off-by: [email protected] <[email protected]>
Signed-off-by: Stefan Pingel <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>
* wrapped WorldUpdater into `EVMWorldupdater` to remove the authority code injection from the implementation of the actual world updaters

Signed-off-by: Daniel Lehrner <[email protected]>

* add CHANGELOG entry

Signed-off-by: Daniel Lehrner <[email protected]>

* first draft for 7702 v2

Signed-off-by: Daniel Lehrner <[email protected]>

* change return value of DelegatedCodeGasCostHelper

Signed-off-by: Daniel Lehrner <[email protected]>

* fix upfront gas cost calculation, fix setting code multiple times in MutableDelegatedCodeAccount

Signed-off-by: Daniel Lehrner <[email protected]>

* fix gas refund for delegated code when account already exists, added gas cost deduction for code delegation resolution to ExtCodeSizeOperation

Signed-off-by: Daniel Lehrner <[email protected]>

* allow accounts with delegated code to send transactions

Signed-off-by: Daniel Lehrner <[email protected]>

* add refund for already existing account after nonce check

Signed-off-by: Daniel Lehrner <[email protected]>

* resolve delegated code only the first time to avoid delegation loops

Signed-off-by: Daniel Lehrner <[email protected]>

* handle invalid authorization signatures properly

Signed-off-by: Daniel Lehrner <[email protected]>

* refactored CodeDelegationProcessor to compute authorizer of a code delegation after the chain id check

Signed-off-by: Daniel Lehrner <[email protected]>

* fix canSetDelegatedCode method by checking code how it is in the trie and not the resolved code

Signed-off-by: Daniel Lehrner <[email protected]>

* optimize code hash calculation for empty code, fix check for empty code delegation list

Signed-off-by: Daniel Lehrner <[email protected]>

* check the all code delegation signatures hava a valid s value

Signed-off-by: Daniel Lehrner <[email protected]>

* rename encoder & decoder, handle invalid signature values in T8nExecutor

Signed-off-by: Daniel Lehrner <[email protected]>

* create the signatures for code delegation authorizations in T8nExecutor without checking if they are valid to test them later during the tx execution

Signed-off-by: Daniel Lehrner <[email protected]>

* check that recid is either 0 or 1

Signed-off-by: Daniel Lehrner <[email protected]>

* fixed acceptance tests, renamed the the remaining instances of set code to code delegation

Signed-off-by: Daniel Lehrner <[email protected]>

* fix delegate encoder & encoder unit tests

Signed-off-by: Daniel Lehrner <[email protected]>

* spotless

Signed-off-by: Daniel Lehrner <[email protected]>

* changed empty check for delegated accounts, fixed test

Signed-off-by: Daniel Lehrner <[email protected]>

* fix hasDelegatedCode method when code is null

Signed-off-by: Daniel Lehrner <[email protected]>

* run acceptance tests without deamon

Signed-off-by: Daniel Lehrner <[email protected]>

* disable CodeDelegationTransactionAcceptanceTest to check if it is causing the stuck ci pipeline

Signed-off-by: Daniel Lehrner <[email protected]>

* check if shouldTransferAllEthOfAuthorizerToSponsor is causing pipeline to stall

Signed-off-by: Daniel Lehrner <[email protected]>

* check if shouldCheckNonceAfterNonceIncreaseOfSender is causing pipeline to stall

Signed-off-by: Daniel Lehrner <[email protected]>

* check if closing the cluster after every test is causing pipeline to stall

Signed-off-by: Daniel Lehrner <[email protected]>

* spotless

Signed-off-by: Daniel Lehrner <[email protected]>

---------

Signed-off-by: Daniel Lehrner <[email protected]>
Co-authored-by: Danno Ferrin <[email protected]>
* wip decoupled parent block header from block creators

---------

Signed-off-by: Justin Florentine <[email protected]>
* Added help for setting up Stratum mining

Signed-off-by: ITStarMan100 <[email protected]>

* removed duplication from help message; fixed spotless

Signed-off-by: Sally MacFarlane <[email protected]>

---------

Signed-off-by: ITStarMan100 <[email protected]>
Signed-off-by: Sally MacFarlane <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>
…yperledger#7592)

* feat: exported metrics for current finalized and safe block number

Signed-off-by: Jones Ho <[email protected]>

* Update ethereum/core/src/main/java/org/hyperledger/besu/ethereum/chain/DefaultBlockchain.java

Co-authored-by: Fabio Di Fabio <[email protected]>
Signed-off-by: jonesho <[email protected]>

* Update ethereum/core/src/main/java/org/hyperledger/besu/ethereum/chain/DefaultBlockchain.java

Co-authored-by: Fabio Di Fabio <[email protected]>
Signed-off-by: jonesho <[email protected]>

* fix: spotless issue

Signed-off-by: Jones Ho <[email protected]>

---------

Signed-off-by: Jones Ho <[email protected]>
Signed-off-by: jonesho <[email protected]>
Co-authored-by: Fabio Di Fabio <[email protected]>
* fix 7702 tx validation in tx pool

Signed-off-by: Daniel Lehrner <[email protected]>

* check that 7702 transactions always have a mandatory to field

Signed-off-by: Daniel Lehrner <[email protected]>

---------

Signed-off-by: Daniel Lehrner <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>
* remove unused classes

Signed-off-by: George Tebrean <[email protected]>

* removed duplication from changelog

Signed-off-by: Sally MacFarlane <[email protected]>

---------

Signed-off-by: George Tebrean <[email protected]>
Signed-off-by: George Tebrean <[email protected]>
Signed-off-by: Sally MacFarlane <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>
… initilization order (hyperledger#7607)

* Change test to tease NPEs during DefaultBlockchain.<init>() caused by initilization order
* fixup! Change test to tease NPEs during DefaultBlockchain.<init>() caused by initilization order
* move counters into their own method

Signed-off-by: Luis Pinto <[email protected]>
)

* Fix debug_traceCall to handle underpriced transactions

Signed-off-by: 7suyash7 <[email protected]>

* remove unused methods

Signed-off-by: 7suyash7 <[email protected]>

* Add test case and changelog entry for debug_traceCall fix

Signed-off-by: 7suyash7 <[email protected]>

---------

Signed-off-by: 7suyash7 <[email protected]>
Signed-off-by: Fabio Di Fabio <[email protected]>
Co-authored-by: Fabio Di Fabio <[email protected]>
…perledger#7580)

Signed-off-by: ITStarMan100 <[email protected]>
Co-authored-by: Justin Florentine <[email protected]>
Co-authored-by: Sally MacFarlane <[email protected]>
* upcoming breaking change for k8s nat
* log warning if deprecated NAT method used

Signed-off-by: Sally MacFarlane <[email protected]>

---------

Signed-off-by: Sally MacFarlane <[email protected]>
@matthew1001 matthew1001 marked this pull request as draft September 17, 2024 14:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

IBFT2 fails block validation with BONSAI DB