diff --git a/PLUGINS.md b/PLUGINS.md index 645f3aa3..e6e53c0d 100644 --- a/PLUGINS.md +++ b/PLUGINS.md @@ -5,132 +5,128 @@ The profitability calculator is a shared component, that is used to check if a tx is profitable. It is applied, with different configuration to: 1. `linea_estimateGas` endpoint -2. Tx validation for the txpool +2. Tx validation for the txpool (if tx profitability check is enabled) 3. Tx selection during block creation #### CLI Options -| Option Name | Default Value | Command Line Argument | -|--------------------------|---------------|-------------------------------------------| -| L1_VERIFICATION_GAS_COST | 1_200_000 | `--plugin-linea-verification-gas-cost` | -| L1_VERIFICATION_CAPACITY | 90_000 | `--plugin-linea-verification-capacity` | -| L1_L2_GAS_PRICE_RATIO | 15 | `--plugin-linea-gas-price-ratio` | -| L2_GAS_PRICE_ADJUSTMENT | 0 wei | `--plugin-linea-gas-price-adjustment` | -| MIN_MARGIN | 1.0 | `--plugin-linea-min-margin` | -| ESTIMATE_GAS_MIN_MARGIN | 1.0 | `--plugin-linea-estimate-gas-min-margin` | -| TX_POOL_MIN_MARGIN | 0.5 | `--plugin-linea-tx-pool-min-margin` | -| UNPROFITABLE_CACHE_SIZE | 100_000 | `--plugin-linea-unprofitable-cache-size` | -| UNPROFITABLE_RETRY_LIMIT | 10 | `--plugin-linea-unprofitable-retry-limit` | -| TX_POOL_ENABLE_CHECK_API | true | `--plugin-linea-tx-pool-profitability-check-api-enabled` | -| TX_POOL_ENABLE_CHECK_P2P | false | `--plugin-linea-tx-pool-profitability-check-p2p-enabled` | +| Command Line Argument | Default Value | +|----------------------------------------------------------|---------------| +| `--plugin-linea-fixed-gas-cost-wei` | 0 | +| `--plugin-linea-variable-gas-cost-wei` | 1_000_000_000 | +| `--plugin-linea-extra-data-pricing-enabled` | false | +| `--plugin-linea-min-margin` | 1.0 | +| `--plugin-linea-estimate-gas-min-margin` | 1.0 | +| `--plugin-linea-tx-pool-min-margin` | 0.5 | + +### Module line count validator +The Module line count validator is a shared component, that is used to check if a tx exceeds any of the configured line count limits. +It is used in: +1. `linea_estimateGas` endpoint +2. Tx validation for the txpool (if tx simulation is enabled) +3. Tx selection during block creation + +#### CLI Options + +| Command Line Argument | Default Value | +|-------------------------------------------------------|----------------------| +| `--plugin-linea-module-limit-file-path` | moduleLimitFile.toml | +| `--plugin-linea-over-line-count-limit-cache-size` | 10_000 | + ### L1 L2 Bridge +These values are just passed to the ZkTracer #### CLI Options -| Option Name | Default Value | Command Line Argument | -|------------------------------|---------------|---------------------------------------------| -| L1L2_BRIDGE_CONTRACT_ADDRESS | | `--plugin-linea-l1l2-bridge-contract` | -| L1L2_BRIDGE_LOG_TOPIC | | `--plugin-linea-l1l2-bridge-topic` | +| Command Line Argument | Default Value | +|----------------------------------------|---------------| +| `--plugin-linea-l1l2-bridge-contract` | | +| `--plugin-linea-l1l2-bridge-topic` | | + ## Sequencer -### Transaction Selection - LineaTransactionSelectorPlugin +### Transaction Selection - LineaTransactionSelectorPlugin This plugin extends the standard transaction selection protocols employed by Besu for block creation. -It leverages the TransactionSelectionService to manage and customize the process of transaction selection. +It leverages the `TransactionSelectionService` to manage and customize the process of transaction selection. This includes setting limits such as `TraceLineLimit`, `maxBlockGas`, and `maxCallData`, and check the profitability of a transaction. - +The selectors are in the package `net.consensys.linea.sequencer.txselection.selectors`. #### CLI Options -| Option Name | Default Value | Command Line Argument | -|-------------------------------------|----------------------|-------------------------------------------------------| -| MAX_BLOCK_CALLDATA_SIZE | 70000 | `--plugin-linea-max-block-calldata-size` | -| MODULE_LIMIT_FILE_PATH | moduleLimitFile.toml | `--plugin-linea-module-limit-file-path` | -| OVER_LINE_COUNT_LIMIT_CACHE_SIZE | 10_000 | `--plugin-linea-over-line-count-limit-cache-size` | -| MAX_GAS_PER_BLOCK | 30_000_000L | `--plugin-linea-max-block-gas` | -| TX_POOL_ENABLE_SIMULATION_CHECK_API | true | `--plugin-linea-tx-pool-simulation-check-api-enabled` | -| TX_POOL_ENABLE_SIMULATION_CHECK_P2P | false | `--plugin-linea-tx-pool-simulation-check-p2p-enabled` | +| Command Line Argument | Default Value | +|--------------------------------------------------------|----------------------| +| `--plugin-linea-max-block-calldata-size` | 70000 | +| `--plugin-linea-module-limit-file-path` | moduleLimitFile.toml | +| `--plugin-linea-over-line-count-limit-cache-size` | 10_000 | +| `--plugin-linea-max-block-gas` | 30_000_000L | +| `--plugin-linea-unprofitable-cache-size` | 100_000 | +| `--plugin-linea-unprofitable-retry-limit` | 10 | -### Transaction validation - LineaTransactionValidatorPlugin +### Transaction Validation - LineaTransactionValidatorPlugin This plugin extends the default transaction validation rules for adding transactions to the -transaction pool. It leverages the PluginTransactionValidatorService to manage and customize the -process of transaction validation. This includes, for example, setting a deny list of addresses -that are not allowed to add transactions to the pool. +transaction pool. It leverages the `PluginTransactionValidatorService` to manage and customize the +process of transaction validation. +This includes setting limits such as `TraceLineLimit`, `maxTxGasLimit`, and `maxTxCallData`, and checking the profitability +of a transaction. +The validators are in the package `net.consensys.linea.sequencer.txpoolvalidation.validators`. #### CLI Options -| Option Name | Default Value | Command Line Argument | -|-------------------------|-------------------|---------------------------------------| -| DENY_LIST_PATH | lineaDenyList.txt | `--plugin-linea-deny-list-path` | -| MAX_TX_GAS_LIMIT_OPTION | 30_000_000 | `--plugin-linea-max-tx-gas-limit` | -| MAX_TX_CALLDATA_SIZE | 60_000 | `--plugin-linea-max-tx-calldata-size` | +| Command Line Argument | Default Value | +|----------------------------------------------------------|-------------------| +| `--plugin-linea-deny-list-path` | lineaDenyList.txt | +| `--plugin-linea-max-tx-gas-limit` | 30_000_000 | +| `--plugin-linea-max-tx-calldata-size` | 60_000 | +| `--plugin-linea-tx-pool-simulation-check-api-enabled` | false | +| `--plugin-linea-tx-pool-simulation-check-p2p-enabled` | false | +| `--plugin-linea-tx-pool-profitability-check-api-enabled` | true | +| `--plugin-linea-tx-pool-profitability-check-p2p-enabled` | false | + -## RPC +## RPC Methods ### Linea Estimate Gas #### `linea_estimateGas` -This endpoint simulates a transaction and returns the estimated gas used ( as the standard `eth_estimateGas`) plus the estimated gas price to be used when submitting the tx. +This endpoint simulates a transaction, including line count limit validation, and returns the estimated gas used ( as the standard `eth_estimateGas`) plus the estimated gas price to be used when submitting the tx. #### Parameters - same as `eth_estimateGas` -### Counters - CountersEndpointServicePlugin -#### `rollup_getTracesCountersByBlockNumberV0` - -The CountersEndpointServicePlugin registers an RPC endpoint named `getTracesCountersByBlockNumberV0` -under the `rollup` namespace. When this endpoint is called, returns trace counters based on the provided request parameters. - -#### Parameters - - - `blockNumber`: _string_ - The block number - - - `tracerVersion`: _string_ - The tracer version. It will return an error if the -requested version is different from the tracer runtime +#### Result +```json +{ + "jsonrpc": "2.0", + "id": 53, + "result": { + "gasLimit": "0x5208", + "baseFeePerGas": "0x7", + "priorityFeePerGas": "0x123456" + } +} +``` -### Trace generation - TracesEndpointServicePlugin +### Linea Set Extra Data +#### `linea_setExtraData` -This plugin registers an RPC endpoint named `generateConflatedTracesToFileV0` under the `rollup` namespace. -The endpoint generates conflated file traces. +This endpoint is used to configure the extra data based pricing, and it only makes sense to call it on the sequencer. +Internally it sets runtime pricing configuration and then calls, via the in-process RPC service, `miner_setExtraData` +and `miner_setMinGasPrice` to update internal Besu configuration, and add the extra data pricing to the future built blocks. #### Parameters - -- `fromBlock`: _string_ - the fromBlock number -- `toBlock`: _string_ - The toBlock number -- `tracerVersion`: _string_ - The tracer version. It will return an error if the - requested version is different from the tracer runtime - -## Continuous Tracing - -The continuous tracing plugin allows to trace every newly imported block and use Corset to check if the constraints are -valid. In case of an error a message will be sent to the configured Slack channel. - -### Usage - -The continuous tracing plugin is disabled by default. To enable it, use the `--plugin-linea-continuous-tracing-enabled` -flag. If the plugin is enabled it is mandatory to specify the location of `zkevm.bin` using -the `--plugin-linea-continuous-tracing-zkevm-bin` flag. The user with which the node is running needs to have the -appropriate permissions to access `zkevm.bin`. - -In order to send a message to Slack a webhook URL needs to be specified by setting the `SLACK_SHADOW_NODE_WEBHOOK_URL` -environment variable. An environment variable was chosen instead of a command line flag to avoid leaking the webhook URL -in the process list. - -The environment variable can be set via systemd using the following command: - -```shell -Environment=SLACK_SHADOW_NODE_WEBHOOK_URL=https://hooks.slack.com/services/SECRET_VALUES +same as `miner_setExtraData` with the added constraint that the number of bytes must be 32 + +#### Result +```json +{ + "jsonrpc": "2.0", + "id": 53, + "result": "true" +} ``` -### Invalid trace handling - -In the success case the trace file will simply be deleted. - -In case of an error the trace file will be renamed to `trace_$BLOCK_NUMBER_$BLOCK_HASH.lt` and moved -to `$BESU_USER_HOME/invalid-traces`. The output of Corset will be saved in the same directory in a file -named `corset_output_$BLOCK_NUMBER_$BLOCK_HASH.txt`. After that an error message will be sent to Slack. diff --git a/README.md b/README.md index c5380748..7c8da74d 100644 --- a/README.md +++ b/README.md @@ -1,33 +1,32 @@ # Besu Plugins related to tracer and sequencer functionality -A Linea tracing implementation for [Hyperledger Besu](https://github.com/hyperledger/besu) based on -an [existing implementation in Go](https://github.com/Consensys/zk-evm/). +A set of Linea plugins for the sequencer and RPC nodes. -## Quickstart - Running Besu with Linea Plugins +## Quickstart - Running [Linea Besu](https://github.com/ConsenSys/linea-besu) with plugins - compile linea-plugins `gradlew installDist` - copy jar file to besu runtime plugins/ directory (where you will run besu from, not where you're building besu) -- add `ROLLUP` to besu config to enable the plugin RPC methods - - rpc-http-api=\["ADMIN","ETH","NET","WEB3","ROLLUP"\] +- add `LINEA` to besu config to enable the plugin RPC methods + - rpc-http-api=\["ADMIN","ETH","NET","WEB3","LINEA"\] - start besu (command line or from IDE) and you should see plugins registered at startup -- call the RPC endpoint eg +- call the RPC endpoint eg: ```shell curl --location --request POST 'http://localhost:8545' --data-raw '{ "jsonrpc": "2.0", - "method": "rollup_generateConflatedTracesToFileV0", - "params": [0, 0, "6.16.0"], + "method": "linea_estimateGas", + "params": [ + "from": "0x73b2e0E54510239E22cC936F0b4a6dE1acf0AbdE", + "to": "0xBb977B2EE8a111D788B3477D242078d0B837E72b", + "value": "0x123" + ], "id": 1 }' ``` ## Development Setup -### Install Java 17 - -``` -brew install openjdk@17 -``` +### Install Java 21 ### Native Lib Prerequisites @@ -40,50 +39,7 @@ Windows On release native libs are built for all the supported platforms, if you want to test this process locally run `./gradlew -PreleaseNativeLibs jar`, -jar is generated in `arithmetization/build/libs`. - -### Install Rust - -``` -curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh - -# Use local git executable to fetch from repos (needed for private repos) -echo "net.git-fetch-with-cli=true" >> .cargo/config.toml -``` - -### Install Corset - -```shell -cargo install --git ssh://git@github.com/Consensys/corset -``` - -### Update Constraints [Submodule](https://github.com/Consensys/zkevm-constraints/) - -```shell -git submodule update --init --recursive -``` - -### Install [pre-commit](https://pre-commit.com/) - -```shell -pip install --user pre-commit - -# For macOS users. -brew install pre-commit -``` - -Then run `pre-commit install` to set up git hook scripts. -Used hooks can be found [here](.pre-commit-config.yaml). - -______________________________________________________________________ - -NOTE - -> `pre-commit` aids in running checks (end of file fixing, -> markdown linting, linting, runs tests, json validation, etc.) -> before you perform your git commits. - -______________________________________________________________________ +jar is generated in `sequencer/build/libs`. ### Run tests @@ -91,47 +47,10 @@ ______________________________________________________________________ # Run all tests ./gradlew clean test -# Run only unit tests -./gradlew clean unitTests - # Run only acceptance tests ./gradlew clean acceptanceTests - -# Run EVM test suite BlockchainTests -./gradlew clean referenceBlockchainTests - -# Run EVM test suite GeneralStateTests -./gradlew clean referenceGeneralStateTests - -# Run all EVM test suite reference tests -./gradlew clean referenceTests - -# Run single reference test via gradle, e.g for net.consensys.linea.generated.blockchain.BlockchainReferenceTest_583 -./gradlew :reference-tests:referenceTests --tests "net.consensys.linea.generated.blockchain.BlockchainReferenceTest_583" ``` -______________________________________________________________________ - -NOTE - -> Please be aware if the reference test code generation tasks `blockchainReferenceTests` and -> `generalStateReferenceTests` do not generate any java code, then probably you are missing the Ethereum tests" -> submodule which you can clone via `git submodule update --init --recursive`. - -______________________________________________________________________ - -### Capturing a replay - -For debugging and inspection purposes, it is possible to capture a _replay_, _i.e._ all the minimal information required to replay a series of blocks as they played on the blockchain, which is done with `scripts/capture.pl`. - -A typical invocation would be: - -``` -scripts/capture.pl --start 1300923 -``` - -which would capture a replay of block #1300923 and store it in `arithmetization/src/test/resources/replays`. More options are available, refer to `scripts/capture.pl -h`. - ## IntelliJ IDEA Setup ### Enable Annotation Processing @@ -160,30 +79,11 @@ NOTE ______________________________________________________________________ -### Set Up IDE Code Re-formatting - -- Install [Checkstyle](https://plugins.jetbrains.com/plugin/1065-checkstyle-idea) plugin and set IDE code - reformatting to comply with the project's Checkstyle configuration: - - - Go to `Settings | Editor | Code Style | Java | | Import Scheme | Checkstyle configuration`: - - ![idea_checkstyle_reformat.png](images/idea_checkstyle_reformat.png) - - and select `/config/checkstyle.xml`. - ### Install Optional Plugins - Install [Spotless Gradle](https://plugins.jetbrains.com/plugin/18321-spotless-gradle) plugin to re-format through the IDE according to spotless configuration. -## Debugging Traces - -- JSON files can be debugged with the following command: - -```shell -corset check -T -v zkevm-constraints/zkevm.bin -``` - ## Plugins Plugins are documented [here](PLUGINS.md). diff --git a/images/idea_checkstyle_reformat.png b/images/idea_checkstyle_reformat.png deleted file mode 100644 index 9fff93db..00000000 Binary files a/images/idea_checkstyle_reformat.png and /dev/null differ