[compatible] Stop txn/network after slot feature #14516

joaosreis · 2023-11-05T19:35:32Z

TL;DR implements RFC #14138

This PR introduces two new optional compile-time configuration parameters, slot_tx_end and slot_chain_end. These define, respectively, the slot at which the node will stop producing/validating non-empty blocks (defined as being blocks that produce an empty staged ledger diff), and the slop at which the node stops producing/accepting any blocks at all. This is a feature part of the hard-fork mechanism.

Summary of changes:

Add optional slot-tx-end parameter to CLI.
BP always uses an empty set of transactions to produce blocks after the slot-tx-end slot instead of fetching them from the transaction pool.
The RPC function for sending user commands rejects commands after the slot-tx-end slot.
The node doesn't try to add incoming transactions to the network pool after the slot-tx-end slot.
The transaction pool drops all transactions when the slot-tx-end slot is reached.

Some unit tests (specifically, for the transaction pool) were added. The complete set of expected behaviors was demonstrated in a sandbox network.

Merge PRs to other branches:

entrypoint

integration_tests.mlh (to be reverted later)

…op-txn-after-slot-compatible

…tible

joaosreis · 2024-01-10T15:26:27Z

!ci-nightly-me

georgeee · 2024-01-18T17:04:38Z

There will be a configuration parameter set at compile-time that will define the slot at which the node will stop processing transactions.
The previous configuration cannot be overridable at runtime, by design, as this compromises the safety of the daemon software.

Does this excerpt from RFC mean that there should be no option to set the two parameters via CLI flag?

CC @mrmr1993 @nholland94

nholland94 · 2024-01-18T17:16:19Z

@georgeee Yes, there should not be a CLI flag. We did align in the RFC discussion to allow a runtime configuration override for integration testing purposes, which can be left undocumented.

georgeee · 2024-01-18T18:04:45Z

src/app/test_executive/slot_end_test.ml

+             ok_if_true msg
+               ( Mina_numbers.Global_slot.(
+                   of_uint32 block.slot_since_genesis < slot_tx_end)
+               || block.command_transaction_count = 0


NIT: I'd put conjunction group into braces for clarity (not necessary, but required me to remember operator precedence to be sure it works)

I certainly agree. It just turns out the formatter doesn't let me keep them :)

src/lib/network_pool/indexed_pool.ml

georgeee · 2024-01-18T18:23:42Z

src/lib/network_pool/indexed_pool.ml

-  match Map.find t.all_by_sender fee_payer with
-  | None ->
-      (* nothing queued for this sender *)
+  match slot_tx_end with


I suggest to flatten this code a little bit.

Something like the following would fit nicely:

let%bind () = Result.ok_if_true ~error:After_slot_tx_end @@ Option.value_map ~f:Global_slot.(fun slot_tx_end' -> current_global_slot >= slot_tx_end') ~default:true slot_tx_end in

TODO for George: re-review the code of the function after flattening (now changes are inflated by increase of layerness)

Good suggestion. In this case, the slot comparison condition needs to be inverted. Before it was the condition where it would fail, now it should be the condition where it should proceed.

georgeee · 2024-01-18T18:25:47Z

src/lib/network_pool/indexed_pool.ml

+        (of_time_exn ~constants:consensus_constants
+           (Block_time.now time_controller) ))
+  in
+  match slot_tx_end with


Please flatten this code as suggested above

georgeee · 2024-01-18T18:34:48Z

src/lib/network_pool/transaction_pool.ml

  end

  module Rejected = struct
    [%%versioned
    module Stable = struct
      [@@@no_toplevel_latest_type]

+      module V2 = struct


Note for future: discuss on how this affects berkeley versioning types (whould we bump some types' versions there too?)

This is the first time I'm updating versioned types, so I'm unsure on how this would affect
berkeley types.
Regardless, this PR has merge conflicts with berkeley and develop, so we need to merge this feature into there as well. We can then upgrade these types there. Does this solve the issue?

Yes, for sure. This is the way to proceed. My main point was that if we add a new numbered type here (V2) which existed in different form in berkeley, we probably should make sure to use V3 in berkeley. I.e. if types between compatible and berkeley differ, berkeley should be +1 from compatible. That said, if you will find an existing V3 in berkeley for a type that is V2 in compatible, it's safe to change this type right away without introducing a new number.

berkeley and develop are expected to have the same types, so once you remake this change against berkeley, there should be no problem with preparing a PR against develop.

georgeee · 2024-01-18T18:35:55Z

src/lib/network_pool/transaction_pool.ml

+          assert_pool_txs independent_cmds' ;
+          Deferred.unit )
+
+    let%test_unit "transactions added after slot_tx_end are rejected" =


Suggested change

let%test_unit "transactions added after slot_tx_end are rejected" =

let%test_unit "transactions added at slot_tx_end are rejected" =

Should we add a test case for "after"?

It surely doesn't hurt.

…op-txn-after-slot-compatible

joaosreis · 2024-01-22T12:14:40Z

@georgeee please give a look at the changes I just pushed.

…tible

joaosreis · 2024-01-24T08:44:40Z

!ci-build-me

…tible

georgeee · 2024-01-24T18:34:50Z

!ci-build-me

joaosreis · 2024-01-31T19:12:10Z

!ci-build-me

Nathan delegated this PR to me

joaosreis added 6 commits September 25, 2023 09:54

Add slot_tx_end to compile config

11dc67b

Create CLI flag. Add config to daemon config

7aefd8a

Create block after stop slot using empty staged ledger diff

b273c44

Split CLI flag into enable/disable

ef37faf

Add stop tx feature to validator

7cd90d5

Add stop slot feature to mina commands

6be8cd7

joaosreis force-pushed the joaosreis/hf-stop-txn-after-slot-compatible branch from e54c02f to f096e48 Compare November 5, 2023 22:21

joaosreis changed the title ~~Stop txn/network after slot feature [compatible]~~ [compatible] Stop txn/network after slot feature Dec 20, 2023

joaosreis force-pushed the joaosreis/hf-stop-txn-after-slot-compatible branch from d98b47f to 5ac8a89 Compare December 20, 2023 09:54

joaosreis added 15 commits December 20, 2023 09:56

Add slot_chain_end to compile config

c638cf0

Update slot transaction and chain end flags in CLI

f13a7a4

entrypoint

Convert slot numbers to global slot type

5aaba0a

Move txn slot check from mina_commands to graphql

fabb4c9

Add SlotTxEndTest to test suite

d9cdec8

Add slot_tx_end and slot_chain_end definitions to

d2f3649

integration_tests.mlh (to be reverted later)

Removed one BP

737b567

Fix stop tx slot comparison

c08d084

Add missing slot_chain_end features

07d6076

Better test structure

4628a04

Change build configs

cdd6a70

Remove CLI configuration flags. Use compile config directly.

840ea35

Adjustments to block production and validation. Add notifications

2560a66

Add slot_chain_end tests and other fixes to integration test

21ce836

Revert uneeded changes

da70971

joaosreis force-pushed the joaosreis/hf-stop-txn-after-slot-compatible branch from 5ac8a89 to da70971 Compare December 20, 2023 10:09

joaosreis added 2 commits December 20, 2023 16:14

Run test even when slot params are not set (validate usual behaviour)

1f3b8bc

Merge remote-tracking branch 'origin/compatible' into joaosreis/hf-st…

a440364

…op-txn-after-slot-compatible

joaosreis self-assigned this Dec 21, 2023

Reformat slot_end_test.ml

0b70f8a

joaosreis mentioned this pull request Dec 21, 2023

[compatible] Add parameter to stop the chain after given slot #13570

Closed

joaosreis added 5 commits January 10, 2024 09:34

Allow overriding slot_*_end with runtime config

ce59ea4

Reject transactions at transaction pool level

697b9af

Fix missing versioned type update

77fc86f

Remove unnecessary changes

dc8ca49

Merge branch 'compatible' into joaosreis/hf-stop-txn-after-slot-compa…

2073d43

…tible

joaosreis added the P0 Urgent label Jan 18, 2024

georgeee requested changes Jan 18, 2024

View reviewed changes

joaosreis added 3 commits January 22, 2024 12:03

Add new txn pool test case

8897d8b

Flatten logic

e1d5168

Merge remote-tracking branch 'origin/compatible' into joaosreis/hf-st…

9f75c63

…op-txn-after-slot-compatible

joaosreis requested a review from georgeee January 22, 2024 12:16

georgeee approved these changes Jan 22, 2024

View reviewed changes

joaosreis added 2 commits January 23, 2024 15:33

Revert format changes

8f19650

Merge branch 'compatible' into joaosreis/hf-stop-txn-after-slot-compa…

c9c86fa

…tible

Merge branch 'compatible' into joaosreis/hf-stop-txn-after-slot-compa…

cb1c7ec

…tible

ymekuria approved these changes Jan 25, 2024

View reviewed changes

joaosreis mentioned this pull request Jan 30, 2024

[rampup] Stop txn/network after slot feature #15000

Merged

Fix integration test slowness and flakiness

df06db5

georgeee merged commit 46a00e9 into compatible Jan 31, 2024
16 checks passed

georgeee deleted the joaosreis/hf-stop-txn-after-slot-compatible branch January 31, 2024 19:53

joaosreis mentioned this pull request Feb 2, 2024

Remove dependency from mina_numbers from compile config #15022

Merged

mrmr1993 mentioned this pull request Apr 11, 2024

Use Global_slot_since_genesis for stop slots #15513

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[compatible] Stop txn/network after slot feature #14516

[compatible] Stop txn/network after slot feature #14516

joaosreis commented Nov 5, 2023 •

edited

Loading

joaosreis commented Jan 10, 2024

georgeee commented Jan 18, 2024

nholland94 commented Jan 18, 2024

georgeee Jan 18, 2024

joaosreis Jan 22, 2024

georgeee Jan 18, 2024

georgeee Jan 18, 2024

joaosreis Jan 22, 2024

georgeee Jan 18, 2024

georgeee Jan 18, 2024

joaosreis Jan 22, 2024

georgeee Jan 22, 2024

georgeee Jan 22, 2024

georgeee Jan 18, 2024

georgeee Jan 18, 2024

joaosreis Jan 22, 2024

joaosreis commented Jan 22, 2024

joaosreis commented Jan 24, 2024

georgeee commented Jan 24, 2024

joaosreis commented Jan 31, 2024

	let%test_unit "transactions added after slot_tx_end are rejected" =
	let%test_unit "transactions added at slot_tx_end are rejected" =

[compatible] Stop txn/network after slot feature #14516

[compatible] Stop txn/network after slot feature #14516

Conversation

joaosreis commented Nov 5, 2023 • edited Loading

joaosreis commented Jan 10, 2024

georgeee commented Jan 18, 2024

nholland94 commented Jan 18, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

joaosreis commented Jan 22, 2024

joaosreis commented Jan 24, 2024

georgeee commented Jan 24, 2024

joaosreis commented Jan 31, 2024

joaosreis commented Nov 5, 2023 •

edited

Loading