Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

revive: Include immutable storage deposit into the contracts storage_base_deposit #7230

Open
wants to merge 41 commits into
base: master
Choose a base branch
from

Conversation

athei
Copy link
Member

@athei athei commented Jan 18, 2025

This PR is centered around a main fix regarding the base deposit and a bunch of drive by or related fixtures that make sense to resolve in one go. It could be broken down more but I am constantly rebasing this PR and would appreciate getting those fixes in as-one.

This adds a multi block migration to Westend AssetHub that wipes the pallet state clean. This is necessary because of the changes to the ContractInfo storage item. It will not delete the child storage though. This will leave a tiny bit of garbage behind but won't cause any problems. They will just be orphaned.

Record the deposit for immutable data into the storage_base_deposit

The storage_base_deposit are all the deposit a contract has to pay for existing. It included the deposit for its own metadata and a deposit proportional (< 1.0x) to the size of its code. However, the immutable code size was not recorded there. This would lead to the situation where on terminate this portion wouldn't be refunded staying locked into the contract. It would also make the calculation of the deposit changes on set_code_hash more complicated when it updates the immutable data (to be done in #6985). Reason is because it didn't know how much was payed before since the storage prices could have changed in the mean time.

In order for this solution to work I needed to delay the deposit calculation for a new contract for after the contract is done executing is constructor as only then we know the immutable data size. Before, we just charged this eagerly in charge_instantiate before we execute the constructor. Now, we merely send the ED as free balance before the constructor in order to create the account. After the constructor is done we calculate the contract base deposit and charge it. This will make set_code_hash much easier to implement.

As a side effect it is now legal to call set_immutable_data multiple times per constructor (even though I see no reason to do so). It simply overrides the immutable data with the new value. The deposit accounting will be done after the constructor returns (as mentioned above) instead of when setting the immutable data.

Don't pre-charge for reading immutable data

I noticed that we were pre-charging weight for the max allowable immutable data when reading those values and then refunding after read. This is not necessary as we know its length without reading the storage as we store it out of band in contract metadata. This makes reading it free. Less pre-charging less problems.

Remove delegate locking

Fixes #7092

This is also in the spirit of making #6985 easier to implement. The locking complicates set_code_hash as we might need to block settings the code hash when locks exist. Check #7092 for further rationale.

Enforce "no terminate in constructor" eagerly

We used to enforce this rule after the contract execution returned. Now we error out early in the host call. This makes it easier to be sure to argue that a contract info still exists (wasn't terminated) when a constructor successfully returns. All around this his just much simpler than dealing this check.

Moved refcount functions to CodeInfo

They never really made sense to exist on Stack. But now with the locking gone this makes even less sense. The refcount is stored inside CodeInfo to lets just move them there.

Set CodeHashLockupDepositPercent for test runtime

The test runtime was setting CodeHashLockupDepositPercent to zero. This was trivializing many code paths and excluded them from testing. I set it to 30% which is our default value and fixed up all the tests that broke. This should give us confidence that the lockup doeposit collections properly works.

Reworked the MockExecutable to have both a deploy and a call entry point

This type used for testing could only have either entry points but not both. In order to fix the immutable_data_set_overrides I needed to a new function add_both to MockExecutable that allows to have both entry points. Make sure to make use of it in the future :)

@athei athei added the T7-smart_contracts This PR/Issue is related to smart contracts. label Jan 18, 2025
@athei athei requested a review from a team as a code owner January 23, 2025 00:06
@github-actions github-actions bot deleted a comment from athei Jan 23, 2025
@github-actions github-actions bot deleted a comment from athei Jan 23, 2025
@github-actions github-actions bot deleted a comment from athei Jan 23, 2025
@github-actions github-actions bot deleted a comment from athei Jan 23, 2025
@github-actions github-actions bot deleted a comment from athei Jan 23, 2025
@github-actions github-actions bot deleted a comment from athei Jan 23, 2025
…tend --pallet pallet_migrations --fail-fast --clean'
@github-actions github-actions bot deleted a comment from athei Jan 23, 2025
substrate/frame/revive/src/exec.rs Show resolved Hide resolved
substrate/frame/revive/src/wasm/mod.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@pgherveou pgherveou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, apart from the migration that probably need proper weight instead of just using RuntimeDbWeight

substrate/frame/revive/src/exec.rs Outdated Show resolved Hide resolved
substrate/frame/revive/src/wasm/mod.rs Outdated Show resolved Hide resolved
substrate/frame/support/src/migrations.rs Outdated Show resolved Hide resolved
@athei
Copy link
Member Author

athei commented Jan 27, 2025

/cmd bench-omni --runtime="dev asset-hub-westend people-rococo people-westend rococo westend" --pallet pallet_migrations --fail-fast --clean

@github-actions github-actions bot deleted a comment from athei Jan 27, 2025
@github-actions github-actions bot deleted a comment from athei Jan 27, 2025
@github-actions github-actions bot deleted a comment from athei Jan 27, 2025
@athei
Copy link
Member Author

athei commented Jan 27, 2025

/cmd bench-omni --runtime="asset-hub-westend" --pallet pallet_migrations --fail-fast

@mordamax
Copy link
Contributor

/cmd bench-omni --runtime dev asset-hub-westend people-rococo people-westend rococo westend --pallet pallet_migrations --fail-fast --clean

Copy link
Contributor

Command "bench-omni --runtime dev asset-hub-westend people-rococo people-westend rococo westend --pallet pallet_migrations --fail-fast --clean" has started 🚀 See logs here

…hub-westend people-rococo people-westend rococo westend --pallet pallet_migrations --fail-fast --clean'
Copy link
Contributor

Command "bench-omni --runtime dev asset-hub-westend people-rococo people-westend rococo westend --pallet pallet_migrations --fail-fast --clean" has finished ✅ See logs here

Subweight results:
File Extrinsic Old New Change [%]
cumulus/pallets/collator-selection/src/weights.rs leave_intent - - ERROR
cumulus/pallets/collator-selection/src/weights.rs new_session - - ERROR
cumulus/pallets/collator-selection/src/weights.rs register_as_candidate - - ERROR
cumulus/pallets/collator-selection/src/weights.rs set_invulnerables - - ERROR
cumulus/pallets/collator-selection/src/weights.rs take_candidate_slot - - ERROR
cumulus/pallets/collator-selection/src/weights.rs update_bond - - ERROR
cumulus/parachains/runtimes/assets/asset-hub-rococo/src/weights/pallet_collator_selection.rs take_candidate_slot - - ERROR
cumulus/parachains/runtimes/assets/asset-hub-rococo/src/weights/pallet_collator_selection.rs update_bond - - ERROR
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_collator_selection.rs take_candidate_slot - - ERROR
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_collator_selection.rs update_bond - - ERROR
cumulus/parachains/runtimes/bridge-hubs/bridge-hub-rococo/src/weights/pallet_collator_selection.rs take_candidate_slot - - ERROR
cumulus/parachains/runtimes/bridge-hubs/bridge-hub-rococo/src/weights/pallet_collator_selection.rs update_bond - - ERROR
cumulus/parachains/runtimes/bridge-hubs/bridge-hub-westend/src/weights/pallet_collator_selection.rs take_candidate_slot - - ERROR
cumulus/parachains/runtimes/bridge-hubs/bridge-hub-westend/src/weights/pallet_collator_selection.rs update_bond - - ERROR
cumulus/parachains/runtimes/collectives/collectives-westend/src/weights/pallet_collator_selection.rs take_candidate_slot - - ERROR
cumulus/parachains/runtimes/collectives/collectives-westend/src/weights/pallet_collator_selection.rs update_bond - - ERROR
cumulus/parachains/runtimes/collectives/collectives-westend/src/weights/pallet_preimage.rs ensure_updated - - ERROR
cumulus/parachains/runtimes/people/people-rococo/src/weights/pallet_collator_selection.rs take_candidate_slot - - ERROR
cumulus/parachains/runtimes/people/people-rococo/src/weights/pallet_collator_selection.rs update_bond - - ERROR
cumulus/parachains/runtimes/people/people-westend/src/weights/pallet_collator_selection.rs take_candidate_slot - - ERROR
cumulus/parachains/runtimes/people/people-westend/src/weights/pallet_collator_selection.rs update_bond - - ERROR
polkadot/runtime/westend/src/weights/pallet_preimage.rs ensure_updated - - ERROR
substrate/frame/election-provider-support/src/weights.rs phragmen - - ERROR
substrate/frame/election-provider-support/src/weights.rs phragmms - - ERROR
substrate/frame/revive/src/weights.rs seal_gas_limit 280.00ns 433.00ns +54.64
substrate/frame/revive/src/weights.rs seal_call_data_load 256.00ns 391.00ns +52.73
substrate/frame/revive/src/weights.rs seal_call_data_size 267.00ns 391.00ns +46.44
substrate/frame/revive/src/weights.rs seal_return_data_size 267.00ns 388.00ns +45.32
substrate/frame/revive/src/weights.rs seal_origin 265.00ns 379.00ns +43.02
substrate/frame/revive/src/weights.rs seal_value_transferred 274.00ns 390.00ns +42.34
substrate/frame/revive/src/weights.rs seal_caller_is_origin 338.00ns 477.00ns +41.12
substrate/frame/revive/src/weights.rs seal_address 271.00ns 382.00ns +40.96
substrate/frame/revive/src/weights.rs seal_ref_time_left 273.00ns 384.00ns +40.66
substrate/frame/revive/src/weights.rs seal_block_number 274.00ns 375.00ns +36.86
substrate/frame/revive/src/weights.rs seal_caller 305.00ns 416.00ns +36.39
substrate/frame/revive/src/weights.rs seal_minimum_balance 279.00ns 378.00ns +35.48
substrate/frame/revive/src/weights.rs seal_caller_is_root 299.00ns 403.00ns +34.78
substrate/frame/revive/src/weights.rs seal_base_fee 290.00ns 384.00ns +32.41
substrate/frame/revive/src/weights.rs seal_now 290.00ns 382.00ns +31.72
substrate/frame/revive/src/weights.rs seal_call_data_copy 30.22us 39.27us +29.98
substrate/frame/revive/src/weights.rs seal_own_code_hash 305.00ns 396.00ns +29.84
substrate/frame/revive/src/weights.rs get_transient_storage_empty 1.53us 1.85us +20.73
substrate/frame/revive/src/weights.rs seal_weight_left 732.00ns 883.00ns +20.63
polkadot/runtime/rococo/src/weights/pallet_migrations.rs on_init_loop 193.00ns 232.00ns +20.21
substrate/frame/revive/src/weights.rs seal_deposit_event 5.38us 6.35us +18.00
substrate/frame/revive/src/weights.rs seal_gas_price 323.00ns 377.00ns +16.72
substrate/frame/revive/src/weights.rs seal_copy_to_contract 53.96us 62.68us +16.17
substrate/frame/revive/src/weights.rs set_transient_storage_full 1.98us 2.29us +15.74
substrate/frame/revive/src/weights.rs set_transient_storage_empty 1.61us 1.86us +15.58
substrate/frame/revive/src/weights.rs get_transient_storage_full 1.71us 1.98us +15.54
substrate/frame/revive/src/weights.rs seal_return 54.79us 63.03us +15.04
substrate/frame/revive/src/weights.rs seal_contains_transient_storage 1.99us 2.26us +13.35
substrate/frame/revive/src/weights.rs seal_get_transient_storage 2.23us 2.51us +12.80
substrate/frame/revive/src/weights.rs seal_take_transient_storage 2.71us 3.05us +12.48
substrate/frame/revive/src/weights.rs seal_set_transient_storage 2.75us 3.07us +11.74
substrate/frame/revive/src/weights.rs seal_to_account_id 29.00us 32.17us +10.92
substrate/frame/revive/src/weights.rs seal_clear_transient_storage 2.53us 2.78us +9.96
substrate/frame/revive/src/weights.rs seal_weight_to_fee 1.49us 1.64us +9.65
polkadot/runtime/westend/src/weights/pallet_migrations.rs exec_migration_skipped_historic 60.96us 66.02us +8.30
polkadot/runtime/westend/src/weights/pallet_migrations.rs on_init_loop 193.00ns 207.00ns +7.25
substrate/frame/revive/src/weights.rs seal_sr25519_verify 1.39ms 1.47ms +5.74
cumulus/parachains/runtimes/people/people-westend/src/weights/pallet_migrations.rs exec_migration_advance 60.40us 57.32us -5.09
cumulus/parachains/runtimes/people/people-rococo/src/weights/pallet_migrations.rs on_init_loop 193.00ns 183.00ns -5.18
cumulus/parachains/runtimes/people/people-rococo/src/weights/pallet_migrations.rs exec_migration_skipped_historic 60.96us 57.66us -5.41
cumulus/parachains/runtimes/people/people-rococo/src/weights/pallet_migrations.rs exec_migration_advance 60.40us 57.11us -5.44
cumulus/parachains/runtimes/people/people-westend/src/weights/pallet_migrations.rs force_onboard_mbms 56.32us 53.23us -5.48
cumulus/parachains/runtimes/people/people-rococo/src/weights/pallet_migrations.rs force_onboard_mbms 56.32us 53.17us -5.59
substrate/frame/revive/src/weights.rs seal_delegate_call 112.09us 105.59us -5.80
substrate/frame/revive/src/weights.rs seal_balance 4.84us 4.52us -6.65
substrate/frame/revive/src/weights.rs on_process_deletion_queue_batch 28.01us 25.98us -7.23
substrate/frame/revive/src/weights.rs seal_get_immutable_data 37.52us 34.71us -7.49
substrate/frame/revive/src/weights.rs dispatch_as_fallback_account 63.90us 58.82us -7.95
substrate/frame/migrations/src/weights.rs progress_mbms_none 28.09us 25.75us -8.32
substrate/frame/migrations/src/weights.rs force_onboard_mbms 58.00us 53.11us -8.44
substrate/frame/revive/src/weights.rs seal_is_contract 35.34us 32.32us -8.54
substrate/frame/revive/src/weights.rs seal_code_hash 36.65us 33.42us -8.80
substrate/frame/migrations/src/weights.rs exec_migration_advance 62.80us 56.89us -9.40
substrate/frame/migrations/src/weights.rs on_init_loop 197.00ns 174.00ns -11.68
substrate/frame/revive/src/weights.rs get_storage_empty 36.28us 31.96us -11.91
substrate/frame/revive/src/weights.rs seal_get_storage 38.64us 33.97us -12.07
substrate/frame/revive/src/weights.rs seal_contains_storage 37.62us 33.04us -12.17
cumulus/parachains/runtimes/people/people-westend/src/weights/pallet_migrations.rs on_init_loop 193.00ns 168.00ns -12.95
substrate/frame/migrations/src/weights.rs exec_migration_skipped_historic 67.08us 57.63us -14.09
substrate/frame/revive/src/weights.rs seal_terminate 4.63ms 493.00us -89.36
substrate/frame/migrations/src/weights.rs reset_pallet_migration 257.88ms Added
polkadot/runtime/westend/src/weights/pallet_migrations.rs reset_pallet_migration 257.89ms Added
polkadot/runtime/rococo/src/weights/pallet_migrations.rs reset_pallet_migration 257.84ms Added
cumulus/parachains/runtimes/people/people-westend/src/weights/pallet_migrations.rs reset_pallet_migration 257.95ms Added
cumulus/parachains/runtimes/people/people-rococo/src/weights/pallet_migrations.rs reset_pallet_migration 257.92ms Added
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_migrations.rs reset_pallet_migration 257.92ms Added
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_migrations.rs progress_mbms_none 27.81us Added
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_migrations.rs onboard_new_mbms 159.00us Added
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_migrations.rs on_init_loop 188.00ns Added
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_migrations.rs force_set_cursor 102.92us Added
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_migrations.rs force_set_active_cursor 103.32us Added
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_migrations.rs force_onboard_mbms 56.76us Added
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_migrations.rs exec_migration_skipped_historic 62.54us Added
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_migrations.rs exec_migration_fail 164.12us Added
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_migrations.rs exec_migration_completed 131.46us Added
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_migrations.rs exec_migration_complete 163.27us Added
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_migrations.rs exec_migration_advance 61.58us Added
cumulus/parachains/runtimes/assets/asset-hub-westend/src/weights/pallet_migrations.rs clear_historic 32.44ms Added
Command output:

✅ Successful benchmarks of runtimes/pallets:
-- dev: ['pallet_migrations']
-- westend: ['pallet_migrations']
-- rococo: ['pallet_migrations']
-- asset-hub-westend: ['pallet_migrations']
-- people-rococo: ['pallet_migrations']
-- people-westend: ['pallet_migrations']

@athei
Copy link
Member Author

athei commented Jan 27, 2025

/cmd fmt

substrate/frame/migrations/src/migrations.rs Outdated Show resolved Hide resolved
substrate/frame/migrations/src/migrations.rs Outdated Show resolved Hide resolved
substrate/frame/migrations/src/migrations.rs Outdated Show resolved Hide resolved
substrate/frame/migrations/src/migrations.rs Outdated Show resolved Hide resolved
if meter.remaining().any_lt(required) {
return Err(SteppedMigrationError::InsufficientWeight { required })
}
P::on_genesis();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

on_genesis has some narrow meaning, I would not use it here.

We should use GetStorageVersion and implement & seal a common trait for NoVersionSet and StorageVersion that gives you the function write_to_storage or something like that. While NoVersionSet would write 0 as StorageVersion.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that we probably should not use OnGenesis here. But isn't the point of this NoVersionSet that no version should be set in storage? Meaning when NoVersionSet is the associated type we simply should not write any version?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But isn't the point of this NoVersionSet that no version should be set in storage?

No. 0 is always the default. https://docs.rs/frame-support/latest/frame_support/traits/struct.NoStorageVersionSet.html the docs are not ultra clear 🙈 But basically this is just a hint for a user that tries to implement a migration and forgets to set a storage_version.

@paritytech-review-bot paritytech-review-bot bot requested a review from a team January 28, 2025 15:02
@paritytech-workflow-stopper
Copy link

All GitHub workflows were cancelled due to failure one of the required jobs.
Failed workflow url: https://github.com/paritytech/polkadot-sdk/actions/runs/13013529585
Failed job name: test-linux-stable

return Err(SteppedMigrationError::InsufficientWeight { required })
}
P::on_genesis();
meter.consume(required);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: There is a try_consume to do the check and consumption in the if.

}

let base_weight = T::WeightInfo::reset_pallet_migration(0);
let weight_per_key = T::WeightInfo::reset_pallet_migration(1) - base_weight;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let weight_per_key = T::WeightInfo::reset_pallet_migration(1) - base_weight;
let weight_per_key = T::WeightInfo::reset_pallet_migration(1).saturating_sub(base_weight);

Am always a bit paranoid with this in the runtime 😅


if key_budget == 0 {
return Err(SteppedMigrationError::InsufficientWeight {
required: T::WeightInfo::reset_pallet_migration(1),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
required: T::WeightInfo::reset_pallet_migration(1),
required: T::WeightInfo::reset_pallet_migration(0).saturating_add(T::WeightInfo::reset_pallet_migration(1)),

Otherwise it cannot make progress?


if keys_before <= keys_now {
log::error!("ResetPallet<{}>: Did not remove any keys.", P::name());
Err("ResetPallet failed")?;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This and the as _ is some big brain code 😲

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
T7-smart_contracts This PR/Issue is related to smart contracts.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

revive: Remove delegate locking mechanism
6 participants