fix: outdated storage versions on Demo #1820

wischli · 2024-04-19T12:29:44Z

Description

There are a bunch of pallets which have a lower version than the latest one. Some of these our local pallets, others are frame ones. There are still some undecodable storage entries after applying these migrations but it is out of scope to clean up all of these.

Includes the reset of underlying rewards instance of BlockRewards. Apparently, the re-initialization is necessary!

Dev Logs

[2024-04-19T12:32:38Z INFO  runtime::collator-selection] Sorted 1 Invulnerables, upgraded storage to version 1
[2024-04-19T12:32:38Z INFO  development_runtime::migrations::cleanup_foreign_investments] CleanupForeignInvestments Storage cleanup can be skipped because all keys can be decoded
[2024-04-19T12:32:38Z INFO  development_runtime::migrations::cleanup_foreign_investments] CleanupForeignInvestments pre_upgrade done!
[2024-04-19T12:32:38Z INFO  development_runtime::migrations::cleanup_foreign_investments] CleanupForeignInvestments Initiating removal of undecodable keys
[2024-04-19T12:32:38Z INFO  development_runtime::migrations::cleanup_foreign_investments] CleanupForeignInvestments Removed 0 undecodable keys
[2024-04-19T12:32:38Z INFO  development_runtime::migrations::cleanup_foreign_investments] CleanupForeignInvestments post_upgrade done with 0 remaining storage keys!
[2024-04-19T12:32:38Z INFO  runtime::balances] Migration did not execute. This probably should be removed
[2024-04-19T12:32:38Z WARN  runtime_common::migrations::increase_storage_version] BumpStorageVersion: Mismatching versions. Wanted to upgrade from 0 to 1 but would instead upgrade from StorageVersion(1) to StorageVersion(1)
[2024-04-19T12:32:38Z WARN  pallet_xcm::migration::v1] skipping v1, should be removed
[2024-04-19T12:32:38Z WARN  runtime_common::migrations::increase_storage_version] BumpStorageVersion: Mismatching versions. Wanted to upgrade from 0 to 2 but would instead upgrade from StorageVersion(2) to StorageVersion(2)
[2024-04-19T12:32:38Z WARN  runtime_common::migrations::increase_storage_version] BumpStorageVersion: Mismatching versions. Wanted to upgrade from 0 to 3 but would instead upgrade from StorageVersion(3) to StorageVersion(3)
[2024-04-19T12:32:38Z WARN  runtime_common::migrations::increase_storage_version] BumpStorageVersion: Mismatching versions. Wanted to upgrade from 0 to 1 but would instead upgrade from StorageVersion(1) to StorageVersion(1)
[2024-04-19T12:32:38Z WARN  runtime_common::migrations::increase_storage_version] BumpStorageVersion: Mismatching versions. Wanted to upgrade from 0 to 2 but would instead upgrade from StorageVersion(2) to StorageVersion(2)
[2024-04-19T12:32:38Z WARN  runtime_common::migrations::increase_storage_version] BumpStorageVersion: Mismatching versions. Wanted to upgrade from 0 to 1 but would instead upgrade from StorageVersion(1) to StorageVersion(1)
[2024-04-19T12:32:38Z WARN  runtime_common::migrations::increase_storage_version] BumpStorageVersion: Mismatching versions. Wanted to upgrade from 0 to 1 but would instead upgrade from StorageVersion(1) to StorageVersion(1)
[2024-04-19T12:32:38Z WARN  runtime_common::migrations::increase_storage_version] BumpStorageVersion: Mismatching versions. Wanted to upgrade from 0 to 2 but would instead upgrade from StorageVersion(2) to StorageVersion(2)
[2024-04-19T12:32:38Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Pre migration checks successful
[2024-04-19T12:32:38Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Initiating migration
[2024-04-19T12:32:38Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Checking whether group is ready
[2024-04-19T12:32:38Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Checking whether session data is set
[2024-04-19T12:32:38Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Checking whether session data is set true
[2024-04-19T12:32:38Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Getting list of collators
[2024-04-19T12:32:38Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Checking stake of collator
[2024-04-19T12:32:38Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Migration not necessary because all data is already initialized
[2024-04-19T12:32:38Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Post migration checks successful

Demo Logs

[2024-04-19T12:32:01Z INFO  remote-ext] initialized state externalities with storage root 0xc90caff5ce7fc0bdbddc60dd180a0acbc17d23304c9262c3a1379375f5b40c3e and state_version V0
[2024-04-19T12:32:01Z INFO  try-runtime::cli] Original runtime [Name: RuntimeString::Owned("centrifuge-devel")] [Version: 1046] [Code hash: 0x1c61...4084]
[2024-04-19T12:32:01Z INFO  try-runtime::cli] New runtime      [Name: RuntimeString::Owned("centrifuge-devel")] [Version: 1047] [Code hash: 0x0b88...aa98]
[2024-04-19T12:32:02Z INFO  try_runtime_core::commands::on_runtime_upgrade] 🔬 Running TryRuntime_on_runtime_upgrade with checks: PreAndPost
[2024-04-19T12:32:02Z INFO  runtime::collator-selection] Sorted 3 Invulnerables, upgraded storage to version 1
[2024-04-19T12:32:02Z ERROR frame_support::storage] key failed to decode at [70, 74, 237, 145, 57, 25, 186, 185, 47, 121, 243, 199, 183, 157, 40, 247, 239, 186, 193, 94, 147, 243, 120, 17, 137, 94, 38, 6, 5, 205, 196, 135, 164, 245, 206, 214, 102, 137, 87, 187, 42, 154, 149, 78, 126, 80, 245, 181, 4, 0, 0, 0, 0, 0, 0, 0]: Error
[2024-04-19T12:32:02Z INFO  development_runtime::migrations::cleanup_foreign_investments] CleanupForeignInvestments Failed to decode 1 keys, cleanup necessary
[2024-04-19T12:32:02Z INFO  development_runtime::migrations::cleanup_foreign_investments] CleanupForeignInvestments pre_upgrade done!
[2024-04-19T12:32:02Z INFO  development_runtime::migrations::cleanup_foreign_investments] CleanupForeignInvestments Initiating removal of undecodable keys
[2024-04-19T12:32:02Z ERROR runtime::storage] Corrupted state at `[70, 74, 237, 145, 57, 25, 186, 185, 47, 121, 243, 199, 183, 157, 40, 247, 239, 186, 193, 94, 147, 243, 120, 17, 137, 94, 38, 6, 5, 205, 196, 135, 164, 245, 206, 214, 102, 137, 87, 187, 42, 154, 149, 78, 126, 80, 245, 181, 4, 0, 0, 0, 0, 0, 0, 0]: Error`
[2024-04-19T12:32:02Z INFO  development_runtime::migrations::cleanup_foreign_investments] CleanupForeignInvestments Removed 1 undecodable keys
[2024-04-19T12:32:02Z INFO  development_runtime::migrations::cleanup_foreign_investments] CleanupForeignInvestments post_upgrade done with 4 remaining storage keys!
[2024-04-19T12:32:02Z INFO  runtime::multisig] [2394239] ✍️ Number of calls to refund and delete: 0
[2024-04-19T12:32:02Z INFO  runtime::balances] Storage to version 1
[2024-04-19T12:32:02Z INFO  runtime_common::migrations::increase_storage_version] BumpStorageVersion: Increasing storage version of "Preimage" from StorageVersion(0) to StorageVersion(1)
[2024-04-19T12:32:02Z INFO  runtime::democracy::migration::v1] 0 public proposals will be migrated.
[2024-04-19T12:32:02Z INFO  runtime::democracy::migration::v1] 0 referenda will be migrated.
[2024-04-19T12:32:02Z INFO  runtime::democracy::migration::v1] 0 public proposals migrated, 0 referenda migrated
[2024-04-19T12:32:02Z ERROR runtime::storage] Corrupted state at `[227, 143, 24, 82, 7, 73, 138, 187, 92, 33, 61, 15, 176, 89, 179, 216, 216, 55, 28, 116, 117, 23, 21, 29, 134, 63, 146, 111, 129, 151, 24, 223, 191, 178, 127, 30, 174, 240, 107, 185, 3, 0, 0, 0, 42, 239, 46, 158, 78, 171, 249, 198, 214, 140, 245, 74, 179, 82, 164, 63, 3, 1, 1, 0, 161, 15]: Error`
[2024-04-19T12:32:02Z ERROR frame_support::storage] old key failed to decode at [227, 143, 24, 82, 7, 73, 138, 187, 92, 33, 61, 15, 176, 89, 179, 216, 216, 55, 28, 116, 117, 23, 21, 29, 134, 63, 146, 111, 129, 151, 24, 223, 191, 178, 127, 30, 174, 240, 107, 185, 3, 0, 0, 0, 42, 239, 46, 158, 78, 171, 249, 198, 214, 140, 245, 74, 179, 82, 164, 63, 3, 1, 1, 0, 161, 15]
[2024-04-19T12:32:02Z ERROR runtime::storage] Corrupted state at `[227, 143, 24, 82, 7, 73, 138, 187, 92, 33, 61, 15, 176, 89, 179, 216, 216, 55, 28, 116, 117, 23, 21, 29, 134, 63, 146, 111, 129, 151, 24, 223, 191, 178, 127, 30, 174, 240, 107, 185, 3, 0, 0, 0, 96, 15, 180, 143, 173, 7, 149, 151, 21, 112, 71, 148, 112, 212, 184, 115, 3, 1, 0]: Error`
[2024-04-19T12:32:02Z ERROR frame_support::storage] old key failed to decode at [227, 143, 24, 82, 7, 73, 138, 187, 92, 33, 61, 15, 176, 89, 179, 216, 216, 55, 28, 116, 117, 23, 21, 29, 134, 63, 146, 111, 129, 151, 24, 223, 191, 178, 127, 30, 174, 240, 107, 185, 3, 0, 0, 0, 96, 15, 180, 143, 173, 7, 149, 151, 21, 112, 71, 148, 112, 212, 184, 115, 3, 1, 0]
[2024-04-19T12:32:02Z INFO  pallet_xcm::migration::v1] v1 applied successfully
[2024-04-19T12:32:02Z INFO  runtime_common::migrations::increase_storage_version] BumpStorageVersion: Increasing storage version of "PoolSystem" from StorageVersion(0) to StorageVersion(2)
[2024-04-19T12:32:02Z INFO  runtime_common::migrations::increase_storage_version] BumpStorageVersion: Increasing storage version of "InterestAccrual" from StorageVersion(0) to StorageVersion(3)
[2024-04-19T12:32:02Z INFO  runtime_common::migrations::increase_storage_version] BumpStorageVersion: Increasing storage version of "Investments" from StorageVersion(0) to StorageVersion(1)
[2024-04-19T12:32:02Z INFO  runtime_common::migrations::increase_storage_version] BumpStorageVersion: Increasing storage version of "BlockRewards" from StorageVersion(0) to StorageVersion(2)
[2024-04-19T12:32:02Z INFO  runtime_common::migrations::increase_storage_version] BumpStorageVersion: Increasing storage version of "OraclePriceFeed" from StorageVersion(0) to StorageVersion(1)
[2024-04-19T12:32:02Z INFO  runtime_common::migrations::increase_storage_version] BumpStorageVersion: Increasing storage version of "OraclePriceCollection" from StorageVersion(0) to StorageVersion(1)
[2024-04-19T12:32:02Z INFO  runtime_common::migrations::increase_storage_version] BumpStorageVersion: Increasing storage version of "OrmlAssetRegistry" from StorageVersion(0) to StorageVersion(2)
[2024-04-19T12:32:02Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Pre migration checks successful
[2024-04-19T12:32:02Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Initiating migration
[2024-04-19T12:32:02Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Checking whether group is ready
[2024-04-19T12:32:02Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Checking whether session data is set
[2024-04-19T12:32:02Z ERROR runtime::storage] Corrupted state at `[194, 173, 136, 19, 144, 29, 204, 205, 246, 244, 182, 206, 17, 37, 119, 52, 86, 202, 82, 105, 5, 164, 29, 75, 234, 17, 156, 20, 152, 175, 123, 209]: Error`
[2024-04-19T12:32:02Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Checking whether session data is set false
[2024-04-19T12:32:02Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Getting list of collators
[2024-04-19T12:32:02Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Checking stake of collator
[2024-04-19T12:32:02Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Attaching currency to collator group
[2024-04-19T12:32:02Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Setting session data
[2024-04-19T12:32:02Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Adding stake for collator
[2024-04-19T12:32:02Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Adding stake for collator
[2024-04-19T12:32:02Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Adding stake for collator
[2024-04-19T12:32:02Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Migration complete
[2024-04-19T12:32:02Z INFO  pallet_block_rewards::migrations::init] InitBlockRewards Post migration checks successful
[2024-04-19T12:32:02Z INFO  runtime::frame-support] ⚠️ Ethereum declares internal migrations (which *might* execute). On-chain `StorageVersion(0)` vs current storage version `NoStorageVersionSet`

Checklist:

I have added Rust doc comments to structs, enums, traits and functions
I have made corresponding changes to the documentation
I have performed a self-review of my code
I have added tests that prove my fix is effective or that my feature works

codecov · 2024-04-19T13:39:36Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 48.55%. Comparing base (fbec2ed) to head (8eb0bf5).

❗ Current head 8eb0bf5 differs from pull request most recent head 7e718cd. Consider uploading reports for the commit 7e718cd to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1820      +/-   ##
==========================================
- Coverage   48.58%   48.55%   -0.03%     
==========================================
  Files         168      168              
  Lines       13359    13345      -14     
==========================================
- Hits         6490     6480      -10     
+ Misses       6869     6865       -4

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

lemunozm · 2024-04-19T17:20:08Z

runtime/development/src/migrations.rs

+	pallet_democracy::migrations::v1::v1::Migration<crate::Runtime>,
+	// v0 -> v1
+	pallet_xcm::migration::v1::VersionUncheckedMigrateToV1<crate::Runtime>,
+	runtime_common::migrations::increase_storage_version::Migration<crate::PoolSystem, 0, 2>,


FYI, we have now a legacy version for this: paritytech/substrate#14311

lemunozm · 2024-04-19T17:20:57Z

runtime/development/src/migrations.rs

 /// The migration set for Development & Demo.
 /// It includes all the migrations that have to be applied on that chain.
-pub type UpgradeDevelopment1046 =
-	pallet_collator_selection::migration::v1::MigrateToV1<crate::Runtime>;
+pub type UpgradeDevelopment1046 = (


Wow! a lot of migrations. How can we ensure this is done in the demo but not in dev? Is it because all of them are idempotent?

Yes exactly! Dev is up-to-date with all the latest versions whereas Demo is not. This PR fixes that.

lemunozm

Thanks for this!!

lemunozm

Reapproving!

wischli added 2 commits April 19, 2024 14:27

chore: update block rewards init migration

921e39f

fix: add missing demo storage version bumps

9eae281

wischli added the I3-annoyance The code behaves as expected, but "expected" is an issue. label Apr 19, 2024

wischli self-assigned this Apr 19, 2024

wischli requested a review from lemunozm April 19, 2024 12:29

Merge remote-tracking branch 'origin/main' into fix/demo-migrations

bcf2d82

wischli changed the title ~~fix: outdated storage versions on Demo~~ fix: outdated storage versions on Demo Apr 19, 2024

fix: pre-emptive version bump

b3073c5

lemunozm previously approved these changes Apr 19, 2024

View reviewed changes

Merge remote-tracking branch 'origin/main' into fix/demo-migrations

337eeaa

wischli dismissed lemunozm’s stale review via 337eeaa April 23, 2024 13:02

lemunozm previously approved these changes Apr 23, 2024

View reviewed changes

wischli added 2 commits April 23, 2024 16:50

refactor: nuke some pallets

398f01a

fmt: clippy

8eb0bf5

wischli dismissed lemunozm’s stale review via 8eb0bf5 April 23, 2024 15:57

Merge branch 'main' into fix/demo-migrations

7e718cd

wischli enabled auto-merge (squash) April 25, 2024 15:37

wischli requested a review from lemunozm April 25, 2024 15:38

lemunozm approved these changes Apr 25, 2024

View reviewed changes

wischli merged commit 5ab100b into main Apr 25, 2024
10 checks passed

lemunozm deleted the fix/demo-migrations branch April 25, 2024 16:14

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: outdated storage versions on Demo #1820

fix: outdated storage versions on Demo #1820

wischli commented Apr 19, 2024 •

edited

Loading

codecov bot commented Apr 19, 2024 •

edited

Loading

lemunozm Apr 19, 2024

lemunozm Apr 19, 2024

wischli Apr 23, 2024

lemunozm left a comment

lemunozm left a comment

fix: outdated storage versions on Demo #1820

fix: outdated storage versions on Demo #1820

Conversation

wischli commented Apr 19, 2024 • edited Loading

Description

Dev Logs

Demo Logs

Checklist:

codecov bot commented Apr 19, 2024 • edited Loading

Codecov Report

lemunozm Apr 19, 2024

Choose a reason for hiding this comment

lemunozm Apr 19, 2024

Choose a reason for hiding this comment

wischli Apr 23, 2024

Choose a reason for hiding this comment

lemunozm left a comment

Choose a reason for hiding this comment

lemunozm left a comment

Choose a reason for hiding this comment

wischli commented Apr 19, 2024 •

edited

Loading

codecov bot commented Apr 19, 2024 •

edited

Loading