Release centrifuge 1024 client 1035 #1607

mustermeiszer · 2023-11-15T15:18:54Z

Description

Release candidate for Centrifuge 1024 runtime and v0.10.35 client.

Update weights
Bump versions

Checklist:

I have added Rust doc comments to structs, enums, traits and functions
I have made corresponding changes to the documentation
I have performed a self-review of my code
I have added tests that prove my fix is effective or that my feature works

wischli · 2023-11-15T18:14:29Z

Have you checked how many accounts need to be migrated? I dont think its realistic to do this in a single block migration. Why dont we use Parity's script?

cdamian · 2023-11-20T13:21:59Z

runtime/common/src/migrations/balances.rs

+				hex::encode(previous_key.clone())
+			);
+
+			/// The difference between the old and new account data structures


Tested by running:

./target/debug/centrifuge-chain try-runtime --runtime target/debug/wbuild/centrifuge-runtime/centrifuge_runtime.wasm --chain centrifuge on-runtime-upgrade live --uri wss://fullnode.centrifuge.io:443

Some entries decode without any issues, however, the majority fail when decoding to both old and new, also with "unexpected" values, as per the checks listed below.

After running the above, using the live centrifuge chain, the total number of account keys processed was - 102015.

cc @wischli

I suppose that number reflects the decodable acocunts? I'd be interested in the accounts with reserved and/or frozen balance.

We can add those as well, I'm concerned with the accounts that fail to decode into the old structure, I wasn't expecting that tbh.

wischli · 2023-11-20T14:14:14Z

runtime/common/src/migrations/balances.rs

+				Some(old) => {
+					if !old.data.fee_frozen.is_zero() {
+						log::warn!("Balances Migration - Old account data with non zero frozen fee")
+					}


If we eventually apply this migration, we should return counters for frozen and reserved accounts and check against these in post_upgrade.

No migration to be applied here from what I understand.

@wischli we just have these here in order to verify our storage prior to upgrading.

@cdamian already found some corrupted storage IIRC, and we wanted to 100% sure that the lazy migration parity is using actually works.

wischli · 2023-11-20T14:20:41Z

runtime/common/src/migrations/balances.rs

+		// * Research whether we need to migrate locks in orml (maybe first check if
+		//   there exist any. ^^


orml-tokens has not updated their AccountData yet:
https://github.com/open-web3-stack/open-runtime-module-library/blob/8f59b50ab45d15572d18cee0425966e43567a954/tokens/src/lib.rs#L142-L160

I would assume, they won't do that to not force consumers into synchronous migrations. In Substrate, the misc_frozen field of AccountData was replaced with flags. However, both fields have the same size which removes the requirement of a sync migration: https://github.com/paritytech/substrate/pull/12951/files#r1140113642

Yup, just thought that it might be a good idea to have similar checks like we do for the normal accounts. No actual migrations taking place, just us having some checks before the upgrade itself. WDYT?

I think that is a good idea!

…ance

cdamian · 2023-11-21T11:02:20Z

runtime/common/src/migrations/balances.rs

+		// TEST ACCOUNT DATA - START
+		let test_account_data = get_test_account_data::<T>();
+
+		for (account_id, account_data) in test_account_data {


@mustermeiszer @wischli do you think this should suffice or should we add more?

I think that suffices. Thanks a lot!

Note: on_runtime_upgrade already uses the new WASM sucht that we cannot write the old AccountInfo. Upgraded the test data to set new AccountInfo despite having a bit less purpose.

I was just setting the storage with the raw encoded value of the old account data. How does the WASM version affect this?

I don't have an answer for you but none of the accounts set during migration could be decoded whereas pre-existing ones with reserved or misc_frozen could. So it didn't simulate a real scenario.

On that note: Putting the raw encoded data for the new AccountInfo also led to decoding issues. I don't have capacity to look into this. Writing via pallet_balances::Account worked and was sufficient to me given that this is just a testing migration.

* Fix main branch detection for docker build * Try out GHA for Pages deployment * Docker build clean up workflow * publish docker in all non-PR events * fix docs cache (github token) * Deploy docs only from main branch * revert docker build machine size

* fix: blocking transfer debt by epoch and redemptions. * feat: make change_id deterministic * fix: build

Bumps [google-github-actions/auth](https://github.com/google-github-actions/auth) from 1.1.1 to 1.2.0. - [Release notes](https://github.com/google-github-actions/auth/releases) - [Changelog](https://github.com/google-github-actions/auth/blob/main/CHANGELOG.md) - [Commits](google-github-actions/auth@35b0e87...f105ef0) --- updated-dependencies: - dependency-name: google-github-actions/auth dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Guillermo Perez <[email protected]>

cdamian · 2023-11-24T13:31:33Z

runtime/common/src/migrations/balances.rs

+	let mut previous_key = prefix.clone();
+
+	while let Some(next) = sp_io::storage::next_key(&previous_key) {
+		if !next.starts_with(&prefix) {


Great catch! I also had this initially but removed it since I was trying to use the pallet_balances::Account prefix and wasn't getting any results... fun times.

chore: bump lock

fix: weights fix: allowlist weights

mustermeiszer

Can not approve but looks good!

wischli

Proxy-approving for @mustermeiszer

mustermeiszer added 3 commits November 15, 2023 10:39

feat: new weights altair

dbd5934

feat: new weights centrifuge

09bf25e

wip

6a5f505

migrations: Add migration for account data

0654dac

cdamian force-pushed the release-centrifuge_1024-client_1035 branch from 0fc419e to 0654dac Compare November 16, 2023 14:40

migrations: Adjust pre upgrade logic for accounts

4d7e8c5

cdamian reviewed Nov 20, 2023

View reviewed changes

wischli reviewed Nov 20, 2023

View reviewed changes

cdamian and others added 5 commits November 20, 2023 15:21

migrations: Log total count for accounts

44e5e7d

migrations: Log total count for accounts with frozen and reserved bal…

71a1fc5

…ance

Merge branch 'main' into release-centrifuge_1024-client_1035

0e52d38

deps: Remove double orml-tokens dep

6ef6cc5

migrations: Add test account data for balances

f8d871e

cdamian reviewed Nov 21, 2023

View reviewed changes

wischli and others added 8 commits November 22, 2023 16:52

fix: account info checks

08ff938

fix: local docker setup

c4c754e

CI release bug fixes (#1611)

e8fbe8a

* Fix main branch detection for docker build * Try out GHA for Pages deployment * Docker build clean up workflow * publish docker in all non-PR events * fix docs cache (github token) * Deploy docs only from main branch * revert docker build machine size

Fix: Transfer debt (#1615)

b80639b

* fix: blocking transfer debt by epoch and redemptions. * feat: make change_id deterministic * fix: build

fix: minor stuff balances migration

cb2e474

chore: bump development version, rm dummy migration

2dd0a6b

chore: bump MaxFeedValues from 10 to 500

205024a

wischli temporarily deployed to production November 22, 2023 17:20 — with GitHub Actions Inactive

wischli added I9-release A specific release. P7-asap Issue should be addressed in the next days. D2-notify Pull request can be merged and notification about changes should be documented. labels Nov 22, 2023

cdamian reviewed Nov 24, 2023

View reviewed changes

wischli added 2 commits December 4, 2023 11:47

chore: bump dev to 1035

bf0885e

chore: bump lock

fix: missed renaming

52ee34a

fix: weights fix: allowlist weights

wischli force-pushed the release-centrifuge_1024-client_1035 branch from 8fda552 to 52ee34a Compare December 4, 2023 10:47

wischli temporarily deployed to production December 4, 2023 10:48 — with GitHub Actions Inactive

wischli temporarily deployed to production December 4, 2023 14:46 — with GitHub Actions Inactive

wischli mentioned this pull request Dec 5, 2023

Sync StorageVersion of external dependencies #1641

Closed

5 tasks

chore: bump client to 0.10.35

f01b929

wischli temporarily deployed to production December 6, 2023 13:31 — with GitHub Actions Inactive

wischli temporarily deployed to production December 6, 2023 14:51 — with GitHub Actions Inactive

Merge branch 'main' into release-centrifuge_1024-client_1035

372606f

wischli temporarily deployed to production December 6, 2023 16:31 — with GitHub Actions Inactive

wischli temporarily deployed to production December 7, 2023 08:25 — with GitHub Actions Inactive

wischli self-requested a review December 7, 2023 08:28

wischli self-assigned this Dec 7, 2023

NunoAlexandre approved these changes Dec 7, 2023

View reviewed changes

mustermeiszer commented Dec 7, 2023

View reviewed changes

wischli approved these changes Dec 7, 2023

View reviewed changes

wischli merged commit ccbc3a1 into main Dec 7, 2023
17 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Release centrifuge 1024 client 1035 #1607

Release centrifuge 1024 client 1035 #1607

mustermeiszer commented Nov 15, 2023 •

edited by wischli

Loading

wischli commented Nov 15, 2023

cdamian Nov 20, 2023

cdamian Nov 20, 2023

wischli Nov 20, 2023

cdamian Nov 20, 2023

wischli Nov 20, 2023

cdamian Nov 20, 2023

mustermeiszer Nov 21, 2023

wischli Nov 20, 2023 •

edited

Loading

cdamian Nov 20, 2023

mustermeiszer Nov 21, 2023

cdamian Nov 21, 2023

wischli Nov 21, 2023

wischli Nov 22, 2023

cdamian Nov 24, 2023

wischli Nov 24, 2023

wischli Nov 24, 2023

cdamian Nov 24, 2023

mustermeiszer left a comment

wischli left a comment

		// * Research whether we need to migrate locks in orml (maybe first check if
		// there exist any. ^^

Release centrifuge 1024 client 1035 #1607

Release centrifuge 1024 client 1035 #1607

Conversation

mustermeiszer commented Nov 15, 2023 • edited by wischli Loading

Description

Checklist:

wischli commented Nov 15, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wischli Nov 20, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mustermeiszer left a comment

Choose a reason for hiding this comment

wischli left a comment

Choose a reason for hiding this comment

mustermeiszer commented Nov 15, 2023 •

edited by wischli

Loading

wischli Nov 20, 2023 •

edited

Loading