Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: bump sdk to v1.20.0 #2860

Merged
merged 13 commits into from
Nov 22, 2023
Merged

chore: bump sdk to v1.20.0 #2860

merged 13 commits into from
Nov 22, 2023

Conversation

cmwaters
Copy link
Contributor

@cmwaters cmwaters commented Nov 20, 2023

Overview

This includes the change to persist the app version in state for v2 onwards

I opened this PR so docker could build the image I need to run the e2e tests (apparently opening them in draft mode doesn't work)

Checklist

  • New and updated code has appropriate documentation
  • New and updated code has new and/or updated testing
  • Required CI checks are passing
  • Visual proof for any user facing features like CLI or documentation updates
  • Linked issues closed with keywords

Summary by CodeRabbit

  • New Features

    • Implemented a new versioning system for the application.
    • Introduced a feature to log upgrade messages for better user awareness.
    • Added a new case to handle specific version formats in end-to-end tests.
  • Bug Fixes

    • Fixed the application version retrieval to ensure consistency across various functions.
  • Documentation

    • Updated test cases to reflect changes in version handling.
  • Refactor

    • Simplified the retrieval and setting of the application version in the codebase.
    • Enhanced the upgrade process by adjusting the upgrade height value and version checks.
  • Tests

    • Modified end-to-end tests to use a fixed version for consistency.
    • Skipped certain tests to accommodate new versioning logic.
  • Chores

    • Hardcoded the end-to-end latest version in the Makefile for test stability.

Copy link
Contributor

coderabbitai bot commented Nov 20, 2023

Walkthrough

The changes across the codebase reflect a shift in how the application version is managed and retrieved, moving from a protocol version setting to an application version context. The EndBlocker and InitChainer functions in app.go have been updated to accommodate this shift. Additionally, there's a focus on ensuring compatibility and handling of version-related logic, as seen in the test suites and the introduction of hardcoded version values for end-to-end testing. The Makefile and various test files have been adjusted to align with these versioning changes.

Changes

File(s) Change Summary
app/app.go Replaced SetProtocolVersion with SetAppVersion and removed setting protocol version in InitChainer.
app/prepare_proposal.go, app/process_proposal.go, app/square_size.go, test/util/malicious/out_of_order_prepare.go, x/upgrade/upgrade_test.go Updated AppVersion method calls to require sdkCtx parameter.
Makefile Changed E2E_LATEST_VERSION from dynamic git hash retrieval to hardcoded value ef40921 or b8d2449.
cmd/celestia-appd/cmd/root.go, x/upgrade/keeper.go Added import for "fmt" package and included print statements for upgrade detection and upgrade height.
test/e2e/simple_test.go, test/e2e/upgrade_test.go Added handling for latestVersion with length 7 and updated version assertions to use specific version constants. Introduced t.Skip() in tests.
x/blob/ante/max_total_blob_size_ante.go Changed the source of upperBound value from consensus parameters to block header version in getMaxSquareSize.
app/test/integration_test.go Added import statement for the package v2 and updated version assertions to use specific version constants.

Tips

Chat with CodeRabbit Bot (@coderabbitai)

  • If you reply to a review comment from CodeRabbit, the bot will automatically respond.
  • To engage with CodeRabbit bot directly around the specific lines of code in the PR, mention @coderabbitai in your review comment
  • Note: Review comments are made on code diffs or files, not on the PR overview.
  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

CodeRabbit Commands (invoked as PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger a review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai help to get help.

Note: For conversation with the bot, please use the review comments on code diffs or files.

CodeRabbit Configration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • The JSON schema for the configuration file is available here.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/coderabbit-overrides.v2.json

@cmwaters cmwaters marked this pull request as ready for review November 20, 2023 15:37
@celestia-bot celestia-bot requested a review from a team November 20, 2023 15:48
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 2

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 1eb203c and fc378bd.
Files ignored due to filter (2)
  • go.mod
  • go.sum
Files selected for processing (6)
  • app/app.go (2 hunks)
  • app/prepare_proposal.go (1 hunks)
  • app/process_proposal.go (1 hunks)
  • app/square_size.go (1 hunks)
  • test/util/malicious/out_of_order_prepare.go (1 hunks)
  • x/upgrade/upgrade_test.go (4 hunks)
Additional comments: 4
app/process_proposal.go (1)
  • 113-116: The ProcessProposal function has been correctly updated to pass the sdkCtx parameter to the AppVersion method, aligning with the changes described in the pull request summary. This ensures that the application version is retrieved in a context-aware manner, which is a key part of the versioning system update. The use of sdkCtx in GovSquareSizeUpperBound is also consistent with the new requirement for context-awareness in version retrieval.
app/prepare_proposal.go (1)
  • 62-68: The changes to the PrepareProposal method reflect the updated call to app.GetBaseApp().AppVersion(sdkCtx) and app.GovSquareSizeUpperBound(sdkCtx), which now require the sdkCtx parameter. This aligns with the pull request's goal of making the application version context-aware. It is important to ensure that all other parts of the code that rely on the AppVersion and GovSquareSizeUpperBound methods are updated to pass the sdkCtx parameter as well. Additionally, the error handling here uses a panic, which is appropriate for developer errors that should halt the node, but it's crucial to ensure that this behavior is consistent with the rest of the application's error handling strategy.
app/app.go (2)
  • 576-580: The code correctly updates the application version using SetAppVersion during the upgrade process. However, it is important to ensure that the SetAppVersion method is implemented correctly and that it handles the version update in a thread-safe manner, especially if the application is running in a concurrent environment. Additionally, it would be beneficial to verify that the ShouldUpgrade method of UpgradeKeeper is correctly determining when an upgrade should occur based on the provided height.

  • 587-591: The InitChainer function no longer sets the protocol version, which aligns with the summary stating a move away from protocol versioning to application versioning. This change should be carefully reviewed to ensure that it does not have unintended side effects on the initialization process of the blockchain, especially if there are any dependencies on the protocol version elsewhere in the codebase.

app/square_size.go Outdated Show resolved Hide resolved
test/util/malicious/out_of_order_prepare.go Show resolved Hide resolved
rootulp
rootulp previously approved these changes Nov 20, 2023
Copy link
Collaborator

@rootulp rootulp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there an issue that provides more context on why this change is being made? Can the PR description please link to it?

Comment on lines -590 to -592
if req.ConsensusParams != nil && req.ConsensusParams.Version != nil {
app.SetProtocolVersion(req.ConsensusParams.Version.AppVersion)
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this no longer necessary?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's done in the SDK instead

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Assuming it can be done in either location (app or SDK), why do it in the SDK where it has the risk of getting overwritten when we pull upstream changes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because this behaviour is logically the responsibility of baseapp not app

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then can it be upstreamed to cosmos/cosmos-sdk so it doesn't accidentally get reverted in celestiaorg/cosmos-sdk when we pull upstream changes?

go.mod Outdated
@@ -239,7 +239,7 @@ require (
)

replace (
github.com/cosmos/cosmos-sdk => github.com/celestiaorg/cosmos-sdk v1.18.3-sdk-v0.46.14
github.com/cosmos/cosmos-sdk => github.com/celestiaorg/cosmos-sdk v1.20.0-sdk-v0.46.16-rc0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[not blocking] why not use an official release of celestiaorg/cosmos-sdk? If rc-0 is just for testing, can we test pre merge instead of post-merge?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I plan to cut v1.20.0 once I've tested that the app version works as expected. I opened this PR so docker could build the image I need to run the e2e tests (apparently opening them in draft mode doesn't work)

staheri14
staheri14 previously approved these changes Nov 20, 2023
Copy link
Contributor

@staheri14 staheri14 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

app/app.go Show resolved Hide resolved
@cmwaters cmwaters dismissed stale reviews from staheri14 and rootulp via 9536657 November 21, 2023 10:07
@celestia-bot celestia-bot requested a review from a team November 21, 2023 10:08
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 7

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between fc378bd and 9536657.
Files selected for processing (6)
  • Makefile (1 hunks)
  • app/app.go (2 hunks)
  • cmd/celestia-appd/cmd/root.go (2 hunks)
  • test/e2e/simple_test.go (1 hunks)
  • test/e2e/upgrade_test.go (4 hunks)
  • x/upgrade/keeper.go (2 hunks)
Files skipped from review due to trivial changes (1)
  • cmd/celestia-appd/cmd/root.go
Additional comments: 3
app/app.go (1)
  • 588-592: The InitChainer function no longer sets the protocol version based on req.ConsensusParams.Version.AppVersion. Ensure that this change is intentional and that the protocol version is being set correctly elsewhere in the codebase to avoid issues with version compatibility and consensus.
test/e2e/upgrade_test.go (2)
  • 12-17: The addition of the new import statement for v1 "github.com/celestiaorg/celestia-app/pkg/appconsts/v1" is consistent with the changes described in the pull request summary, which mentions that the application version handling has been updated. This import is necessary for the updated assertion in the TestMajorUpgradeToV2 function.

  • 136-142: The added case for handling latestVersion with a length of 7 characters is likely related to the handling of git commit hashes, which are often abbreviated to 7 characters. This change should be verified to ensure that it aligns with the versioning strategy and that the corresponding docker image tags are indeed using the 7-character commit hash.

x/upgrade/keeper.go Outdated Show resolved Hide resolved
Makefile Outdated Show resolved Hide resolved
x/upgrade/keeper.go Outdated Show resolved Hide resolved
test/e2e/upgrade_test.go Outdated Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 2

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between 9536657 and a6b82fc.
Files ignored due to filter (2)
  • go.mod
  • go.sum
Files selected for processing (2)
  • app/app.go (3 hunks)
  • x/upgrade/upgrade_test.go (4 hunks)
Additional comments: 4
app/app.go (1)
  • 3-7: The import of the "fmt" package is added, but it is not clear if it is used elsewhere in the file. If "fmt" is not used, it should be removed to avoid importing unnecessary packages.
x/upgrade/upgrade_test.go (3)
  • 9-15: The import statements are well-organized and follow the Go convention of grouping standard library imports separately from third-party packages. However, ensure that all imported packages are used within the file to avoid unnecessary dependencies.

  • 22-30: The test TestUpgradeAppVersion has been updated to check the app version before and after a block is processed. The use of sdk.Context{} to create a new context for the AppVersion method is acceptable in a test environment, but it's important to ensure that this context is correctly initialized with all necessary values in a real-world scenario. The assertions are correctly checking the expected app version before and after the block, which is good for ensuring the upgrade logic is working as intended.

  • 42-49: > Note: This review was outside of the patch, so it was mapped to the patch with the greatest overlap. Original lines [42-71]

The setup function setupTestApp is correctly initializing the application and asserting the initial app version. The use of require.EqualValues ensures that the test will fail immediately if the app version does not match the expected value, which is a good practice for critical checks in tests. However, ensure that the genesisState is correctly configured to match the expected initial app version, as this is assumed in the test but not shown in the provided code.

app/app.go Outdated
Comment on lines 577 to 582
// NOTE: this is a specific feature for upgrading to v2 as v3 and onward is expected
// to be coordinated through the signalling protocol
if app.UpgradeKeeper.ShouldUpgrade(req.Height) {
app.SetProtocolVersion(v2.Version)
fmt.Println("upgrading to v2", "height", req.Height)
app.SetAppVersion(ctx, v2.Version)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code introduces a hard-coded check for an upgrade to version 2. While this is acknowledged in the comments as a one-off event, it is important to consider the implications of hard-coding such logic. Future upgrades will require changes to this code, which could be error-prone and not scalable. It's worth considering a more dynamic approach to version management, as suggested in previous comments.

evan-forbes
evan-forbes previously approved these changes Nov 21, 2023
Copy link
Member

@evan-forbes evan-forbes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dope thanks

this is currently blocking #2033

app/app.go Outdated Show resolved Hide resolved
app/app.go Outdated Show resolved Hide resolved
cmd/celestia-appd/cmd/root.go Outdated Show resolved Hide resolved
cmd/celestia-appd/cmd/root.go Outdated Show resolved Hide resolved
x/upgrade/keeper.go Outdated Show resolved Hide resolved
x/upgrade/keeper.go Outdated Show resolved Hide resolved
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Status

Actionable comments generated: 4

Configuration used: CodeRabbit UI

Commits Files that changed from the base of the PR and between a6b82fc and 856c42f.
Files selected for processing (4)
  • Makefile (1 hunks)
  • cmd/celestia-appd/cmd/root.go (2 hunks)
  • test/e2e/simple_test.go (2 hunks)
  • test/e2e/upgrade_test.go (4 hunks)
Files skipped from review due to trivial changes (1)
  • Makefile
Additional comments: 3
cmd/celestia-appd/cmd/root.go (2)
  • 4-9: The removal of the "strconv" package and associated parsing code is not shown in the provided hunk. If the import and code for "strconv" were indeed removed, ensure that there are no remaining references to "strconv" in the codebase that would lead to compilation errors.

  • 236-246: The use of cast.ToInt64(appOpts.Get(UpgradeHeightFlag)) directly retrieves the upgrade height value without manual parsing, which simplifies the code. However, ensure that the appOpts.Get method is guaranteed to return a value that can be cast to an int64 without error. If there's a possibility of a casting error, it should be handled appropriately to prevent runtime panics.

test/e2e/upgrade_test.go (1)
  • 177-181: The assertion on line 180 checks the app version against v1.Version before the upgrade. This is a good practice to ensure that the network is initially running on the expected version. However, ensure that v1.Version is correctly set to the expected version number before the upgrade.

test/e2e/upgrade_test.go Outdated Show resolved Hide resolved
test/e2e/upgrade_test.go Show resolved Hide resolved
test/e2e/simple_test.go Outdated Show resolved Hide resolved
@cmwaters cmwaters self-assigned this Nov 21, 2023
@cmwaters cmwaters marked this pull request as draft November 21, 2023 12:10
@cmwaters cmwaters marked this pull request as ready for review November 21, 2023 13:14
@cmwaters
Copy link
Contributor Author

This can now be reviewed.

I still need to cut the final v1.20.0 release and update it here

app/square_size.go Show resolved Hide resolved
app/app.go Show resolved Hide resolved
app/app.go Show resolved Hide resolved
Comment on lines 33 to 39
switch {
case isSemVer:
case latestVersion == "latest":
case len(latestVersion) == 7:
case len(latestVersion) == 8:
// assume this is a git commit hash (we need to trim the last digit to match the docker image tag)
latestVersion = latestVersion[:7]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The switch statement has cases that fall through without any action or comment, which can be confusing. Specifically, the cases for isSemVer, latestVersion == "latest", and len(latestVersion) == 7 do not have any code or comments. It's important to either implement the necessary logic or add comments explaining why no action is taken. This will improve the maintainability and readability of the code.

		switch {
		case isSemVer:
+			// Handle semantic versioning specific logic here, if any.
		case latestVersion == "latest":
+			// Handle the case when the latest version is denoted by the string "latest".
		case len(latestVersion) == 7:
+			// If the length of latestVersion is exactly 7 characters, no action is needed.
+			// This case assumes that the version is already in the correct format for further processing.
		case len(latestVersion) == 8:
			// assume this is a git commit hash (we need to trim the last digit to match the docker image tag)
			latestVersion = latestVersion[:7]

Commitable suggestion

[!IMPORTANT]
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation.

Suggested change
switch {
case isSemVer:
case latestVersion == "latest":
case len(latestVersion) == 7:
case len(latestVersion) == 8:
// assume this is a git commit hash (we need to trim the last digit to match the docker image tag)
latestVersion = latestVersion[:7]
switch {
case isSemVer:
// Handle semantic versioning specific logic here, if any.
case latestVersion == "latest":
// Handle the case when the latest version is denoted by the string "latest".
case len(latestVersion) == 7:
// If the length of latestVersion is exactly 7 characters, no action is needed.
// This case assumes that the version is already in the correct format for further processing.
case len(latestVersion) == 8:
// assume this is a git commit hash (we need to trim the last digit to match the docker image tag)
latestVersion = latestVersion[:7]

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea!

@cmwaters
Copy link
Contributor Author

It looks as though the ledger test is failing.

@rootulp, do you think actually downgrading the dependency here: celestiaorg/cosmos-sdk#357 (comment) is what is causing it to fail?

@rootulp
Copy link
Collaborator

rootulp commented Nov 21, 2023

I manually verified Ledger works when testing celestiaorg/cosmos-sdk#357 so I guess it could be a false negative with the test? Let's see if Ledger test passes on CI in #2865

Update: ledger test failed on that PR. I'll investigate

Copy link
Member

@evan-forbes evan-forbes left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dope

go.mod Show resolved Hide resolved
Comment on lines 33 to 39
switch {
case isSemVer:
case latestVersion == "latest":
case len(latestVersion) == 7:
case len(latestVersion) == 8:
// assume this is a git commit hash (we need to trim the last digit to match the docker image tag)
latestVersion = latestVersion[:7]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good idea!

@cmwaters cmwaters merged commit be3acf0 into main Nov 22, 2023
30 checks passed
@cmwaters cmwaters deleted the cal/bump-sdk branch November 22, 2023 22:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants