Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: upgrade foyer to 0.11.3 #18313

Merged
merged 20 commits into from
Sep 20, 2024
Merged

feat: upgrade foyer to 0.11.3 #18313

merged 20 commits into from
Sep 20, 2024

Conversation

MrCroxx
Copy link
Contributor

@MrCroxx MrCroxx commented Aug 29, 2024

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

  • Upgrade foyer to 0.11.3
  • Introduce foyer runtime config in config
  • Upgrade opentelemetry kits.

Checklist

  • I have written necessary rustdoc comments
  • I have added necessary unit tests and integration tests
  • I have added test labels as necessary. See details.
  • I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features Sqlsmith: Sql feature generation #7934).
  • My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
  • All checks passed in ./risedev check (or alias, ./risedev c)
  • My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)
  • My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

  • My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note

If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

@MrCroxx MrCroxx self-assigned this Aug 29, 2024
@MrCroxx MrCroxx requested a review from a team as a code owner August 29, 2024 08:34
@MrCroxx MrCroxx requested a review from xxchan August 29, 2024 08:34
@MrCroxx
Copy link
Contributor Author

MrCroxx commented Aug 29, 2024

cc @BugenZhao for the opentelemetry and tonic part.

Cargo.toml Outdated
@@ -78,7 +78,10 @@ license = "Apache-2.0"
repository = "https://github.com/risingwavelabs/risingwave"

[workspace.dependencies]
foyer = { version = "0.10.4", features = ["nightly", "mtrace"] }
foyer = { git = "https://github.com/foyer-rs/foyer", rev = "f8eae8c", features = [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not publishing a version?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let me first build an image of this PR. If everything works fine, I'll bump foyer version, otherwise I need to fix bugs first.

Cargo.lock Outdated
@@ -5353,7 +5384,7 @@ dependencies = [
"http 0.2.9",
"thiserror",
"tokio",
"tonic 0.10.2",
"tonic 0.11.0",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess this should be manually reverted

@@ -171,9 +171,13 @@ reclaimers = 4
recover_concurrency = 8
insert_rate_limit_mb = 0
indexer_shards = 64
compression = "none"
compression = "None"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is lower cased string still supported? (I don't care. Just ask)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, but I think nobody is using this field for now. And none is the default configuration. It's okay to switch to serde defauts now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

case-insensive config is slightly more user-friendly to me (both OK). But it seems other options (like algorithm = "Lru", recover_mode = "None") are also case-sensive, so might be better to keep them consistent.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 Let me try.

Copy link
Member

@xxchan xxchan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

others lgtm

@MrCroxx
Copy link
Contributor Author

MrCroxx commented Aug 29, 2024

@MrCroxx
Copy link
Contributor Author

MrCroxx commented Aug 30, 2024

Cargo.toml Outdated
@@ -181,10 +184,10 @@ tikv-jemallocator = { git = "https://github.com/risingwavelabs/jemallocator.git"
"stats",
], rev = "64a2d9" }
# TODO(http-bump): bump to use tonic 0.12 once minitrace-opentelemetry is updated
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can remove this TODO.

Cargo.toml Outdated
@@ -78,7 +78,10 @@ license = "Apache-2.0"
repository = "https://github.com/risingwavelabs/risingwave"

[workspace.dependencies]
foyer = { version = "0.10.4", features = ["nightly", "mtrace"] }
foyer = { git = "https://github.com/foyer-rs/foyer", rev = "7eeb783", features = [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no release on crates.io?

Copy link
Contributor Author

@MrCroxx MrCroxx Aug 30, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#18313 (comment)

Here. 🤣 I'm waiting for all related bugs are fixed.

@MrCroxx
Copy link
Contributor Author

MrCroxx commented Aug 30, 2024

@MrCroxx
Copy link
Contributor Author

MrCroxx commented Sep 9, 2024

image

Q7 regression. Let me check

@MrCroxx
Copy link
Contributor Author

MrCroxx commented Sep 13, 2024

foyer-rs/foyer#708

@MrCroxx MrCroxx changed the title feat: upgrade foyer to 0.11, introduce runtime config, upgrade otel feat: upgrade foyer to 0.11.3 Sep 20, 2024
@MrCroxx MrCroxx enabled auto-merge September 20, 2024 05:25
@MrCroxx
Copy link
Contributor Author

MrCroxx commented Sep 20, 2024

Performance issue solved. FYI: foyer-rs/foyer#708

image image image

@MrCroxx MrCroxx added this pull request to the merge queue Sep 20, 2024
Merged via the queue into main with commit 82ac534 Sep 20, 2024
33 of 34 checks passed
@MrCroxx MrCroxx deleted the xx/foyer branch September 20, 2024 06:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants