feat: Implement the Buf to avoid extra memory allocation #4090
Conversation
LYZJU2019
requested review from
evenyag,
v0y4g3r,
waynexia and
a team
as code owners
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #4090 +/- ##
==========================================
- Coverage 84.89% 84.62% -0.28%
==========================================
Files 1040 1040
Lines 183506 183506
==========================================
- Hits 155789 155285 -504
- Misses 27717 28221 +504 |
WenyXu
changed the title
|
The extra memory allocation happened in the |
| @@ -46,6 +46,7 @@ log-store = { workspace = true, optional = true } | |||
| memcomparable = "0.2" | |||
| moka = { workspace = true, features = ["sync", "future"] } | |||
| object-store.workspace = true | |||
| opendal = { version = "*" } | |||
There was a problem hiding this comment.
Let's re-export the Buffer via object-store crate
There was a problem hiding this comment.
should it be store-api crate instead of object-store crate?
There was a problem hiding this comment.
How about letting the into_bytes return an enum and importing the object-store in the mito crate?
There was a problem hiding this comment.
@LYZJU2019 Hi, thank you for your contribution. Could you refactor it as @WenyXu said? Let's merge this PR after the refactoring.
|
The |
|
Are we supposed to change the return value type of into_bytes() method? I cannot think of a way to avoid extra memory allocation when we have to flat a 2d array into an 1d array. |
Yes |
|
Warning Rate limit exceeded@tisonkun has exceeded the limit for the number of commits or files that can be reviewed per hour. Please wait 47 minutes and 48 seconds before requesting another review. How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (invoked as PR comments)
Additionally, you can add CodeRabbit Configration File (
|
|
completed in #4585 |
I hereby agree to the terms of the GreptimeDB CLA.
Refer to a related PR or issue link (optional)
#4065
What's changed and what's your intention?
implement Buffer to avoid extra memory allocation in file
src/mito2/src/wal/entry_reader.rs.Checklist