Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

reduce memory consumption during clone #1700

Merged
merged 1 commit into from
Nov 24, 2024
Merged

reduce memory consumption during clone #1700

merged 1 commit into from
Nov 24, 2024

Conversation

Byron
Copy link
Member

@Byron Byron commented Nov 24, 2024

This PR attemps to reuse memory consumption when dealing with pack traversal by trying to avoid over-allocations
due to with_capacity().

…y large vectors

However, when testing it seems to have no effect at all.
It's not faster, not slower, and has the same memory consumption.

```
❯ hyperfine -w1 -N "gix free pack verify $PWD/tests/fixtures/repos/linux.git/objects/pack/pack-b73c0dd365b8ab881a163a1d342dac7416f52591.idx" "./target/release/gix free pack verify $PWD/tests/fixtures/repos/linux.git/objects/pack/pack-b73c0dd365b8ab881a163a1d342dac7416f52591.idx"
Benchmark 1: gix free pack verify /Users/byron/dev/github.com/GitoxideLabs/gitoxide/tests/fixtures/repos/linux.git/objects/pack/pack-b73c0dd365b8ab881a163a1d342dac7416f52591.idx
  Time (mean ± σ):      5.161 s ±  0.078 s    [User: 56.153 s, System: 2.147 s]
  Range (min … max):    5.040 s …  5.318 s    10 runs

Benchmark 2: ./target/release/gix free pack verify /Users/byron/dev/github.com/GitoxideLabs/gitoxide/tests/fixtures/repos/linux.git/objects/pack/pack-b73c0dd365b8ab881a163a1d342dac7416f52591.idx
  Time (mean ± σ):      5.144 s ±  0.067 s    [User: 55.416 s, System: 2.162 s]
  Range (min … max):    5.042 s …  5.245 s    10 runs

Summary
  ./target/release/gix free pack verify /Users/byron/dev/github.com/GitoxideLabs/gitoxide/tests/fixtures/repos/linux.git/objects/pack/pack-b73c0dd365b8ab881a163a1d342dac7416f52591.idx ran
    1.00 ± 0.02 times faster than gix free pack verify /Users/byron/dev/github.com/GitoxideLabs/gitoxide/tests/fixtures/repos/linux.git/objects/pack/pack-b73c0dd365b8ab881a163a1d342dac7416f52591.idx
```
@EliahKagan
Copy link
Member

Am I correct to think that this helps with #1699 (reply in thread)?

@Byron
Copy link
Member Author

Byron commented Nov 24, 2024

It's an attempt to improve memory consumption, but it's also ineffective.

Unfortunately I am unable to profile memory consumption on MacOS as instruments fails to attach itself to the binary, despite having given it the respective entitlements. It seems ad-hoc signatures aren't supported, maybe.
I could probably use a linux VM to do the same.

@Byron Byron marked this pull request as ready for review November 24, 2024 09:26
@Byron Byron merged commit 54ea266 into main Nov 24, 2024
20 checks passed
@Byron Byron deleted the reduce-memory branch November 24, 2024 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants