Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow user to select threads for image upload #900

Merged
merged 3 commits into from
Oct 30, 2024
Merged

Conversation

leftwo
Copy link
Contributor

@leftwo leftwo commented Oct 30, 2024

Added an option, thread-count to the image upload that allows a user to choose how many threads to create.

@leftwo leftwo requested a review from jmpesp October 30, 2024 18:58
@ahl ahl self-requested a review October 30, 2024 19:07
@ahl
Copy link
Collaborator

ahl commented Oct 30, 2024

Why would we expect users to set this value? And how would they know what value to set it to? Should the CLI typically try to upload as fast as possible?

If we're doing this because we've found that more threads increases upload speed for us, let's figure out something that's going to good in most cases. If we're trying to limit bandwidth consumption rather than maximize it, let's figure out the right knob to effect that.

@leftwo
Copy link
Contributor Author

leftwo commented Oct 30, 2024

While users could set this value themselves, I don't know how they would know what to set it to without experimenting.

The purpose here is mainly to work around oxidecomputer/omicron#6771 that we see in CI.
Having the ability to select threads for upload would allow CI jobs to work while we come up with a longer term solution.

Comment on lines 80 to 82
/// The number of parallel threads to use during upload
#[clap(long, default_value = "8")]
thread_count: usize,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// The number of parallel threads to use during upload
#[clap(long, default_value = "8")]
thread_count: usize,
/// The degree of parallelism to use during upload
#[clap(long, default_value = "8", hide = true)]
parallelism: usize,

Let's rename this to be more accurate (even if the underlying API is a little imprecise) and let's hide it so that customer aren't surprised when we remove it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

@ahl ahl merged commit 6d53a72 into main Oct 30, 2024
16 checks passed
@ahl ahl deleted the alan/upload-with-threads branch October 30, 2024 22:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants