Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

crates_io_worker: Replace r2d2 with deadpool #8424

Merged
merged 3 commits into from
Apr 9, 2024

Conversation

Turbo87
Copy link
Member

@Turbo87 Turbo87 commented Apr 9, 2024

While #8385 ported the individual background worker jobs from r2d2 to deadpool, this PR ports the background worker system itself to it.

Note that the default async_pool_size for the test suite had to be increased because the background worker system now requires us to have one more connection available.

@Turbo87 Turbo87 added C-internal 🔧 Category: Nonessential work that would make the codebase more consistent or clear A-backend ⚙️ labels Apr 9, 2024
@Turbo87 Turbo87 requested a review from a team April 9, 2024 13:12
Copy link

codecov bot commented Apr 9, 2024

Codecov Report

Attention: Patch coverage is 86.95652% with 3 lines in your changes are missing coverage. Please review.

Project coverage is 87.61%. Comparing base (83b70cf) to head (bf1b85c).

Files Patch % Lines
src/bin/background-worker.rs 0.00% 2 Missing ⚠️
crates/crates_io_worker/src/runner.rs 90.90% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8424      +/-   ##
==========================================
- Coverage   87.62%   87.61%   -0.01%     
==========================================
  Files         272      272              
  Lines       26321    26303      -18     
==========================================
- Hits        23064    23046      -18     
  Misses       3257     3257              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@bors

This comment was marked as outdated.

Turbo87 added 3 commits April 9, 2024 17:58
With the background worker ported to `deadpool` it requires one extra connection, so setting it to two connections by default isn't sufficient anymore.
Comment on lines -8 to -22
pub async fn spawn_blocking<F, R, E>(f: F) -> Result<R, E>
where
F: FnOnce() -> Result<R, E> + Send + 'static,
R: Send + 'static,
E: Send + From<JoinError> + 'static,
{
let current_span = tracing::Span::current();
let hub = Hub::current();
tokio::task::spawn_blocking(move || current_span.in_scope(|| Hub::run(hub, f)))
.await
// Convert `JoinError` to `E`
.map_err(Into::into)
// Flatten `Result<Result<_, E>, E>` to `Result<_, E>`
.and_then(std::convert::identity)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it wrong that I'm actually most excited by this change?

@Turbo87 Turbo87 merged commit 1031565 into rust-lang:main Apr 9, 2024
8 checks passed
@Turbo87 Turbo87 deleted the deadpool-worker branch April 9, 2024 21:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-backend ⚙️ C-internal 🔧 Category: Nonessential work that would make the codebase more consistent or clear
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants