positioned_io draft to prevent expensive Arc<Mutex<File>> #34
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Hi, I've noticed you're using Arc<Mutex>, which seems like a bottleneck for heavy parallellization. However a preliminary bench didn't yield as much improvement as I'd hoped, however could be due to the fact that those aren't utilizing threads as aggressively.
Anyway here is the code. Note that this is completely unrefined, I was using
Box::leak
to get it to compile. I don't have a real use case for this other than I was playing around a bit, so if you have no interest in this, that's fine for me.For a quick summary, you can read the positioned_io description. Basically it allows parallel reads on a
&File
(note the missing mut) without changing the file pos. It's also used byrc-zip
, which allows extracting in parallel. however it is mainly unmaintained and the underlying library seems to have UB, but investigating into this direction might be worth it.