Reduced shared parquet file contention in testing #440
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Change Description
Since moving to dask-espr, we've seen increased test failures. In #434 there have been recent failures in
test_batch_by_band
where a flux column can sometimes be read in as NaNs during thefrom_parquet
call used in that test invocation. Interestingly this only seems to occur when the tests are run as a suite and not whentest_batch_by_band
is run in isolation.We can achieve the same effect by having the test build an ensemble using duplicate files of the source and object files. This appears to fix the test when run on github actions, but we should still investigate the underlying issue with
from_parquet
and then revert this change.Code Quality
Bug Fix Checklist