Skip to content

Commit

Permalink
turn off client cleanup in multi-ensemble -> 1 client case
Browse files Browse the repository at this point in the history
  • Loading branch information
dougbrn committed Jan 31, 2024
1 parent a1c764b commit 23e692f
Show file tree
Hide file tree
Showing 2 changed files with 11 additions and 1 deletion.
10 changes: 9 additions & 1 deletion src/tape/ensemble.py
Original file line number Diff line number Diff line change
Expand Up @@ -500,9 +500,17 @@ def sample_objects(self, **kwargs):
object_subset.set_dirty(True)

# make a new ensemble
# TODO: Investigate shared client warning
if self.client is not None:
new_ens = Ensemble(client=self.client)

# turn off cleanups -- in the case where multiple ensembles are
# using a client, an individual ensemble should not close the
# client during an __exit__ or __del__ event. This means that
# the client will not be closed without an explicit client.close()
# call, which is unfortunate... not sure of an alternative way
# forward.
self.cleanup_client = False
new_ens.cleanup_client = False
else:
new_ens = Ensemble(client=False)

Check warning on line 515 in src/tape/ensemble.py

View check run for this annotation

Codecov / codecov/patch

src/tape/ensemble.py#L515

Added line #L515 was not covered by tests

Expand Down
2 changes: 2 additions & 0 deletions tests/tape_tests/test_ensemble.py
Original file line number Diff line number Diff line change
Expand Up @@ -853,6 +853,8 @@ def test_sample_objects(parquet_ensemble_with_divisions):
assert len(ens.object) == prior_obj_len
assert len(ens.source) == prior_src_len

ens.client.close()


def test_update_column_map(dask_client):
"""
Expand Down

0 comments on commit 23e692f

Please sign in to comment.