Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up Table.__repr__ and potentially other forms of data access #38

Merged
merged 1 commit into from
Nov 25, 2021

Conversation

astrofrog
Copy link
Member

@astrofrog astrofrog commented Nov 25, 2021

This speeds up the Table.__repr__ by avoiding repeated calls to CasaArrayWrapper.__getitem__ with the same arguments (which is not actually cached by dask, see dask/dask#8420). For the particular example I was trying this speeds things up from 20 seconds to 5 seconds. To speed this up further, I will need to try and profile the dask data access in detail, but at least this was a low hanging fruit improvement, and 5 seconds is not too bad considering the size of the tables we are accessing.

This goes towards addressing #36

@codecov-commenter
Copy link

codecov-commenter commented Nov 25, 2021

Codecov Report

Merging #38 (c5a5739) into main (95403b8) will decrease coverage by 0.05%.
The diff coverage is 22.22%.

❗ Current head c5a5739 differs from pull request most recent head b7112db. Consider uploading reports for the commit b7112db to get more accurate results
Impacted file tree graph

@@            Coverage Diff             @@
##             main      #38      +/-   ##
==========================================
- Coverage   44.94%   44.89%   -0.06%     
==========================================
  Files          17       17              
  Lines        2118     2125       +7     
==========================================
+ Hits          952      954       +2     
- Misses       1166     1171       +5     
Impacted Files Coverage Δ
casa_formats_io/casa_dask.py 22.30% <22.22%> (+0.35%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 95403b8...b7112db. Read the comment docs.

@astrofrog
Copy link
Member Author

Since this is reasonably straightforward and to make it easier to try out, I'll go ahead and merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants