Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

compaso: add passthrough support #162

Merged
merged 3 commits into from
Dec 13, 2024
Merged

compaso: add passthrough support #162

merged 3 commits into from
Dec 13, 2024

Conversation

lgarrison
Copy link
Member

For the halo light cone catalogs in the Backlight project, we're doing halo finding on the time slices, then interpolating them onto the light cone with merger trees. Many time slice halos won't appear in the light cone, so there's no point in saving them. When we create the "squashed" catalogs that remove unused halos, we don't want to unpack all the columns, we want them to stay in their original format (e.g. we don't want to unpack rvint subsamples, because the floats won't compress as well). But we still want to load things through the CompaSOHaloCatalog interface, because it does all sorts of non-trivial merging of the uncleaned and cleaned halo info and subsamples, as well as row-based filtering, which we're going to use to get rid of unused halos.

This PR adds a passthrough bool argument to the CHC constructor to load halo info and subsamples without unpacking them. This effectively provides a way to get the raw data through CHC. Most users won't need this unless they're writing a pipeline where they want to defer the unpacking of the data to a later stage (often at the end-user stage, as CHC usually does).

passthrough lets us keep data in its unpacked format, which is useful when you want to defer the unpacking as part of a pipeline, for example.
@lgarrison lgarrison merged commit db865d1 into main Dec 13, 2024
8 checks passed
@lgarrison lgarrison deleted the passthrough branch December 13, 2024 22:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant