Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EnsembleFrame Refactor #191

Closed
dougbrn opened this issue Aug 11, 2023 · 0 comments · Fixed by #308
Closed

EnsembleFrame Refactor #191

dougbrn opened this issue Aug 11, 2023 · 0 comments · Fixed by #308
Labels
enhancement New feature or request

Comments

@dougbrn
Copy link
Collaborator

dougbrn commented Aug 11, 2023

Currently, the TAPE ensemble is a pair of Dask Dataframes, the Object Dataframe and the Source Dataframe. We are feeling the limitations of only holding these two, primarily for handling downstream result dataframes. This issue tracks a refactor of the current structure of the Ensemble, that formalizes the EnsembleFrame Class. The EnsembleFrame will be a subclass of the Dask Dataframe, that has additional properties to allow the Ensemble class to track multiple EnsembleFrames and manage them (adding/dropping them, selecting them for running functions).

The scope of the refactor is to add the following:

  • Ensemble scope is greatly decreased, encompassing the management of EnsembleFrames and special handling of the ObjectFrame and SourceFrame
  • Introduce the EnsembleFrame object, inheriting from the Dask Dataframe class (and thereby getting all of the Dask API we were previously wrapping innately). Additionally includes properties to be managed by the Ensemble, and will also include some additional API functions that are astronomy/time-domain specific (flux to magnitude conversions for example).
  • Formalize the Source and Object Dataframes into the SourceFrame and ObjectFrame objects. These inherit from EnsembleFrame, but have more structure to follow the Object and Source format that TAPE expects to be present in a given dataset.

This refactor will be tracked by several milestone issues (#192, #193, #194, #195)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
1 participant