Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Tracking] Transfer data to OSN -> Virtualize OSN dataset with beam -> View dataset __repr__ in catalog #157

Open
norlandrhagen opened this issue Oct 9, 2024 · 2 comments

Comments

@norlandrhagen
Copy link
Contributor

norlandrhagen commented Oct 9, 2024

Meta tracking task to see if we can:

  1. Move a set of archival files off of NCAR or another cluster into LEAP's OSN bucket.
  2. Create a virtualizarr reference from the files on OSN.
  3. Add the virtual dataset reference to the LEAP catalog with a repr that includes: engine = 'kerchunk'. Can we configure the 'engine' argument in the displayed xarray/python repr? #150

cc @leap-stc/data-and-compute

@jbusecke
Copy link
Contributor

jbusecke commented Oct 9, 2024

@norlandrhagen this is the dataset I mentioned yesterday. We should figure this out with something smaller first, but I think this should be the first 'real world' target!

@jbusecke
Copy link
Contributor

jbusecke commented Oct 9, 2024

Another one could be Climsim

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants