Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] incorrect typing on get_s3_filesystem has led open_virtual_mfdataset astray #906

Open
1 task done
itcarroll opened this issue Dec 20, 2024 · 4 comments · May be fixed by #923
Open
1 task done

[BUG] incorrect typing on get_s3_filesystem has led open_virtual_mfdataset astray #906

itcarroll opened this issue Dec 20, 2024 · 4 comments · May be fixed by #923

Comments

@itcarroll
Copy link
Collaborator

Is this issue already tracked somewhere, or is this a new report?

  • I've reviewed existing issues and couldn't find a duplicate for this problem.

Current Behavior

The earthaccess.open_virtual_mfdataset function with access="direct" returns a KeyError: 0.

Here the earthaccess.get_s3_filesystem is given granules[0] when it should be given granules.

fs = earthaccess.get_s3_filesystem(results=granules[0])

Here the typing is misleading and should be Optional[list[DataGranule]].

results: Optional[DataGranule] = None,

Expected Behavior

The earthaccess.open_virtual_mfdataset function with access="direct" returns a dataset.

Steps To Reproduce

import earthaccess

results = earthaccess.search_data(
    short_name="MUR-JPL-L4-GLOB-v4.1",
    temporal=("2010-01-01", "2010-01-31"),
)
earthaccess.open_virtual_mfdataset(
    results,
    access="direct",
    load=False,
    concat_dim="time",
    coords="all",
    compat="override",
    combine_attrs="drop_conflicts",
)

Environment

- OS: openscapes.2i2c
- Python: 3.12.0

Additional Context

No response

@ayushnag
Copy link
Collaborator

ayushnag commented Dec 20, 2024

@betolink This should be a pretty quick fix. Can I also change this line of get_s3_filesystem to just use a single granule result rather than results[0]? Or should the type hint be modified to accept Optional[list[DataGranule]]?

@itcarroll
Copy link
Collaborator Author

itcarroll commented Dec 20, 2024

I suspect if you change the signature of get_s3_filesystem (results -> result) there would be much breakage among calls that pass a list. I think you want to correct the typing. If that breaks the mypy checks though, then I don't know!

Oh, and it looks like we use List from typing rather than list. Just noticed that.

@betolink
Copy link
Member

@ayushnag Agreed, I think using a list is still the recommended way to go and this is an artifact of some missions having their own endpoints like SWOT and NISAR etc. @itcarroll is there a difference in List vs list? which one should we use in this case?

@itcarroll
Copy link
Collaborator Author

typing.List is a deprecated alias to list. Matter of preference, so I guess I would just be consistent within any given module.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 🆕 New
Development

Successfully merging a pull request may close this issue.

3 participants