Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve ISO 8601 parsing for temporal filter #486

Closed
wants to merge 8 commits into from

Conversation

itcarroll
Copy link
Collaborator

@itcarroll itcarroll commented Mar 6, 2024

Fixes #330.

As suggested by @jhkennedy, earthaccess will now pass a datetime object (or None) to cmr, essentially replacing the basic string parsing that cmr does (using datetime.strptime) with flexible and timezone aware parsing by dateutil.parser.

The change created the opportunity to create a new suite of tests for possible user inputs, that includes checking exceptions.


📚 Documentation preview 📚: https://earthaccess--486.org.readthedocs.build/en/486/

@itcarroll itcarroll marked this pull request as draft March 6, 2024 03:57
@itcarroll itcarroll changed the title WIP: improve ISO 8601 parsing for temporal filter improve ISO 8601 parsing for temporal filter Mar 6, 2024
earthaccess/search.py Outdated Show resolved Hide resolved
earthaccess/search.py Outdated Show resolved Hide resolved
earthaccess/search.py Outdated Show resolved Hide resolved
itcarroll and others added 2 commits March 7, 2024 19:39
@itcarroll
Copy link
Collaborator Author

Believe the pre 3.10 checks are failing due to None | dt.datetime as a type hint for the return, which must be newer syntax.

@jhkennedy
Copy link
Collaborator

jhkennedy commented Mar 7, 2024

Believe the pre 3.10 checks are failing due to None | dt.datetime as a type hint for the return, which must be newer syntax.

Yes, using | instead of typing.Union is new as of Python 3.10. In general, you can write | like:

from typing import Union

def foo(bar: Union[int, float]):

For None | thing, specifically, there's a special Optional type:

from typing import Optional

def fizz(buzz: Optional[thing]):   # buzz can be None or thing

But that's an easy circle back once we're happy with the implementation

@itcarroll
Copy link
Collaborator Author

Test error is ModuleNotFoundError: No module named 'numpy'. How could the test environment be missing numpy?

Copy link
Collaborator

@jhkennedy jhkennedy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I don't love about the simplification here is the call to parse when it may not have been necessary (if we already had a datetime).

I think that roundtrip significantly simplifies the logic in the function; your latest version has nested branches/leafs which I find makes it harder to follow the logic all the way through without spending some time in the function.

That said, we've got really good tests now, so I'm fine with whatever implementation you prefer that passes the tests.

tests/unit/test_search.py Show resolved Hide resolved
tests/unit/test_search.py Show resolved Hide resolved
@itcarroll
Copy link
Collaborator Author

With nasa/python_cmr#31 merged, I believe we can close this as a won't fix in favor of a new PR that passes user's temporal input directly to python_cmr. Sound good?

@itcarroll itcarroll closed this Apr 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

dateutil parser breaks search for dates provided in ISO 8601 format by adding timezone offset
3 participants