Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Python 3.12 to automated test matrix #571

Merged
merged 9 commits into from
Jun 8, 2024

Conversation

chuckwondo
Copy link
Collaborator

@chuckwondo chuckwondo commented May 13, 2024

Fixes #457


📚 Documentation preview 📚: https://earthaccess--571.org.readthedocs.build/en/571/

mfisher87
mfisher87 previously approved these changes May 13, 2024
Copy link
Collaborator

@mfisher87 mfisher87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great as usual ;)

run: |
curl -sSL https://install.python-poetry.org | python3 -
echo "$HOME/.poetry/bin" >> $GITHUB_PATH
uses: abatilo/actions-poetry@v3
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

@@ -59,4 +63,6 @@ jobs:
EARTHACCESS_TEST_PASSWORD: ${{ secrets.EDL_PASSWORD }}
run: poetry run bash scripts/integration-test.sh
- name: Upload coverage
# Don't upload coverage when using the `act` tool to run the workflow locally
if: ${{ !env.ACT }}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TIL!

pyproject.toml Show resolved Hide resolved
@mfisher87
Copy link
Collaborator

Not sure about the build failure.

@chuckwondo
Copy link
Collaborator Author

@mfisher87, I sorted out the build errors by appropriately pinning numpy and pyproj based upon Python version.

Copy link
Collaborator

@mfisher87 mfisher87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Too bad we needed to set up those messy pins :(

I'm especially curious about pyproj, as the included proj database can be materially different in different versions. Would we be able to avoid these "dynamic" pins if we drop support for Python 3.8? Looks like numpy 1.26 and pyproj 3.6 have both dropped support for Python 3.8, so I think that's the only reason we need the dynamic pins?

@chuckwondo
Copy link
Collaborator Author

Too bad we needed to set up those messy pins :(

I'm especially curious about pyproj, as the included proj database can be materially different in different versions. Would we be able to avoid these "dynamic" pins if we drop support for Python 3.8?

Yes, I believe dropping support for Python 3.8 would do the trick, but do we want to drop support prior to its EOL, which is Oct 2024 (see https://devguide.python.org/versions/)?

@mfisher87
Copy link
Collaborator

Since members of our ecosystem (Numpy + Pyproj) are doing it already, I think it makes sense for us to follow suit. Numpy dropped support for 3.8 over a year ago. Here's their policy:

All minor versions of Python released 42 months prior, and at minimum the two latest minor versions are supported. Python support is only dropped in a major/minor version, and never on a patch release.

@chuckwondo
Copy link
Collaborator Author

Since members of our ecosystem (Numpy + Pyproj) are doing it already, I think it makes sense for us to follow suit. Numpy dropped support for 3.8 over a year ago. Here's their policy:

All minor versions of Python released 42 months prior, and at minimum the two latest minor versions are supported. Python support is only dropped in a major/minor version, and never on a patch release.

ok, I'll drop 3.8 and cleanup the pins

@chuckwondo
Copy link
Collaborator Author

Should I set min versions of numpy and pyproj to be the min versions that support Python 3.12? That is, do we want users using Python 3.9-3.11 to also get the min versions of numpy and pyproj that only support >=3.12, or is that too restrictive for dependency resolution?

More specifically, do we want these (regardless of Python version, user gets only what works for Python >=3.12):

numpy = { version = ">=1.26.0", optional = true }
pyproj = ">=3.6.1"

or these (looser for Python <3.12)?

numpy = { version = ">=1.26.0", optional = true, python = ">=3.12" }
pyproj = [
  { version = ">=3.5.0", python = "<3.12" },
  { version = ">=3.6.1", python = ">=3.12" }
]

cc: @betolink, @jhkennedy

@chuckwondo
Copy link
Collaborator Author

I've pinned numpy (optional) and pyproj to their versions that are compatible with Python 3.12. This means that those using Python <3.12 also get those 3.12-compatible versions. Do we want to loosen the pin to allow for earlier versions of numpy and pyproj for those using Python <3.12?

@mfisher87
Copy link
Collaborator

Do we want to loosen the pin to allow for earlier versions of numpy and pyproj for those using Python <3.12?

I think so. I think we should make our pins as loose as we can given our dependency policy (for now, safe to assume SPEC0, I think?) to maximize compatibility with users' environments.

We should probably include a pinning policy in our dependency policy :)

@chuckwondo
Copy link
Collaborator Author

Do we want to loosen the pin to allow for earlier versions of numpy and pyproj for those using Python <3.12?

I think so. I think we should make our pins as loose as we can given our dependency policy (for now, safe to assume SPEC0, I think?) to maximize compatibility with users' environments.

Thanks for confirming. I suspected as much. I'll adjust accordingly.

chuckwondo and others added 9 commits June 3, 2024 08:24
The following error occurs when building the docs, so we must explicitly
include `lxml-html-clean` as a dev dependency until `mkdocs-jupyter` is
updated:

    ImportError: lxml.html.clean module is now a separate project lxml_html_clean.
    Install lxml[html_clean] or lxml_html_clean directly.
Fix for: Backend 'setuptools.build_meta:__legacy__' is not available
@chuckwondo chuckwondo force-pushed the issue457-python-3.12 branch from 60b0150 to ff163cf Compare June 3, 2024 13:31
@chuckwondo chuckwondo requested a review from mfisher87 June 3, 2024 13:46
@chuckwondo
Copy link
Collaborator Author

@mfisher87, I loosened the numpy and pyproj versions for Python <3.12. I believe this should be good to go now.

I set pyproj to its previous pin, but added a new min version for Python >=3.12. For numpy, there was no existing pin, so I added a min version for Python >=3.12 (the version that numpy added support for 3.12), and a min version 2 versions earlier for Python <3.12.

@chuckwondo
Copy link
Collaborator Author

@mfisher87, as you requested, so this appears at the top of your queue :-)

Copy link
Collaborator

@mfisher87 mfisher87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, sorry for the delay. I appreciated the repeated reminder!

@chuckwondo chuckwondo merged commit 9dac08e into nsidc:main Jun 8, 2024
11 checks passed
@chuckwondo chuckwondo deleted the issue457-python-3.12 branch June 8, 2024 00:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Test against Python 3.12
2 participants