Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI(Travis): Set python version to 3.10 through pyenv global #4731

Merged
merged 10 commits into from
Nov 21, 2024

Conversation

echoix
Copy link
Member

@echoix echoix commented Nov 21, 2024

TL;DR: I resigned myself to use the pyenv available on the Travis runner to set the global python version to 3.10, already installed in the image.


I don't understand why or how Travis does this. Without any announcements, github repo activity, community forum post, docs update or nothing, in the last image update yesterday, the default python version went from python 3.10.2 (https://app.travis-ci.com/github/OSGeo/grass/jobs/628247792#L121) with gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0, to Python 3.7.17 (https://app.travis-ci.com/github/OSGeo/grass/jobs/628271965#L119). Python 3.10 is supposed to be the default version from ubuntu jammy's apt repos, so something is going on with how they create their images, or don't control enough.

Complicated by the fact that the availability to select what version of a language/tool depends on the "language: " set in the config. https://config.travis-ci.com/ref/language. Using "c" here doesn't have any configurable keys (https://config.travis-ci.com/ref/language/c), but if we took "python" or "ruby", or many others, it allows to set various keys of what versions to use. From what I read (mostly users complaining), the keys aren't respected when the "wrong" language type is used, including generic. Since most of the partial docs they have consist on linking to examples of projects that used Travis, without any permalinks, and these projects don't use Travis anymore, the only hints I got was that they use or used pyenv in at least one of their language build-environments. I encounter pyenv and use it when working on a cloud env on github (most of my grass programming time is on gitpod). So I know how it works, but it can be slow to have a different version, as you end up downloading and compiling from source. There was a hint somewhere talking about caching that build pyenv python version, but it isn't clear how to cache that on Travis, unless I go explicit and find the correct folders for that.

There's also many community posts complaining and requesting other python versions when they were released, with various time spans taken after release to be available. So I'm not sure whether I'd want to depend on them for these builds to work.

So, my approach would be to use "uv" to install python, and is way faster since they have some standalone builds. The thing is, full replacement of pyenv isn't quite there yet. If I look at what uv was even a month or 2 before on that subject, I suppose that it is a question of weeks for what I expect to do to work out of the box.

In the meantime, I'll be doing a couple of pushes to reverse-engineer what the Travis config is, and how it interacts with the pyenv installed (if it is there), and how to make sure the uv-installed python is sawn and used without any other aliasing or PATH changes. If this PR is ready, we might need to revisit in a month or two to make sure that we still use the best pattern for us.

uv is from the same authors of ruff, and since publicly announced in february, have made an incredibly fast and very versatile python dependency manager, package installer, venv creator, tool runner (like pipx), and all, but at the same fraction of the time as ruff. Its marvellous (on the things they do support).

…t up for non-python language build environments
@echoix echoix marked this pull request as draft November 21, 2024 00:24
@github-actions github-actions bot added the CI Continuous integration label Nov 21, 2024
@echoix echoix changed the title CI(Travis): Set python versions CI(Travis): Set python version to 3.10 through pyenv global Nov 21, 2024
@echoix echoix marked this pull request as ready for review November 21, 2024 02:04
Might become unavailable, it will remain dependant on what Travis installs in their images
@echoix
Copy link
Member Author

echoix commented Nov 21, 2024

This is now ready to review and merge.

I finally ended up having to play with the already installed pyenv python versions. This means that we still depend on the availability of these in the runner image. If ever they change, I left a pyenv versions call before setting the global version, so we could see what to choose from. I could've also set to install the requested version, skipping already installed and compiling from source otherwise, but it would be even worse, as we would need to ensure that python build dependencies are already installed in the runner, again to the mercy of the Travis image.

I specifically chose to set the "global" pyenv version instead of only "local" (for the current directory), or "shell", as we install outside that directory and if we'd want to run something like a smoke test, it would already work without having to discover why it fails.

For the merge message, We can keep the second to last commit message or the last two commit messages. I didn't do a rebase so we could still find the commits and the job runs for the Travis logs to discover the info used to achieve the final state

@echoix echoix requested a review from neteler November 21, 2024 10:04
@echoix echoix enabled auto-merge (squash) November 21, 2024 12:26
@echoix echoix merged commit f63024f into OSGeo:main Nov 21, 2024
24 of 26 checks passed
@echoix echoix deleted the travis-python3.9 branch November 21, 2024 14:18
@github-actions github-actions bot added this to the 8.5.0 milestone Nov 21, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Continuous integration
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants