Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Publish dev releases to PyPi and automatize version computation #42

Open
benoit74 opened this issue May 31, 2024 · 1 comment
Open

Publish dev releases to PyPi and automatize version computation #42

benoit74 opened this issue May 31, 2024 · 1 comment

Comments

@benoit74
Copy link
Collaborator

benoit74 commented May 31, 2024

Disclaimer: this is an exploratory issue, I'm still not totally convinced this is the proper way of working, so far this is more a dump of which issue I have and what I've found which could help.

Context

Issue openzim/zimit#300 shows that it might be interesting to publish dev versions to Pypi for proper integration of warc2zim package inside zimit. This would also help to test dev versions more easily for external contributors. Note that by default, pip considers only stable versions, so this is not a problem ; should we want dev versions, we need to use the --pre flag.

Currently, we setup the version manually in main package __init__.py :

from great_project.__about__ import __version__

This however means that we need an automated way to bump dev versions for every commit on main branch. It would be good to not have to make a new commit just to bump the version, since this would double the number of commits for almost nothing, making the git history quite dirty.

At the same time, we currently rely on manual setup of the next release, meaning we have to manually make commits (again on main branch) at every release to push the proper version, and after release to prepare for next version. Since we also place a tag for the release, there is a significant chance of mismanipulation leading to discrepancy between the tag and the version in Pypi.

We use a major.minor.patch scheme for production, or major.minor.patch-devN for development.

Currently, right after a release we bump the version to the next minor and zero the patch. I.e. if version released is 1.2.3, we bump the version to 1.3.0-dev0. This is not quite appropriate, since we usually never know yet if the next code going to be merged to main is going to be for a patch, for a minor or for a major. Solving this would be great as well.

Proposition

  • publish dev versions to Pypi
  • remove the version from __init__.py and compute it automatically with versioningit : https://versioningit.readthedocs.io/en/stable/index.html
    • basically we use tags to compute the released version
    • for dev, the version is computed based on last tag + number of commits since this tag
  • do not commit current version anymore to source code, this information will be stored in git history
  • retrieve version at runtime from importlib.metadata.version("mypackage")

This means we need (but it is a good opportunity) to change our way of working:

  • by default, we assumes that next version is going to be a patch of latest released tag (released tag meaning a tag without a .devN suffix)
  • once we know we have significant changes needing a minor, the associated merge commit is tagged with the minor dev version (e.g. v1.3.0.dev0).
    • every subsequent commit is automatically numbered based on the distance to latest tag commit, hence being the dev version of next minor
  • should we decide to make breaking changes needing a major, again the merge commit is tagged with the major dev version (e.g. v2.0.0.dev0).
    • again, every subsequent commit is automatically numbered based on the distance to current commit, hence being the dev version of next major

Ideally the merge commit going to move from a patch to a minor or major release should be pushed first as a tag, then as a commit to main branch. This way when the CI trigger on push to main branch, the tag is already appropriately incremented (otherwise it will build a 1.2.x.devN instead of a 1.3.0.dev0 for instance). Since we know that this is not going to always be the case (people love to click the "Merge PR" in Github UI), we can modify the publish-dev workflow to also be triggered on push to tags matching the v*.dev0 pattern.

This means we need a custom next_version computation (don't know why it is not standard in versioningit) which:

  • compute the next patch if latest tag has no -dev suffix
    • e.g. if latest tag is 1.2.0 (because it has been released), then next commit on main branch will compute next_version as 1.2.1, which once formatted will become 1.2.1.dev1
  • keep the same released version if it has a -dev suffix
    • e.g. if latest tag is 1.3.0.dev0 (because we decided to bump to a minor), then next commit on main branch will compute next_version as 1.3.0, which once formatted will become 1.3.0.dev1

I propose to share this next_version method in hatch-openzim package since it is also used to distribute shared stuff across openZIM organization.

Final configuration:

[tool.versioningit.next-version]
method = { module = "hatch-openzim.versioningit", value = "next_version" }

[tool.versioningit.format]
distance = "{next_version}.dev{distance}+{vcs}{rev}"
# Example formatted version: 1.2.4.dev42+ge174a1f

dirty = "{base_version}+d{build_date:%Y%m%d}"
# Example formatted version: 1.2.3+d20230922

distance-dirty = "{next_version}.dev{distance}+{vcs}{rev}.d{build_date:%Y%m%d}"
# Example formatted version: 1.2.4.dev42+ge174a1f.d20230922

WDYT?

@rgaudin
Copy link
Member

rgaudin commented May 31, 2024

I'm sorry but I don't understand what problem we are trying to fix with this

Issue openzim/zimit#300 shows that it might be interesting to publish dev versions to Pypi for proper integration of warc2zim package inside zimit.

It's debatable ; that's not what I get from this discussion.

This would also help to test dev versions more easily for external contributors.

I disagree. External contributors who can't setup the dev environment? One of the main goal of this bootstrap is to ease the setup. We are talking about git clone xx ; cd xx ; pip install . vs pip install xxx==x.y.z.devA

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants