Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CI job to publish on crates.io when a release is tagged #713

Merged
merged 2 commits into from
May 30, 2024

Conversation

ids1024
Copy link
Member

@ids1024 ids1024 commented Apr 18, 2024

This uses katyo/publish-crates@v1, which checks which crates have had their versions bumped, and releases them on crates.io. While making sure to publish dependencies first before the crate that depends on them. This has apparently worked well in drm-rs.

Since subcrates are versioned differently, when we discussed this earlier date-based tags names seemed best. So as @elinorbgr suggested, this goes with release-YYYY-MM-DD as the tagging scheme. (The CI job checks for tags prefixed with release-). I can't really think of anything better than that.

So to release, we should be able to simply update the versions in Cargo.toml files, update the changelog files, and tag a release.

The publish-crates action requires that all subcrates depends on the latest version of any other crate in the repository. And it requires that all crates with changes have a bumped version so they can be released. This seems a bit annoying if we wanted to just release a change to wayland-backend, for instance, but should help avoid mistakes. Manually dealing with the releases for all the crates here, without any automated checks, is error-prone.

The action here does a dry-run for all CI runs (and uses ignore-unpublished-changes to not error if crates have changes without already having their versions bumped). But disables dry-run when a tag is pushed with the right format.

A CRATES_TOKEN secret will need to be added to the repository for this to work. I am not currently a member of smithay/publishers on crates.io, and wayland-backend and wayland-protocols-* don't currently have smithay/publishers as an owner, so @elinorbgr will have to add that. (And make sure smithay/publishers is an owner of all the crates is probably a good idea.)

@ids1024 ids1024 requested a review from elinorbgr April 18, 2024 19:00
@kchibisov
Copy link
Member

We've used such a thing in e.g. glutin/winit and I'd say that it's very tricky. Especially when your crates depend on each self, etc. It's also very easy to mess things up.

Though, I don't object the changes.

@ids1024
Copy link
Member Author

ids1024 commented Apr 18, 2024

 Found packages: wayland-backend, wayland-egl, wayland-cursor, wayland-client, wayland-protocols, wayland-protocols-plasma, wayland-protocols-misc, wayland-protocols-wlr, wayland-scanner, wayland-server, wayland-sys
Checking packages consistency
Error: Unable to determine latest modification time for local package 'wayland-protocols-misc' due to: 'TypeError: Cannot read properties of null (reading 'target')'
Error: Unable to determine latest modification time for local package 'wayland-protocols-wlr' due to: 'TypeError: Cannot read properties of null (reading 'target')'
Error: Unable to determine latest modification time for local package 'wayland-cursor' due to: 'TypeError: Cannot read properties of null (reading 'target')'
Error: Unable to determine latest modification time for local package 'wayland-egl' due to: 'TypeError: Cannot read properties of null (reading 'target')'
Error: Unable to determine latest modification time for local package 'wayland-backend' due to: 'TypeError: Cannot read properties of null (reading 'target')'
Error: Unable to determine latest modification time for local package 'wayland-protocols-plasma' due to: 'TypeError: Cannot read properties of null (reading 'target')'
Error: Unable to determine latest modification time for local package 'wayland-sys' due to: 'TypeError: Cannot read properties of null (reading 'target')'
Error: Unable to determine latest modification time for local package 'wayland-server' due to: 'TypeError: Cannot read properties of null (reading 'target')'
Error: Unable to determine latest modification time for local package 'wayland-scanner' due to: 'TypeError: Cannot read properties of null (reading 'target')'
Error: Unable to determine latest modification time for local package 'wayland-protocols' due to: 'TypeError: Cannot read properties of null (reading 'target')'
Error: Unable to determine latest modification time for local package 'wayland-client' due to: 'TypeError: Cannot read properties of null (reading 'target')'
Error: Error: 11 packages consistency error(s) found

Huh. That worked testing on my fork...

Some things might be different when running on a PR from a fork, but that particular message doesn't make much sense.

@ids1024
Copy link
Member Author

ids1024 commented Apr 18, 2024

We've used such a thing in e.g. glutin/winit and I'd say that it's very tricky. Especially when your crates depend on each self, etc. It's also very easy to mess things up.

Hopefully the checks done by something like katyo/publish-crates@v1 make it harder to mess things up, relative to doing it manually. But yeah, there are plenty of things that can go wrong when dealing with a bunch of crates like this.

https://github.com/crate-ci/cargo-release also seems worth looking at, though it seems it still has some limitations. I haven't figured out exactly how that works.

@elinorbgr
Copy link
Member

LGTM overall, thanks a lot for taking the time to setup that!

Regarding the crates that are missing an owner, I'll take care of that before we merge this.

Regarding the CI failure, given the error messages mention reading 'target', I wonder if maybe we need to build the crates before invoking the publish-crates action, maybe?

ids1024 added 2 commits May 3, 2024 11:43
The `katyo/publish-crates` action requires that all crates in a
repository depend on the latest version of any other crate in the repo.

This seems reasonable, and is probably a best practice.
This uses `katyo/publish-crates@v1`, which checks which crates have had
their versions bumped, and releases them on crates.io. While making sure
to publish dependencies first before the crate that depends on them.
This has apparently worked well in drm-rs.

Since subcrates are versioned differently, when we discussed this
earlier date-based tags names seemed best. So as @elinorbgr suggested,
this goes with `release-YYYY-MM-DD` as the tagging scheme. (The CI job
checks for tags prefixed with `release-`). I can't really think of
anything better than that.

So to release, we should be able to simply update the versions in
`Cargo.toml` files, update the changelog files, and tag a release.

The `publish-crates` action requires that all subcrates depends on the
latest version of any other crate in the repository. And it requires
that all crates with changes have a bumped version so they can be
released. This seems a bit annoying if we wanted to just release a
change to `wayland-backend`, for instance, but should help avoid
mistakes. Manually dealing with the releases for all the crates here,
without any automated checks, is error-prone.

The action here does a `dry-run` for all CI runs (and uses
`ignore-unpublished-changes` to not error if crates have changes without
already having their versions bumped). But disables `dry-run` when a
tag is pushed with the right format.

A `CRATES_TOKEN` secret will need to be added to the repository for this
to work.
@ids1024
Copy link
Member Author

ids1024 commented May 3, 2024

Okay, it's not failing now, after changing katyo/publish-crates@v1 to katyo/publish-crates@v2. (I had copied from drm-rs, which used v1, but there's a v2 now.)

So hopefully this will work.

The CI log lists some errors, but doesn't fail:

Error: It seems package 'wayland-protocols-wlr' modified since '0.2.0' so new version should be published
Error: It seems package 'wayland-protocols-misc' modified since '0.2.0' so new version should be published
Error: It seems package 'wayland-cursor' modified since '0.31.1' so new version should be published
Error: It seems package 'wayland-backend' modified since '0.3.3' so new version should be published
Error: It seems package 'wayland-egl' modified since '0.32.0' so new version should be published
Error: It seems package 'wayland-protocols-plasma' modified since '0.2.0' so new version should be published
Error: It seems package 'wayland-protocols' modified since '0.31.2' so new version should be published
Error: It seems package 'wayland-sys' modified since '0.31.1' so new version should be published
Error: It seems package 'wayland-client' modified since '0.31.2' so new version should be published
Error: It seems package 'wayland-server' modified since '0.31.1' so new version should be published
Error: It seems package 'wayland-scanner' modified since '0.31.1' so new version should be published

So to release, we'd want to bump all the crates it complains about, make sure it prints no other messages, then tag a release.

@ids1024
Copy link
Member Author

ids1024 commented May 3, 2024

Okay, I understand how cargo-release works a bit better now.

So as an alternative to this CI job, a command like cargo release patch will (do a dry run of):

  • Bump the patch versions for subcrates
  • Create tags, with the same naming convention as the tags wayland-rs already uses
  • Publish each crate
  • Push the branch and tags

So that actually automates more of the process than katyo/publish-crates@v2. Though if we want to have a changelog for each crate, with dates for releases, I don't think it really makes things any simpler. At least unless cargo-release adds support for changelogs (crate-ci/cargo-release#231). So I think either would work.

@ids1024 ids1024 merged commit cb02eee into Smithay:master May 30, 2024
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants