From f5f2c51e7094c633e344887d04cbaa5592f6b6f1 Mon Sep 17 00:00:00 2001 From: Philipp Krones Date: Sun, 17 Nov 2024 18:44:42 +0100 Subject: [PATCH] Update backport, release and sync documentation --- .../development/infrastructure/backport.md | 126 +++++++++++------ .../src/development/infrastructure/release.md | 128 +++++++++--------- book/src/development/infrastructure/sync.md | 41 ++---- 3 files changed, 162 insertions(+), 133 deletions(-) diff --git a/book/src/development/infrastructure/backport.md b/book/src/development/infrastructure/backport.md index 6920c4e46561..9526d8af1c9d 100644 --- a/book/src/development/infrastructure/backport.md +++ b/book/src/development/infrastructure/backport.md @@ -5,68 +5,108 @@ Backports in Clippy are rare and should be approved by the Clippy team. For example, a backport is done, if a crucial ICE was fixed or a lint is broken to a point, that it has to be disabled, before landing on stable. -Backports are done to the `beta` branch of Clippy. Backports to stable Clippy -releases basically don't exist, since this would require a Rust point release, -which is almost never justifiable for a Clippy fix. +> Note: If you think a PR should be backported you can label it with +> `beta-nominated`. This has to be done before the Thursday the week before the +> release. +## Filtering PRs to backport -## Backport the changes +First, find all labeled PRs using [this filter][beta-accepted-prs]. + +Next, look at each PR individually. There are a few things to check. Those need +some explanation and are quite subjective. Good judgement is required. + +1. **Is the fix worth a backport?** + + This is really subjective. An ICE fix usually is. Moving a lint to a _lower_ + group (from warn- to allow-by-default) usually as well. An FP fix usually not + (on its own). If a backport is done anyway, FP fixes might also be included. + If the PR has a lot of changes, backports must be considered more carefully. + +2. **Is the problem that was fixed by the PR already in `beta`?** + + It could be that the problem that was fixed by the PR hasn't made it to the + `beta` branch of the Rust repo yet. If that's the case, and the fix is + already synced to the Rust repo, the fix doesn't need to be backported, as it + will hit stable together with the commit that introduced the problem. If the + fix PR is not synced yet, the fix PR either needs to be "backported" to the + Rust `master` branch or to `beta` in the next backport cycle. + +3. **Make sure that the fix is on `master` before porting to `beta`** + + The fix must already be synced to the Rust `master` branch. Otherwise, the + next `beta` will be missing this fix again. If it is not yet in `master` it + should probably not be backported. If the backport is really important, do an + out-of-cycle sync first. However, the out-of-cycle sync should be small, + because the changes in that sync will get right into `beta`, without being + tested in `nightly` first. + +[beta-accepted-prs]: https://github.com/rust-lang/rust-clippy/issues?q=label%3Abeta-nominated + +## Preparation + +> Note: All commands in this chapter will be run in the Rust clone. + +Follow the instructions in [defining remotes] to define the `clippy-upstream` +remote in the Rust repository. -Backports are done on the beta branch of the Clippy repository. +After that, fetch the remote with ```bash -# Assuming the current directory corresponds to the Clippy repository -$ git checkout beta -$ git checkout -b backport -$ git cherry-pick # `` is the commit hash of the commit(s), that should be backported -$ git push origin backport +git fetch clippy-upstream master ``` -Now you should test that the backport passes all the tests in the Rust -repository. You can do this with: +Then, switch to the `beta` branch: ```bash -# Assuming the current directory corresponds to the Rust repository -$ git checkout beta -# Make sure to change `your-github-name` to your github name in the following command -$ git subtree pull -p src/tools/clippy https://github.com//rust-clippy backport -$ ./x.py test src/tools/clippy +git switch beta +git fetch upstream +git reset --hard upstream/beta ``` -Should the test fail, you can fix Clippy directly in the Rust repository. This -has to be first applied to the Clippy beta branch and then again synced to the -Rust repository, though. The easiest way to do this is: +[defining remotes]: release.md#defining-remotes + +## Backport the changes + +When a PR is merged with the GitHub merge queue, the PR is closed with the message + +> \ (#\) + +This commit needs to be backported. To do that, find the `` of that commit +and run the following command in the clone of the **Rust repository**: ```bash -# In the Rust repository -$ git diff --patch --relative=src/tools/clippy > clippy.patch -# In the Clippy repository -$ git apply /path/to/clippy.patch -$ git add -u -$ git commit -m "Fix rustup fallout" -$ git push origin backport +git cherry-pick -m 1 `` ``` -After this, you can open a PR to the `beta` branch of the Clippy repository. +Do this for all PRs that should be backported. +## Open PR in the Rust repository -## Update Clippy in the Rust Repository +Next, open the PR for the backport. Make sure, the PR is opened towards the +`beta` branch and not the `master` branch. The PR description should look like +this: -This step must be done, **after** the PR of the previous step was merged. +``` +[beta] Clippy backports -After the backport landed in the Clippy repository, the branch has to be synced -back to the beta branch of the Rust repository. +r? @Mark-Simulacrum -```bash -# Assuming the current directory corresponds to the Rust repository -$ git checkout beta -$ git checkout -b clippy_backport -$ git subtree pull -p src/tools/clippy https://github.com/rust-lang/rust-clippy beta -$ git push origin clippy_backport +Backports: +- +- ... + + ``` -Make sure to test the backport in the Rust repository before opening a PR. This -is done with `./x.py test src/tools/clippy`. If that passes all tests, open a PR -to the `beta` branch of the Rust repository. In this PR you should tag the -Clippy team member, that agreed to the backport or the `@rust-lang/clippy` team. -Make sure to add `[beta]` to the title of the PR. +Mark is from the release team and they ultimately have to merge the PR before +branching a new `beta` version. Tag them to take care of the backport. Next, +list all the backports and give a short summary what's backported and why it is +worth backporting this. + +## Relabel backported PRs + +When a PR is backported to Rust `beta`, label the PR with `beta-accepted`. This +will then get picked up when [writing the changelog]. + +[writing the changelog]: changelog_update.md#31-include-beta-accepted-prs diff --git a/book/src/development/infrastructure/release.md b/book/src/development/infrastructure/release.md index 98fabf8e89ae..20b870eb69a8 100644 --- a/book/src/development/infrastructure/release.md +++ b/book/src/development/infrastructure/release.md @@ -7,112 +7,114 @@ Clippy is released together with stable Rust releases. The dates for these releases can be found at the [Rust Forge]. This document explains the necessary steps to create a Clippy release. -1. [Remerge the `beta` branch](#remerge-the-beta-branch) -2. [Update the `beta` branch](#update-the-beta-branch) -3. [Find the Clippy commit](#find-the-clippy-commit) -4. [Tag the stable commit](#tag-the-stable-commit) -5. [Update `CHANGELOG.md`](#update-changelogmd) - -> _NOTE:_ This document is for stable Rust releases, not for point releases. For -> point releases, step 1. and 2. should be enough. +1. [Defining Remotes](#defining-remotes) +1. [Bump Version](#bump-version) +1. [Find the Clippy commit](#find-the-clippy-commit) +1. [Update the `beta` branch](#update-the-beta-branch) +1. [Update the `stable` branch](#update-the-stable-branch) +1. [Tag the stable commit](#tag-the-stable-commit) +1. [Update `CHANGELOG.md`](#update-changelogmd) [Rust Forge]: https://forge.rust-lang.org/ -## Remerge the `beta` branch +## Defining Remotes + +You may want to define the `upstream` remote of the Clippy project to simplify +the following steps. However, this is optional and you can replace `upstream` +with the full URL instead. + +```bash +git remote add upstream git@github.com:rust-lang/rust-clippy +``` -This step is only necessary, if since the last release something was backported -to the beta Rust release. The remerge is then necessary, to make sure that the -Clippy commit, that was used by the now stable Rust release, persists in the -tree of the Clippy repository. +## Bump Version -To find out if this step is necessary run +When a release needs to be done, `cargo test` will fail, if the versions in the +`Cargo.toml` are not correct. During that sync, the versions need to be bumped. +This is done by running: ```bash -# Assumes that the local master branch of rust-lang/rust-clippy is up-to-date -$ git fetch upstream -$ git branch master --contains upstream/beta +cargo dev release bump_version ``` -If this command outputs `master`, this step is **not** necessary. +This will increase the version number of each relevant `Cargo.toml` file. After +that, just commit the updated files with: ```bash -# Assuming `HEAD` is the current `master` branch of rust-lang/rust-clippy -$ git checkout -b backport_remerge -$ git merge upstream/beta -$ git diff # This diff has to be empty, otherwise something with the remerge failed -$ git push origin backport_remerge # This can be pushed to your fork +git commit -m "Bump Clippy version -> 0.1.XY" **/*Cargo.toml ``` -After this, open a PR to the master branch. In this PR, the commit hash of the -`HEAD` of the `beta` branch must exist. In addition to that, no files should be -changed by this PR. +`XY` should be exchanged with the corresponding version -## Update the `beta` branch +## Find the Clippy commit -This step must be done **after** the PR of the previous step was merged. +For both updating the `beta` and the `stable` branch, the first step is to find +the Clippy commit of the last Clippy sync done in the respective Rust branch. -First, the Clippy commit of the `beta` branch of the Rust repository has to be -determined. +Running the following commands _in the Rust repo_ will get the commit for the +specified ``: ```bash -# Assuming the current directory corresponds to the Rust repository -$ git fetch upstream -$ git checkout upstream/beta -$ BETA_SHA=$(git log --oneline -- src/tools/clippy/ | grep -o "Merge commit '[a-f0-9]*' into .*" | head -1 | sed -e "s/Merge commit '\([a-f0-9]*\)' into .*/\1/g") +git switch +SHA=$(git log --oneline -- src/tools/clippy/ | grep -o "Merge commit '[a-f0-9]*' into .*" | head -1 | sed -e "s/Merge commit '\([a-f0-9]*\)' into .*/\1/g") ``` -After finding the Clippy commit, the `beta` branch in the Clippy repository can -be updated. +Where `` is one of `stable`, `beta`, or `master`. + +## Update the `beta` branch + +After getting the commit of the `beta` branch, the `beta` branch in the Clippy +repository can be updated. ```bash -# Assuming the current directory corresponds to the Clippy repository -$ git checkout beta -$ git reset --hard $BETA_SHA -$ git push upstream beta +git checkout beta +git reset --hard $SHA +git push upstream beta ``` -## Find the Clippy commit +## Update the `stable` branch -The first step is to tag the Clippy commit, that is included in the stable Rust -release. This commit can be found in the Rust repository. +After getting the commit of the `stable` branch, the `stable` branch in the +Clippy repository can be updated. ```bash -# Assuming the current directory corresponds to the Rust repository -$ git fetch upstream # `upstream` is the `rust-lang/rust` remote -$ git checkout 1.XX.0 # XX should be exchanged with the corresponding version -$ SHA=$(git log --oneline -- src/tools/clippy/ | grep -o "Merge commit '[a-f0-9]*' into .*" | head -1 | sed -e "s/Merge commit '\([a-f0-9]*\)' into .*/\1/g") +git checkout stable +git reset --hard $SHA +git push upstream stable ``` -## Tag the stable commit +## Tag the `stable` commit -After finding the Clippy commit, it can be tagged with the release number. +After updating the `stable` branch, tag the HEAD commit and push it to the +Clippy repo. + +> Note: Only push the tag once the Deploy GitHub action of the `beta` branch is +> finished. Otherwise the deploy for the tag might fail. ```bash -# Assuming the current directory corresponds to the Clippy repository -$ git checkout $SHA -$ git tag rust-1.XX.0 # XX should be exchanged with the corresponding version -$ git push upstream rust-1.XX.0 # `upstream` is the `rust-lang/rust-clippy` remote +git tag rust-1.XX.0 # XX should be exchanged with the corresponding version +git push upstream rust-1.XX.0 # `upstream` is the `rust-lang/rust-clippy` remote ``` After this, the release should be available on the Clippy [release page]. [release page]: https://github.com/rust-lang/rust-clippy/releases -## Update the `stable` branch +## Publish `clippy_utils` + +The `clippy_utils` crate is published to `crates.io` without any stability +guarantees. To do this, after the [sync] and the release is done, switch back to +the `upstream/master` branch and publish `clippy_utils`: -At this step you should have already checked out the commit of the `rust-1.XX.0` -tag. Updating the stable branch from here is as easy as: +> Note: The Rustup PR bumping the nightly and Clippy version **must** be merged +> before doing this. ```bash -# Assuming the current directory corresponds to the Clippy repository and the -# commit of the just created rust-1.XX.0 tag is checked out. -$ git push upstream rust-1.XX.0:stable # `upstream` is the `rust-lang/rust-clippy` remote +git switch master && git pull upstream master +cargo publish --manifest-path clippy_utils/Cargo.toml ``` -> _NOTE:_ Usually there are no stable backports for Clippy, so this update -> should be possible without force pushing or anything like this. If there -> should have happened a stable backport, make sure to re-merge those changes -> just as with the `beta` branch. +[sync]: sync.md ## Update `CHANGELOG.md` diff --git a/book/src/development/infrastructure/sync.md b/book/src/development/infrastructure/sync.md index e1fe92f95250..da1ad586607f 100644 --- a/book/src/development/infrastructure/sync.md +++ b/book/src/development/infrastructure/sync.md @@ -21,6 +21,8 @@ to beta. For reference, the first sync following this cadence was performed the This process is described in detail in the following sections. For general information about `subtree`s in the Rust repository see [the rustc-dev-guide][subtree]. +[subtree]: https://rustc-dev-guide.rust-lang.org/external-repos.html#external-dependencies-subtree + ## Patching git-subtree to work with big repos Currently, there's a bug in `git-subtree` that prevents it from working properly @@ -50,23 +52,11 @@ sudo chown --reference=/usr/lib/git-core/git-subtree~ /usr/lib/git-core/git-subt > `bash` instead. You can do this by editing the first line of the `git-subtree` > script and changing `sh` to `bash`. -## Defining remotes - -You may want to define remotes, so you don't have to type out the remote -addresses on every sync. You can do this with the following commands (these -commands still have to be run inside the `rust` directory): - -```bash -# Set clippy-upstream remote for pulls -$ git remote add clippy-upstream https://github.com/rust-lang/rust-clippy -# Make sure to not push to the upstream repo -$ git remote set-url --push clippy-upstream DISABLED -# Set a local remote -$ git remote add clippy-local /path/to/rust-clippy -``` +> Note: The following sections assume that you have set up remotes following the +> instructions in [defining remotes]. -> Note: The following sections assume that you have set those remotes with the -> above remote names. +[gitgitgadget-pr]: https://github.com/gitgitgadget/git/pull/493 +[defining remotes]: release.md#defining-remotes ## Performing the sync from [`rust-lang/rust`] to Clippy @@ -78,9 +68,9 @@ to be run inside the `rust` directory): `rustup check`. 3. Sync the changes to the rust-copy of Clippy to your Clippy fork: ```bash - # Be sure to either use a net-new branch, e.g. `sync-from-rust`, or delete the branch beforehand + # Be sure to either use a net-new branch, e.g. `rustup`, or delete the branch beforehand # because changes cannot be fast forwarded and you have to run this command again. - git subtree push -P src/tools/clippy clippy-local sync-from-rust + git subtree push -P src/tools/clippy clippy-local rustup ``` > _Note:_ Most of the time you have to create a merge commit in the @@ -88,21 +78,22 @@ to be run inside the `rust` directory): > rust-copy of Clippy): ```bash git fetch upstream # assuming upstream is the rust-lang/rust remote - git checkout sync-from-rust + git switch rustup git merge upstream/master --no-ff ``` > Note: This is one of the few instances where a merge commit is allowed in > a PR. -4. Bump the nightly version in the Clippy repository by changing the date in the - rust-toolchain file to the current date and committing it with the message: +4. Bump the nightly version in the Clippy repository by running these commands: ```bash - git commit -m "Bump nightly version -> YYYY-MM-DD" + cargo dev sync update_nightly + git commit -m "Bump nightly version -> YYYY-MM-DD" rust-toolchain clippy_utils/README.md ``` 5. Open a PR to `rust-lang/rust-clippy` and wait for it to get merged (to accelerate the process ping the `@rust-lang/clippy` team in your PR and/or ask them in the [Zulip] stream.) [Zulip]: https://rust-lang.zulipchat.com/#narrow/stream/clippy +[`rust-lang/rust`]: https://github.com/rust-lang/rust ## Performing the sync from Clippy to [`rust-lang/rust`] @@ -111,11 +102,7 @@ All the following commands have to be run inside the `rust` directory. 1. Make sure you have checked out the latest `master` of `rust-lang/rust`. 2. Sync the `rust-lang/rust-clippy` master to the rust-copy of Clippy: ```bash - git checkout -b sync-from-clippy + git switch -c clippy-subtree-update git subtree pull -P src/tools/clippy clippy-upstream master ``` 3. Open a PR to [`rust-lang/rust`] - -[gitgitgadget-pr]: https://github.com/gitgitgadget/git/pull/493 -[subtree]: https://rustc-dev-guide.rust-lang.org/external-repos.html#external-dependencies-subtree -[`rust-lang/rust`]: https://github.com/rust-lang/rust