-
Notifications
You must be signed in to change notification settings - Fork 2.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Cargo update -p <localcrate> in a workspace has inconsistent behaviors #12599
Comments
So exhibit 2 involves running I ran The sad case has a long list of
that when only 1 exists in the happy case. |
It looks like our tracking of what should be ignored is order dependent, depending on whether the package with the changed version is seen first or not. |
This address one of the problems mentioned in rust-lang#12599
@roblabla your Exhibit One reproduction cases have |
Correction, I'm assuming that difference is intended to showcase the difference in behavior. We have a check where we only avoid a path dependency if its changed. That "if" i what is causing the problem here. |
Yeah, the trigger.sh in the happy case changes the version in the manifest, while in the sad case it doesn't, which causes the unexpected difference in behavior. |
@Eh2406 for Exhibit One ( |
@roblabla seems like these problems are very different, even in reproduction steps (one being order dependent while the other one isn't). Please in the future create separate Issues so we can more easily track the individual conversations and progress. |
Alright. Reason they're together is that I actually discovered exhibit 2 while trying to reproduce exhibit 1 😅 |
This address one of the problems mentioned in rust-lang#12599
This address one of the problems mentioned in rust-lang#12599
This addresses the ordering issue of for one of the problems from rust-lang#12599. The intent behind the `path_pkg` check is to make sure we update workspace members in the lockfile if their version number changed. In this case, we don't need to recursively walk, because the change doesn't affect dependencies. However, we also shouldn't *prevent* recursive walks which is what we are doing today, both for packages marked to keep and for packages that have been "poisoned".
This addresses the ordering issue of for one of the problems from rust-lang#12599. The intent behind the `path_pkg` check is to make sure we update workspace members in the lockfile if their version number changed. In this case, we don't need to recursively walk, because the change doesn't affect dependencies. However, we also shouldn't *prevent* recursive walks which is what we are doing today, both for packages marked to keep and for packages that have been "poisoned". So we fix this by moving that call after all recursive adds are complete so as to not interfere with them.
@roblabla as an update, my change in #12602 was incorrect and it is now changed so you'll be seeing git updates in more cases. We still need to dig into if its possible to skip these git updates. They go through a bit of a different path to evaluate if we should update because we only allow one version in the dependency graph and we'll need to further evaluate what can and shouldn't be changed there. |
This addresses the ordering issue of for one of the problems from rust-lang#12599. The intent behind the `path_pkg` check is to make sure we update workspace members in the lockfile if their version number changed. In this case, we don't need to recursively walk, because the change doesn't affect dependencies. However, we also shouldn't *prevent* recursive walks which is what we are doing today, both for packages marked to keep and for packages that have been "poisoned". So we fix this by moving that call after all recursive adds are complete so as to not interfere with them.
This addresses the ordering issue of for one of the problems from rust-lang#12599. The intent behind the `path_pkg` check is to make sure we update workspace members in the lockfile if their version number changed. In this case, we don't need to recursively walk, because the change doesn't affect dependencies. However, we also shouldn't *prevent* recursive walks which is what we are doing today, both for packages marked to keep and for packages that have been "poisoned". So we fix this by moving that call after all recursive adds are complete so as to not interfere with them.
This addresses the ordering issue of for one of the problems from rust-lang#12599. The intent behind the `path_pkg` check is to make sure we update workspace members in the lockfile if their version number changed. In this case, we don't need to recursively walk, because the change doesn't affect dependencies. However, we also shouldn't *prevent* recursive walks which is what we are doing today, both for packages marked to keep and for packages that have been "poisoned". So we fix this by moving that call after all recursive adds are complete so as to not interfere with them.
fix(resolver): Make resolver behavior independent of package order This address one of the problems mentioned in #12599 The intent behind the `path_pkg` check is to make sure we update workspace members in the lockfile if their version number changed. In this case, we don't need to recursively walk, because the change doesn't affect dependencies. However, we also shouldn't *prevent* recursive walks which is what we are doing today, both for packages marked to keep and for packages that have been "poisoned". So we fix this by moving that call after all recursive adds are complete so as to not interfere with them. This should not affect `Cargo.lock` at rest, so no upgrade compatibility concerns. This just allows more packages to be considered available to change which can prevent unclear failures. The main case I can think of that this does something "undesirable" is when wanting to prevent another "bug" from manifesting: the updating of git dependencies when updating workspace members (#12599). I think I'm ok with that as that needs to be looked into separately.
Before, when running `cargo update <member>`, we'd not reuse the previous resolve result and when the resolver started walking into the dependencies, it would do a git fetch. Now, we won't even try to resolve the workspace members and so we won't look at those dependencies and do git fetch. This will make `cargo update <workspace-member>` match `cargo update --workspace`. I considered whether there were other ways of handling this but I figured aiming for consistency in approaches was the best way. We can investigate improving those approaches separately. There are other discrepancies in the different code paths (handling of patches, adding sources) but I'm deferring looking over those. Between this and rust-lang#12602, this should finnally resolve rust-lang#12599. Fixes rust-lang#12599
Before, when running `cargo update <member>`, we'd not reuse the previous resolve result and when the resolver started walking into the dependencies, it would do a git fetch. Now, we won't even try to resolve the workspace members and so we won't look at those dependencies and do git fetch. This will make `cargo update <workspace-member>` match `cargo update --workspace`. I considered whether there were other ways of handling this but I figured aiming for consistency in approaches was the best way. We can investigate improving those approaches separately. There are other discrepancies in the different code paths (handling of patches, adding sources) but I'm deferring looking over those. Between this and rust-lang#12602, this should finnally resolve rust-lang#12599. Fixes rust-lang#12599
Before, when running `cargo update <member>`, we'd not reuse the previous resolve result and when the resolver started walking into the dependencies, it would do a git fetch. Now, we won't even try to resolve the workspace members and so we won't look at those dependencies and do git fetch. This will make `cargo update <workspace-member>` match `cargo update --workspace`. I considered whether there were other ways of handling this but I figured aiming for consistency in approaches was the best way. We can investigate improving those approaches separately. There are other discrepancies in the different code paths (handling of patches, adding sources) but I'm deferring looking over those. Between this and rust-lang#12602, this should finnally resolve rust-lang#12599. Fixes rust-lang#12599
fix(resolver): Don't do git fetches when updating workspace members ### What does this PR try to resolve? Before, when running `cargo update <member>`, we'd not reuse the previous resolve result and when the resolver started walking into the dependencies, it would do a git fetch. Now, we won't even try to resolve the workspace members and so we won't look at those dependencies and do git fetch. This will make `cargo update <workspace-member>` match `cargo update --workspace`. Fixes #12599 Fixes #8821 ### How should we test and review this PR? I considered whether there were other ways of handling this but I figured aiming for consistency in approaches was the best way. We can investigate improving those approaches separately. There are other discrepancies in the different code paths (handling of patches, adding sources) but I'm deferring looking over those. I added a test to demonstrate the `--workspace` behavior to use as a base line to compare with. ### Additional information Between this and #12602, this should finally resolve #12599.
Cargo's handling of
cargo update -p <cratename>
is extremely inconsistent. Depending on how local crates are named or how up-to-date the Cargo.toml is compared to the Cargo.lock file, it will behave very differently with regards to git dependencies.I have created a repository explaining the problems I found @ https://github.com/roblabla/cargo-wtf-why-are-you-drunk/
In that repository, you'll find two examples of inconsistencies in the behavior of
cargo update -p
:In the first one, we show that cargo seems to handle git dependencies differently depending on whether it's got "work" to do on local crates or not. In particular, in my local monorepo, if I bump the version of a local crate in Cargo.toml, then
cargo update -p <crate>
will update the local lockfile to bump the versions ofcrate
and nothing else. However, if I don't bump the version and run the samecargo update -p <crate>
command, cargo will update the git dependencies.This discrepancy is especially frustrating, as it complicates our release process by making bumping the version more complicated.
In the second one, we show that whether or not cargo updates the git dependencies after bumping the version seem to depend on how your local crates in your workspace are named. This makes any call to
cargo update -p <crate>
inherently fragile, as what it does will depend on how your dependencies are named.Possible Solutions
IMO,
cargo update -p <crate>
should never update an unrelated git dependency, as it does not match what the user intended.Rust version:
rust 1.71.0 (8ede3aae2 2023-07-12)
The text was updated successfully, but these errors were encountered: