Skip to content

Commit

Permalink
docs: Generalize Cargo.lock guidance
Browse files Browse the repository at this point in the history
Before, we were fairly prescriptive of when to check-in the `Cargo.lock`
file.   The guidance has been updated to instead explain the trade offs
and to mostly leave it in users hands.  As a starting point, we do say
to check it in for those that want an "easy answer".
  • Loading branch information
epage committed Jul 19, 2023
1 parent 82a78e3 commit 6af7497
Show file tree
Hide file tree
Showing 3 changed files with 96 additions and 33 deletions.
60 changes: 35 additions & 25 deletions src/doc/src/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -102,33 +102,43 @@ issue][cargo-issues].

[cargo-issues]: https://github.com/rust-lang/cargo/issues

### Why do binaries have `Cargo.lock` in version control, but not libraries?
### Why have `Cargo.lock` in version control?

Whether you do is dependent on the needs of your package.

The purpose of a `Cargo.lock` lockfile is to describe the state of the world at
the time of a successful build. Cargo uses the lockfile to provide
deterministic builds on different times and different systems, by ensuring that
the exact same dependencies and versions are used as when the `Cargo.lock` file
was originally generated.

This property is most desirable from applications and packages which are at the
very end of the dependency chain (binaries). As a result, it is recommended that
all binaries check in their `Cargo.lock`.

For libraries the situation is somewhat different. A library is not only used by
the library developers, but also any downstream consumers of the library. Users
dependent on the library will not inspect the library’s `Cargo.lock` (even if it
exists). This is precisely because a library should **not** be deterministically
recompiled for all users of the library.

If a library ends up being used transitively by several dependencies, it’s
likely that just a single copy of the library is desired (based on semver
compatibility). If Cargo used all of the dependencies' `Cargo.lock` files,
then multiple copies of the library could be used, and perhaps even a version
conflict.

In other words, libraries specify SemVer requirements for their dependencies but
cannot see the full picture. Only end products like binaries have a full
picture to decide what versions of dependencies should be used.
the time of a successful build.
Cargo uses the lockfile to provide deterministic builds at different times and
on different systems,
by ensuring that the exact same dependencies and versions are used as when the
`Cargo.lock` file was originally generated.

Deterministic builds help with
- Running `git bisect` to find the root cause of a bug
- Ensuring CI only fails due to new commits and not external factors
- Reducing confusion when contributors see different behavior as compared to
other contributors or CI

Having this snapshot of dependencies can also help when projects need to be
verified against consistent versions of dependencies, like when
- Verifying a minimum-supported Rust version that is less than the what latest
version of a dependency supports
- Verifying human readable output which won't have compatibility guarantees
(e.g. snapshot testing error messages to ensure they are "understandable", a
metric too fuzzy to automate)

However, this determinism can give a false sense of security because
`Cargo.lock` does not affect the consumers of your package, only `Cargo.toml` does that.
For example:
- [`cargo install`] will select the latest dependencies unless `--locked` is
passed in.
- New dependencies, like those added with [`cargo add`], will be locked to the latest version

For strategies to verify newer dependencies,
see [Verifying Latest Dependencies](guide/continuous-integration.md#verifying-latest-dependencies).

[`cargo add`]: commands/cargo-add.md
[`cargo install`]: commands/cargo-install.md

### Can libraries use `*` as a version for their dependencies?

Expand Down
13 changes: 5 additions & 8 deletions src/doc/src/guide/cargo-toml-vs-cargo-lock.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,14 +8,11 @@ about them, here’s a summary:
* `Cargo.lock` contains exact information about your dependencies. It is
maintained by Cargo and should not be manually edited.

If you’re building a non-end product, such as a rust library that other rust
[packages][def-package] will depend on, put `Cargo.lock` in your
`.gitignore`. If you’re building an end product, which are executable like
command-line tool or an application, or a system library with crate-type of
`staticlib` or `cdylib`, check `Cargo.lock` into `git`. If you're curious
about why that is, see
["Why do binaries have `Cargo.lock` in version control, but not libraries?" in the
FAQ](../faq.md#why-do-binaries-have-cargolock-in-version-control-but-not-libraries).
When in doubt, check `Cargo.lock` into git.
For a better understanding of why and what the alternatives might be, see
[“Why have Cargo.lock in version control?” in the FAQ](../faq.md#why-have-cargolock-in-version-control).
We recommend pairing this with
[Verifying Latest Dependencies](continuous-integration.md#verifying-latest-dependencies)

Let’s dig in a little bit more.

Expand Down
56 changes: 56 additions & 0 deletions src/doc/src/guide/continuous-integration.md
Original file line number Diff line number Diff line change
Expand Up @@ -104,3 +104,59 @@ This will test and build documentation on the stable channel and nightly
channel, but any breakage in nightly will not fail your overall build. Please
see the [builds.sr.ht documentation](https://man.sr.ht/builds.sr.ht/) for more
information.

### Verifying Latest Dependencies

When [specifying dependencies](../reference/specifying-dependencies.md) in
`Cargo.toml`, they generally match a range of versions.
Exhaustively testing all version combination would be unwieldy.
Verifying the latest versions would at least test for users who run [`cargo
add`] or [`cargo install`].

When testing the latest versions some considerations are:
- Minimizing external factors affecting local development or CI
- Rate of new dependencies being published
- Level of risk a project is willing to accept
- CI costs, including indirect costs like if a CI service has a maximum for
parallel runners, causing new jobs to be serialized when at the maxium.

Some potential solutions include:
- [Not checking in the `Cargo.lock`](../faq.md#why-have-cargolock-in-version-control)
- Depending on PR velocity, many versions may go untested
- This comes at the cost of determinism
- Have a CI job verify the latest dependencies but mark it to "continue on failure"
- Depending on the CI service, failures might not be obvious
- Depending on PR velocity, may use more resources than necessary
- Have a scheduled CI job to verify latest dependencies
- A hosted CI service may disable scheduled jobs for repositories that
haven't been touched in a while, affecting passively maintained packages
- Depending on the CI service, notifications might not be routed to people
who can act on the failure
- If not balanced with dependency publish rate, may not test enough versions
or may do redundant testing
- Regularly update dependencies through PRs, like with [Dependabot] or [RenovateBot]
- Can isolate dependencies to their own PR or roll them up into a single PR
- Only uses the resources necessary
- Can configure the frequency to balance CI resources and coverage of dependency versions

An example CI job to verify latest dependencies, using Github Actions:
```yaml
jobs:
latest_deps:
name: Latest Dependencies
runs-on: ubuntu-latest
continue-on-error: true
steps:
- uses: actions/checkout@v3
- run: rustup update stable && rustup default stable
- run: cargo update --verbose
- run: cargo build --verbose
- run: cargo test --verbose
```
For projects with higher risks of per-platform or per-Rust version failures,
more combinations may want to be tested.

[`cargo add`]: ../commands/cargo-add.md
[`cargo install`]: ../commands/cargo-install.md
[Dependabot]: https://docs.github.com/en/code-security/dependabot/working-with-dependabot
[RenovateBot]: https://renovatebot.com/

0 comments on commit 6af7497

Please sign in to comment.