Want cargo publish --idempotent

[ijackson]

Problem

Sometimes a cargo publish run of multiple packages will fail. It is then necessary to restart it.

It would be nice if we could run the whole publication from the top, and simply skip crates that were already published. There are probably other uses.

Proposed Solution

cargo publish --idempotent

If the current version of the crate has already been published, skips the crate and exits with a success status.

(Ideally it would download the crate and check it's identical to what is proposed to publish now, but I think that crates.io does unpredictable-to-cargo mangling of the uploaded data which makes such a check infeasible.)

Notes

No response

[weihanglo]

Sometimes a cargo publish run of multiple packages will fail. It is then necessary to restart it.

Could you clarify the use case? Did you mean one cargo publish invocation to publish multiple packages? cargo publish doesn't support that at the moment. Workspace publish is a feature tracked here.

If the current version of the crate has already been published, skips the crate and exits with a success status.

There is a similar request #3662. I wonder if we should keep that and close this one.

[ijackson]

Could you clarify the use case? Did you mean one cargo publish invocation to publish multiple packages? cargo publish doesn't support that at the moment. Workspace publish is a feature tracked here.

Yes. Because cargo can't publish a workspace in one go, we all have add-on tools of varying sophistication to publish our multi-crate workspaces. The add-on tools iterate over the crates in a workspace, calling cargo publish for each one.

I think #1169 looks quite hard. It has been open for a very long time and although there's work on it, a good solution there will be a little while.

One of the ways that these add-on tools are suboptimal is that after a failure of any kind, the workspace can be left "half-published". Recovering from such a situation involves either complicated predictive logic in the tool, or manual tweaking of intermediate publication lists, etc.

So in the meantime, while #1169 is worked on, our add-on tools would be simpler and more reliable of cargo offered an idempotent publish operation.

And, I think any reasonable solution to #1169 needs to cope with failures halfway through, and for that situation to be recoverable. That would mean that to implement #1169, cargo needs an idempotent publish operation for its own use anyway. Ie, the underlying functionality in my request here is a necessary part of #1169.

But it is also separately useful, so it's worth exposing as an operation to external callers. The notion is fairly well-defined and, I think, not that hard to implement.

If the current version of the crate has already been published, skips the crate and exits with a success status.

There is a similar request #3662. I wonder if we should keep that and close this one.

#3662 requests early failure. I'm requesting treating "this one is already published" as a success. Similar underlying logic might be involved with both, but the overall semantics are rather different.

Failure on republish is precisely the opposite of what is needed for reliable publication of a whole workspace (whether as part of a first-class cargo feature for this, or as part of an add-on tool that invokes cargo). Since, after a failure, one ought to stop. Whereas when publication of a whole workspace, "you already did this one" should mean "carry on".

[epage]

So in the meantime, while #1169 is worked on, our add-on tools would be simpler and more reliable of cargo offered an idempotent publish operation.

As an author of such an add-on tool (cargo-release), I don't see how having this would make the tool much simpler or more reliable.

But I also feel automatic recovery is a tricky enough subject that I give the underlying steps to the user so they can decide how they want to recover, which includes a cargo release publish --unpublished command.

And, I think any reasonable solution to #1169 needs to cope with failures halfway through, and for that situation to be recoverable. That would mean that to implement #1169, cargo needs an idempotent publish operation for its own use anyway. Ie, the underlying functionality in my request here is a necessary part of #1169.

In terms of core functionality, I see workspace publishing as the next step and we can better survey where things are at and expand from there. Having workspace-publishing in will help us better evaluate how this mode should work rather than designing this most first and then hoping it remains coherent.

[epage]

Had some more conversations with people about this and a problem that came up is that if a version is already in the index, its unclear if its already published or the user forgot to bump the version. At least in this proposal, it requires opt-in.

[ijackson]

Had some more conversations with people about this and a problem that came up is that if a version is already in the index, its unclear if its already published or the user forgot to bump the version. At least in this proposal, it requires opt-in.

Indeed so. Absolutely, it has to be an opt-in for this reason.

[epage]

In discussing #15006, a thought that might be relevant here:

All three options involve magic in a way and a concern I have with that is the defaults for cargo publish: the default is unreversable and there is no confirmation prompt. If we could go back and change it, I would make --dry-run the default and have a confirmation prompt on --execute (like cargo release). In this state, erroring out on unexpected input is much safer than a "guess my intent flag" as users have to go out of there way to be able to review what will happen.