Tags with +suffix breaking vendoring?

[dpc]

We've tried to use a dependency in Cargo.lock that refers to a git repo with tag that contained +suffix, and we got a weird error:

pkgs-group-ci-deps> Caused by:
pkgs-group-ci-deps>   failed to load source for dependency `repo-core`
pkgs-group-ci-deps> Caused by:
pkgs-group-ci-deps>   Unable to update https://github.com/repo/repo-repo?tag=v0.3.0-alpha.0+suffix
pkgs-group-ci-deps> Caused by:
pkgs-group-ci-deps>   the source https://github.com/repo/repo-fedi?tag=v0.3.0-alpha.0+suffix requires a lock file to be present first before it can be
pkgs-group-ci-deps>   used against vendored source code
pkgs-group-ci-deps>   remove the source replacement configuration, generate a lock file, and then
pkgs-group-ci-deps>   restore the source replacement configuration to continue the build

cargo build and cargo vendor work locally just fine. After changing the tag name to not contains +suffix, the problem went away.

[ipetkov]

Hmm interesting, there's two parts of the code which come to mind:

1. Parsing the git dependency in Cargo.lock (though nothing about the code immediately jumps out as the culprit):

   crane/lib/vendorGitDeps.nix

   Lines 42 to 75 in 10484f8

   	parseGitUrl = p:
   	let
   	lockUrl = removePrefix "git+" p.source;
   	revSplit = split "#" (removePrefix "git+" lockUrl);
   	# uniquely identifies the repo in terms of what cargo can address via
   	# source replacement (e.g. the url along with any branch/tag/rev).
   	id = head revSplit;
   	# NB: this is distict from the `rev` query param which may show up
   	# if the dependency is explicitly listed with a `rev` value.
   	lockedRev = if 3 == length revSplit then last revSplit else
   	throw ''
   	Cargo.lock is missing a locked revision for ${p.name}@${p.version}.
   	you can try to resolve this by running `cargo update -p ${lockUrl}#${p.name}@${p.version}`
   	'';
   	querySplit = split "\\?" (head revSplit);
   	git = head querySplit;
   	queryParamSplit = filter
   	(qp: (isString qp) && any (p: hasPrefix p qp) knownGitParams)
   	(split "&" (last querySplit));
   	extractedParams = listToAttrs (map
   	(qp:
   	let
   	kvSplit = (split "=" qp);
   	in
   	nameValuePair (head kvSplit) (last kvSplit)
   	)
   	queryParamSplit
   	);
   	in
   	extractedParams // {
   	inherit git id lockedRev;
   	};

2. How we specify the tag when asking Nix to download it:

   crane/lib/vendorGitDeps.nix

   Lines 91 to 104 in 10484f8

   	if p ? tag then "refs/tags/${p.tag}"
   	else if p ? branch then "refs/heads/${p.branch}"
   	else null;
   	extractedPackages = downloadCargoPackageFromGit {
   	inherit (p) git;
   	inherit ref;
   	rev = p.lockedRev;
   	sha256 = outputHashes.${p.package.source} or (lib.warnIf
   	(outputHashes != { })
   	"No output hash provided for ${p.package.source}"
   	null
   	);
   	};

Do you have a reproduction I can test out further?

[avnik]

@ipetkov one more repro -- this time with branch name
https://github.com/avnik/ghaf-givc/tree/avnik/listeners%2Bvsock%2Btonic

[thecaralice]

This is a bug in cargo, not in crane. See rust-lang/cargo#14584.

[thecaralice]

Update: this issue has already been fixed in cargo, but the lockfile needs to be updated to version = 4, as mentioned in rust-lang/cargo#14584 (comment).

[ipetkov]

Thank you for the reproduction @avnik and thank you for the insight that this is a cargo bug @thecaralice !

Sounds like there isn't anything we can do on the crane side here. Anyone who hits this will need to manually update their Cargo.lock file to have version = 4 (until cargo is updated to migrate from v3 to v4 in the future)