cargoLlvmCov produces different result in Nix derivation than a regular invocation

[phip1611]

When I execute cargo llvm-cov directly, results seem to be fine. When I execute it in a Nix derivation using crane.cargoLlvmCov, it seems like it takes all dependencies into account as well. However, this is not something that you want. The difference is significant:

Without Nix: lines......: 65.5% (16880 of 25762 lines)
Within Nix: lines......: 22.4% (38873 of 173539 lines)

I suppose that in the regular workflow, llvm-cov filters out all dependencies (by some regex into the target folder), while in the Nix derivation the target folder is not within the same location as the src folder, hence, some regex probably doesn't match anymore.

I tried to workaround this with the --ignore-filenames-regex parameter of cargo-llvm-cov, but without success. I can't find the location where the build artifacts are stored. It is neither in $CARGO_TARGET_DIR nor in $CARGO_BUILD_TARGET_DIR

[ipetkov]

I'm not entirely sure why that happens, maybe it's the default --output-path $out argument that we set?

cc @figsoda in case you have more context around this?

[figsoda]

I'm not sure what could be causing this, could it have something to do with cargoArtifacts?

[dpc]

FWIW. In our CI we're doing:

  # Build only deps, but with llvm-cov so `workspaceCov` can reuse them cached
  workspaceDepsCov = craneLib.buildDepsOnly (commonArgsDepsOnly // {
    pnameSuffix = "-lcov-deps";
    version = "0.0.1";
    buildPhaseCargoCommand = "cargo llvm-cov --locked --workspace --profile $CARGO_PROFILE --no-report";
    cargoBuildCommand = "dontuse";
    cargoCheckCommand = "dontuse";
    nativeBuildInputs = commonArgs.nativeBuildInputs ++ [ cargo-llvm-cov ];
    doCheck = false;
  });

  workspaceCov = craneLib.buildPackage (commonArgs // {
    pnameSuffix = "-lcov";
    version = "0.0.1";
    cargoArtifacts = workspaceDepsCov;
    buildPhaseCargoCommand = "mkdir -p $out ; env RUST_LOG=info,timing=debug cargo llvm-cov --locked --workspace --profile $CARGO_PROFILE --lcov --all-targets --tests --output-path $out/lcov.info --  --test-threads=$(($(nproc) * 2))";
    installPhaseCommand = "true";
    nativeBuildInputs = commonArgs.nativeBuildInputs ++ [ cargo-llvm-cov ];
    doCheck = false;
  });

and it works fine, AFAICT, a 60% coverage.

[phip1611]

and it works fine, AFAICT, a 60% coverage.

With and also without Nix?

[dpc]

We never run it locally, AFAICT. So with, in our CI. But by calling cargo llvm-cov directly, no with crane.cargoLlvmCov