Support native StoreKey in FilesystemStore

[KGrewal1]

Description

Please include a summary of the changes and the related issue. Please also
include relevant motivation and context.

Fixes # 1108

/claim #1108

Design Document

https://hackmd.io/@HPxqrI77Q02izyGstTHbdg/ByS50OhGJx

Type of change

Please delete options that aren't relevant.

• Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

Passes all of current test suite

Checklist

• [] Updated documentation if needed
• Tests added/amended
• bazel test //... passes locally
• PR is contained in a single commit, using git amend see some docs

---

This change is 

[CLAassistant]

All committers have signed the CLA.

[algora-pbc]

💵 To receive payouts, sign up on Algora, link your Github account and connect with Stripe.

[KGrewal1]

Unsure what is happening with the local runner on CI, seems to be a Tekton issue

[MarcusSorealheis]

BTW, with the exception of the missing performance data, your design document is basically done. Thanks for the hardwork there. Now we just need to cover all the important areas of the code, in addition to including the perf data in the document.

[KGrewal1]

As of PR #1536

cargo t -p nativelink-store                                                                                                                                                                                                                                                                                                 (base) 
warning: /home/kirpal/Documents/rust/nativelink/Cargo.toml: unused manifest key: workspace.cargo-features-manager
   Compiling serde_json v1.0.133
   Compiling h2 v0.4.7
   Compiling aws-smithy-http v0.60.11
   Compiling h2 v0.3.26
   Compiling aws-credential-types v1.2.1
   Compiling cbor-diag v0.1.12
   Compiling fred v10.0.1
   Compiling nativelink-config v0.5.3 (/home/kirpal/Documents/rust/nativelink/nativelink-config)
   Compiling opentelemetry v0.27.1
   Compiling aws-types v1.3.3
error[E0432]: unresolved import `thiserror`
  --> /home/kirpal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/opentelemetry-0.27.1/src/propagation/mod.rs:23:5
   |
23 | use thiserror::Error;
   |     ^^^^^^^^^ use of undeclared crate or module `thiserror`

   Compiling aws-sigv4 v1.2.6
   Compiling aws-smithy-checksums v0.60.13
error: cannot find attribute `error` in this scope
  --> /home/kirpal/.cargo/registry/src/index.crates.io-6f17d22bba15001f/opentelemetry-0.27.1/src/propagation/mod.rs:67:3
   |
67 | #[error("Cannot {} from {}, {}", ops, message, propagator_name)]
   |   ^^^^^

For more information about this error, try `rustc --explain E0432`.
error: could not compile `opentelemetry` (lib) due to 2 previous errors
warning: build failed, waiting for other jobs to finish...

running on the previous commit (PR #1535 ) works so assume this broke with dependency upgrade

[KGrewal1]

Have a working (locally at least) version using the specified rename function to move old cache files from their previous location to under DigestFolder - specified function was chosen instead of async wrapper as felt it was choice of least suprise (would be odd if a rename function was specified but that part of the code used a different function)

[aaronmondal]

assume this broke with dependency upgrade

I can reproduce with cargo. Working on finding a fix. In the meantime, the Bazel test still works.

bazel test ...

[KGrewal1]

assume this broke with dependency upgrade

I can reproduce with cargo. Working on finding a fix. In the meantime, the Bazel test still works.

bazel test ...

Thanks - managed to get a working version of the rename with no deadlock now (the advantage of cargo test was so I could have had more fine grained control of which tests to run / no-capture to see debug output, compared to bazel test //...)

[allada]

Thanks!

Reviewed 1 of 1 files at r13, all commit messages.
Reviewable status: 2 of 2 LGTMs obtained, and all files reviewed, and pending CI: Bazel Dev / macos-13, and 1 discussions need to be resolved

---

a discussion (no related file):
@aaronmondal, can you please double check this.

[KGrewal1]

Reviewable status: 2 of 2 LGTMs obtained, and all files reviewed, and 1 discussions need to be resolved

---

nativelink-store/src/filesystem_store.rs line 538 at r7 (raw file):

Previously, KGrewal1 (Kirpal Grewal) wrote…

I have removed this as the tests were working locally but failing in CI - corresponding test was oldest_entry_evicted_with_access_times_loaded_from_disk() that ensured that on startup only the newest file stayed in cache

Edit: think an old edit got stuck in reviewable

[MarcusSorealheis]

Reviewable status: 2 of 2 LGTMs obtained, and all files reviewed, and 1 discussions need to be resolved

[aaronmondal]

Reviewed 1 of 2 files at r10, 1 of 1 files at r13, 1 of 1 files at r14, all commit messages.
Reviewable status: 2 of 2 LGTMs obtained, and all files reviewed, and pending CI: Analyze (javascript-typescript), Cargo Dev / ubuntu-22.04, Coverage, asan / ubuntu-22.04, pre-commit-checks, ubuntu-20.04 / stable, ubuntu-22.04, and 2 discussions need to be resolved

---

-- commits line 4 at r13:
nit: Looks like a leftover comment

[aaronmondal]

@KGrewal1 Great work ❤️

Reviewable status: 2 of 2 LGTMs obtained, and all files reviewed, and pending CI: Bazel Dev / macos-13, Bazel Dev / macos-14, Bazel Dev / ubuntu-24.04, Cargo Dev / macos-13, Cargo Dev / ubuntu-22.04, Coverage, Installation / macos-13, Installation / macos-14, Installation / ubuntu-22.04, NativeLink.com Cloud / Remote Cache / macos-14, NativeLink.com Cloud / Remote Cache / ubuntu-24.04, Publish image, Publish nativelink-worker-init, Publish nativelink-worker-lre-cc, Remote / large-ubuntu-22.04, asan / ubuntu-22.04, docker-compose-compiles-nativelink (22.04), integration-tests (22.04), macos-13, ubuntu-20.04 / stable, ubuntu-22.04, ubuntu-22.04 / stable, windows-2022 / stable, and 2 discussions need to be resolved

Aaron double checked the work. We are now waiting for the CI to complete. Thanks everyone here.

[aaronmondal]

Reviewable status: complete! 2 of 2 LGTMs obtained, and all files reviewed