chore: add shell command builder

[Itay-Tsabary-Starkware]

commit-id:96859c27

---

Stack:

• chore(tests_integration): replace run with build and direct invocation #2270
• chore(sequencer_node): replace run node shell command #2266
• chore(sequencer_node): compile the node, not the entire project #2265
• chore: add shell command builder #2264 ⬅
• chore(infra): add project dir path fn #2255
• chore(infra): canonicalize path resolution #2254

⚠️ Part of a stack created by spr. Do not merge manually using the UI - doing so may have unexpected results.

[reviewable-StarkWare]

This change is 

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.37%. Comparing base (e3165c4) to head (ae3a0f8).
Report is 572 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff             @@
##             main    #2264       +/-   ##
===========================================
+ Coverage   40.10%   77.37%   +37.26%     
===========================================
  Files          26      386      +360     
  Lines        1895    40309    +38414     
  Branches     1895    40309    +38414     
===========================================
+ Hits          760    31188    +30428     
- Misses       1100     6820     +5720     
- Partials       35     2301     +2266     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

---

🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests
• JS Bundle Analysis - Avoid shipping oversized bundles

[github-actions]

Artifacts upload workflows:

• Started at 2024-11-25T10:18:58Z
• Started at 2024-11-25T11:52:46Z
• Started at 2024-11-25T12:35:37Z
• Started at 2024-11-25T12:38:16Z
• Started at 2024-11-25T12:38:16Z
• Started at 2024-11-25T12:46:56Z
• Started at 2024-11-25T12:46:56Z
• Started at 2024-11-25T12:46:56Z
• Started at 2024-11-25T12:51:28Z

[github-actions]

Benchmark movements:
full_committer_flow performance regressed!
full_committer_flow time: [30.112 ms 30.161 ms 30.214 ms]
change: [+1.6244% +1.8821% +2.1460%] (p = 0.00 < 0.05)
Performance has regressed.
Found 5 outliers among 100 measurements (5.00%)
4 (4.00%) high mild
1 (1.00%) high severe

[github-actions]

Benchmark movements:
tree_computation_flow performance regressed!
tree_computation_flow time: [35.112 ms 35.615 ms 36.198 ms]
change: [+1.1938% +2.7710% +4.4111%] (p = 0.00 < 0.05)
Performance has regressed.
Found 9 outliers among 100 measurements (9.00%)
1 (1.00%) high mild
8 (8.00%) high severe

[ArniStarkware]

Reviewed 2 of 5 files at r1, 2 of 2 files at r2, all commit messages.
Reviewable status: 4 of 5 files reviewed, 2 unresolved discussions (waiting on @Itay-Tsabary-Starkware)

---

a discussion (no related file):
Is the end goal of the function create_shell_command to replace every other instance of Command::new in the repo?

Playing devil's advocate:
Isn't it an overkill?
Answer:
We just spent a couple of weeks fighting this kind of inconsistency. This assures the CI runs more similarly to an executable.

---

crates/infra_utils/src/command.rs line 27 at r2 (raw file):

    env::vars().filter(|(key, _)| key.starts_with("CARGO_")).for_each(|(key, _)| {
        command.env_remove(key);
    });

This is just for safety, right? to ensure that every shell command is run consistently.

Code quote:

    // Filter out all CARGO_ environment variables.
    env::vars().filter(|(key, _)| key.starts_with("CARGO_")).for_each(|(key, _)| {
        command.env_remove(key);
    });

[Itay-Tsabary-Starkware]

Reviewable status: 4 of 5 files reviewed, 2 unresolved discussions (waiting on @ArniStarkware)

---

crates/infra_utils/src/command.rs line 27 at r2 (raw file):

Previously, ArniStarkware (Arnon Hod) wrote…

This is just for safety, right? to ensure that every shell command is run consistently.

This is to ensure (or, make an effort in the direction of ensuring) that we do not have side effects of running our executables due to using cargo. I.e., I'd like this to be a step in a direction of reducing the differences of cargo run X and cargo build X; ./X .

[Itay-Tsabary-Starkware]

Reviewable status: 4 of 5 files reviewed, all discussions resolved (waiting on @ArniStarkware)

---

a discussion (no related file):

Previously, ArniStarkware (Arnon Hod) wrote…

Is the end goal of the function create_shell_command to replace every other instance of Command::new in the repo?

Playing devil's advocate:
Isn't it an overkill?
Answer:
We just spent a couple of weeks fighting this kind of inconsistency. This assures the CI runs more similarly to an executable.

No, only those that we need to make sure we can run without cargo. Example, the sequencer node.

[ArniStarkware]

Reviewed 1 of 5 files at r1.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @Itay-Tsabary-Starkware)