Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for static nix-tools #2157

Merged
merged 101 commits into from
Apr 3, 2024
Merged

Add support for static nix-tools #2157

merged 101 commits into from
Apr 3, 2024

Conversation

zeme-wana
Copy link
Collaborator

@zeme-wana zeme-wana commented Feb 5, 2024

Use static versions of nix-tools. Because these form the foundation of IFDs in haskell.nix, having them as download-unpack-and-run available should speed up evaluation and IFDs significantly. Previously we dynamically built them on-demand, but that meant that not only the dependency closures were larger (including potentially build tools, ...) but also when ever there was a change to haskell.nix that invalidated the nix-tools build plan, they were re-built on demand.

By decoupling both, we should be able to cut the worst-case scenario to download time + unpack time + runtime of the nix-tools.

nix-tools/static/outputs.nix Outdated Show resolved Hide resolved
nix-tools/static/outputs.nix Outdated Show resolved Hide resolved
nix-tools/static/packaging.nix Show resolved Hide resolved
static-gmp = (final.gmp.override { withStatic = true; }).overrideDerivation (old: {
configureFlags = old.configureFlags ++ [ "--enable-static" "--disable-shared" ];
});
static-libblst = (final.libblst.override { enableShared = false; }).overrideDerivation (old: {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[diff] reported by reviewdog 🐶

Suggested change
static-libblst = (final.libblst.override { enableShared = false; }).overrideDerivation (old: {
static-libblst = (final.libblst.override { enableShared = false; }).overrideDerivation (_old: {

};


zippedToolsNoIfdFor = fragment-name:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should document this in detail, why we need this, and what the issue with keeping IFDs is, and why using recursive-nix helps us here.

Copy link
Collaborator

@angerman angerman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In general this looks good. I'm not that much a fan of splitting flake files into multiple files 🤔

Copy link
Member

@andreabedini andreabedini left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure why the overlays for the crypto libs are included here but the rest looks good.

nix-tools/static/outputs.nix Show resolved Hide resolved
nix-tools/static/packaging.nix Show resolved Hide resolved
Copy link
Collaborator

@hamishmack hamishmack left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to include the GitHub actions as well?

nix-tools/flake.nix Show resolved Hide resolved

apply-hnix-patches = {
packages.hnix.patches = [
(builtins.toFile "plutus-core.patch" ''
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

?????????

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no idea why this is named the way it's named. I guess I will just rename it.



static-libraries-overlay = final: _prev: {
static-libsodium-vrf = final.libsodium-vrf.overrideDerivation (old: {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't used by nix-tools???

Copy link
Collaborator Author

@zeme-wana zeme-wana Feb 7, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's used here

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean... where you added a series of probably unnecessary static library dependencies? We can't just randomly copy this stuff around, it has to depend on what libraries the package actually uses!

@michaelpj
Copy link
Collaborator

This PR seems to have a lot of stuff in it related to building static Cardano packages which shouldn't be at all necessary for building nix-tools.

@zeme-wana
Copy link
Collaborator Author

This PR seems to have a lot of stuff in it related to building static Cardano packages which shouldn't be at all necessary for building nix-tools.

I see, I can try removing all of these (except static-gmp --- I suppose we need that one) and see what happens. If it builds, then we don't need iohkNix

@zeme-wana zeme-wana force-pushed the static-nix-tools branch 3 times, most recently from 19dbbc3 to 5c67a95 Compare February 13, 2024 09:27
@zeme-wana zeme-wana force-pushed the static-nix-tools branch 2 times, most recently from c1a50f2 to 600297b Compare February 21, 2024 09:04
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This file seems to only have formatting changes. If we want to do formatting, let's do this differently, but not throw into Pull Requests.

@hamishmack hamishmack marked this pull request as ready for review March 31, 2024 12:46
@hamishmack hamishmack merged commit 77d8200 into master Apr 3, 2024
377 of 378 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants