Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wire things up in a flake #4

Open
blaggacao opened this issue Nov 14, 2022 · 11 comments · May be fixed by #15
Open

Wire things up in a flake #4

blaggacao opened this issue Nov 14, 2022 · 11 comments · May be fixed by #15

Comments

@blaggacao
Copy link

I've read the TL;DR.

But I'm looking at a wall to climb, now.

As others have pointed out interop is cheap with: https://github.com/edolstra/flake-compat

@blaggacao
Copy link
Author

... wall is in the context of a service provider to divnix/std#165

@blaggacao
Copy link
Author

blaggacao commented Nov 15, 2022

This comment looks like particularly relevant to your flake TL; DR; → edolstra/flake-compat#39 (comment)

@olebedev
Copy link
Collaborator

Hi @blaggacao,

This comment looks like particularly relevant to your flake TL; DR; → edolstra/flake-compat#39 (comment)

From my understanding, this comment is about to don't copying the whole repo into the Nix store but this is not about what I have been talking in this comment. I was more talking about the flakes implementation and how they are coupled with git tool and rely on it to infer a diff and what files have been affected by a change set.

@blaggacao
Copy link
Author

blaggacao commented Nov 15, 2022

In this case, I don't think I grok the issue, yet.

But it also seems like the undesired behaviour is entirely attached to the use of code paths involved in the new flakes commands.

So a forward-compatible flake layout, per se, wouldn't be affected, at all.

There might be actually a couple of additional requirements to weave js2nix into an integration framework, such as not claiming hard coded contracts (on files such as packages.nix and it's output schema). But maybe that's not even actually the case but just implied (in the eye of the reader) by the Readme's posture. I haven't dug deep enough, yet.

@typetetris
Copy link

Maybe there are some misunderstandings.

I read "Wire things up in a flake" as "Please provide a flake.nix/flake.lock for js2nix".

Together with flake-compat this shouldn't hurt monorepo users
too much, as they can continue to ignore flakes and only have the slight discomfort of having one copy
of the js2nix source code in the store. No "flake code paths", which are not already taken, should be
taken in nix/nix-daemon in this case, I think.

The results produced by js2nix should be easily usable in a flake, if the code generation is used to avoid
import from derivation.

Am I missing something?

@olebedev
Copy link
Collaborator

olebedev commented Jan 20, 2023

Oh, I haven't read this in that way. I am fully on board with having flake.{nix,lock}files along the way! More than happy to review PRs if someone interested in contributing into it. Otherwise, I will try to allocate some time for that, but note, for the @Canva use case it's rather theoretical, ephemeral benefit since we don't use flakes. That is, I most likely I will do it in the blind, without relying on a particular use case, so contribution of those who use flakes in practice is very much appreciated :)

@typetetris
Copy link

typetetris commented Jan 20, 2023

With flake.{nix,lock} and flake-compat, flake.nix would be the source of truth.

So maybe that is not a great idea, if you don't want to look at some flake documentation and doing the occasional nix --extra-experimental-features "flake nix-command" flake update command.

Having separate flake.nix and default.nix/shell.nix (no flake-compat here) would maybe diverge to fast to not be frustrating for everyone.

@olebedev
Copy link
Collaborator

The only one aspect of flake.{nix,lock} + flake-compat usage I haven't thought about before is the ability to define the dependencies at the call site. The idea of flakes is that you describe dependencies of your Nix package (flake) in the repository so you don't have to deal with its dependencies at the call site, but if you want, it's easy to do with flakes as well.

This is great, however, IIUC flake-compat in default.nix file removes capability of passing dependencies with callPackage function call and will anyways read the flake.lock file to determine the project dependency closure regardless of what you passing in callPackage function call, no?

For example, at the call-site, I would like to have the run time dependency of Node.js to be the 12 version:

with import <nixpkgs> { };

let
  js2nix = callPackage (builtins.fetchGit {
    url = "ssh://[email protected]/canva-public/js2nix.git";
    ref = "main";
  }) {
    nodejs = nodejs_12-x;
  };
  nodeModules = js2nix.makeNodeModules ./package.json {
    tree = js2nix.load ./yarn.lock { };
  };
in nodeModules

But, if we use flake-compat in the default.nix, it won't be possible to override, right?

I that is correct understanding of how the flake-compat work, I would suggest to have the default.nix file as an entry point for both the project flake and for the non-flake consumers.

Meaning, the default.nix file returns a function that expects project dependencies passed thought, like any other package in the nixpkgs repository, and in the flake.nix we do callPackage of the default.nix with the dependencies declared in flake.{nix,lock} but not the other way around; i.e. when we use flake-compat in the default.nix.

This will resolve the issue with dependencies injection for non-flake consumers along with make the project a first-class flake for those who use flakes at call sites. And the most importantly we remove the divergence you have been talking above.

@typetetris, @blaggacao, what do you think?

@blaggacao
Copy link
Author

blaggacao commented Jan 21, 2023

{inputs, system}:

let
  inherit (inputs) nixpkgs js2nix;
  inherit (nixpkgs.legacyPackages.${system}) callPackage;
  js2nix' = callPackage js2nix {
    nodejs = nodejs_12-x;
  };
  nodeModules = js2nix'.makeNodeModules ./package.json {
    tree = js2nix'.load ./yarn.lock { };
  };
in nodeModules

Should * work with a top __functor outputs that makes a js2nix flake a callPackage-able package function.

* At least the nixpkgs library version of functionArgs (the implementation detail behind callPackage) is aware of functors.

Iirc, I've made a patch to flake-compat a while back to not remove functors from the output.

@olebedev
Copy link
Collaborator

It removes the __functor property here https://github.com/edolstra/flake-compat/blob/master/default.nix#L193.

I am wondering, would it work if we just do pkgs.callPackage ./default.nix from flake.nix? This should remove the divergence problem and make flake and non-flake folks happy, no?

@blaggacao
Copy link
Author

blaggacao commented Jan 22, 2023

Oh, I did remove the functor, then. Must have been for another reason in the Nix CLI jungle. 😄

That approach would also work and in most cases people can operate with .overrideAttrs if they need to amend any (package function) inputs.

@olebedev olebedev linked a pull request May 1, 2023 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants