Next level of repeatable builds/environments/making nix way more useful

[ddevz]

Is your feature request related to a problem?

There are many use cases for nix repeatable builds being able to prevent broken programs, but are missed out on because it requires manual intervention beforehand.

(I'll actually explain detail for my idea, i'm keeping this "concise")

Proposed solution

Ok, first I'm pretty new to nix, so I'm going to be missing information, so please forgive any terminology mistakes. To explain my feature, I'll need to use a feature that I assume nix has, and I just can't find. So I'll propose that feature first, then get to my actual point.

The feature I assume nix has but I just can't find:
When using a program/environment/profile/whatever, I have found that one can type things like "nix-env --query" to get a list of the packages installed, and the versions of them.
What I need to refer to is being able to do something like "nix-env --query --full", where it spits out the information of the packages, the versions, and the hashes, and all other information needed to be able to rebuild the environment/program/profile/whatever. A important part of this is spitting it out in a format that nix will be able to read (.nix?).
So for example: When running a script on bash 5.2.15 on machine 1 works fine, but running the same script on bash 5.2.15 on machine 2 does not work, you would go to the nix environment/profile/whatever that you are running the script in, and do "nix-env --query --full > make-script-work.nix", then copy make-script-work.nix to machine 2, and then use nix on machine 2 to use make-script-work.nix to create the exact same environment on machine 2 and be able to run the script there.
This also allows a end user to log the environment that they are using for future use.

Now, The actual feature that I am proposing (and am assuming nix does not already have):

Short Version: Automatically generate the files
(see long version below)

So some use cases:

• software development needs repeatable environments (I'll assume you are already familiar with the arguments for using nix instead of docker for this, and not go into it)
• end users need repeatable environments
  For end users, take a end user program like for example, blender. When you load a blender 2.8 file in later versions of blender (like 3.4) most of it works, but there are some things that break. It's not specific to blender. Many programs are like this but i'll say blender instead of saying "the program" every time.
  Now imagine a user that develops in one version of a program, lets say blender 2.8, then stops working on the project, then gets a new computer, then gets another new computer, and now many years later goes to edit the file written with blender 2.8.

The first question is: since not all programs write metadata saying what version of the program was used to generate the file, how does he know what version of the particular program the file was written with?

The idea is to allow a user/system administrator to say "i don't want this to happen to files under /home/user1" (or whatever paths). Then monitor file writes to the filesystem. When a file is written, find the process-id, the command (with path) that is doing the writing, what nix package the binary for the process is in, the parent of that process, and the parent of that process, etc.
The idea is to get to the nix environment/profile/whatever, and then do the equivalent of the "nix-env --query --full > {name of file being written to including path}.nixlog" to generate the nix information that could regenerate the entire environment for the process that just wrote to the file, and add that to the end of the log (and probably add a timestamp).

The answer to the question: "since not all programs write metadata saying what version of the program was used to generate the file, how does he know what version of the particular program the file was written with?" becomes:
just look at the filename.nixlog file

It also answers the question of how to run it again. The answer is to install the old version of blender from the .nixlog and use that version.

Of course checking the .nixlog files into git would allow alignment of the environment to a specific change instead of the environment to just a timestamp, plus hopefully in the commit message they mention if its still working and other information.

There are tons of other features that could be added to it, but hopefully you are getting the basic idea by now.

This feature could be a great for users, but now if we go back to thinking about software development, we are essentially talking about reproducible environments in software development having the option to become a automatic thing instead of the developer having to manually create these environments in order to get the reproducibility. Getting it working for yourself, then saying in git "This version works" is the only information people need to provide in order for others to be able to use the same environment.

Additional context

Possibly useful info:

• This might be able to be implemented using inotify
• The command lsof is one example of getting the information of what process has the file open

Checklist

• [X ] checked latest Nix manual (source)
• [X ] checked open feature issues and pull requests for possible duplicates

---

Add 👍 to issues you find important.

[roberth]

something like "nix-env --query --full", where it spits out the information of the packages, the versions, and the hashes, and all other information needed to be able to rebuild the environment/program/profile/whatever.

Flakes with #11749 would bring something like that within reach. Doing it without Flakes is tricky because then we don't have a standard, stable descriptor for it.

inotify

The "environment recording" idea is interesting. I feel like it's not really Nix specific; maybe something already exists that could be combined with Nix to form a tool that does what you want? The sum would definitely be greater than the parts.