Dotnix follow-up

[Ra33it0]

Project Abstract

Dotnix is a collection of Nix packages and NixOS modules designed for creating and managing Polkadot/Kusama Validator Nodes, emphasizing both security and ease of use.
This application is for a follow-up grant: 0e034e3

Grant level

• Level 1: Up to $10,000, 2 approvals
• Level 2: Up to $30,000, 3 approvals
• [ x ] Level 3: Unlimited, 5 approvals (for >$100k: Web3 Foundation Council approval)

Application Checklist

• The application template has been copied and aptly renamed (project_name.md).
• [ x ] I have read the application guidelines.
• [ x ] Payment details have been provided (Polkadot AssetHub (USDC & DOT) address in the application and bank details via email, if applicable).
• [ x ] I understand that an agreed upon percentage of each milestone will be paid in vested DOT, to the Polkadot address listed in the application.
• [ x ] I am aware that, in order to receive a grant, I (and the entity I represent) have to successfully complete a KYC/KYB check.
• [ x ] The software delivered for this grant will be released under an open-source license specified in the application.
• [ x ] The initial PR contains only one commit (squash and force-push if needed).
• [ x ] The grant will only be announced once the first milestone has been accepted (see the announcement guidelines).

[Noc2]

Thanks a lot for the application. Could you also integrate the DOT percentage? See the new template: https://github.com/w3f/Grants-Program/blob/master/applications/application-template.md#overview-1

[ajk-code]

Hey there
A couple of questions regarding this project, not sure I clearly understood while reading the application:

• In general, to me it sounds like an in-house deployment project, not something for a wider audience
• You mention as ecosystem fit 'people that want to host a validator, don't have the technical knowledge': Honestly I do not think someone without technical knowledge can deploy nixOS or nix packages on their machine. Installing docker on peoples notebooks with network bridges etc. might not be a good idea either. Copy/pasting from the polkadot wiki and using the parity repo is probably way easier, don't you think? A single line edit of the default config file is enough. If necessary in a VM for testing purposes.
• How is the system gonna be scanned for CVEs? What is this integrated scanner and which public CVE databases are used for the enrichment? How is an operator getting the result for high CVSS CVEs?
• How is usability improved by exposing the validator to polkadot.js?
• What is meant with polkadot.js in general? The libraries or the public frontend?
• What is the purpose of exposing a validators RPC in a 'validator' context? And why over VPN?
• Is the deliverable an OCI image or nix packages? Is is it podman compatible or docker only?
• Is dotnix gonna be available in the public nix repos (stable/unstable channels?)
• How is the polkadot binary built? From source, parity binaries or parity OCI/docker images?
• Is the secure validator mode supported? Referring to landlock/seccomp
• Is session key management somehow integrated apart from node key mgmt?
• How/who is gonna maintain this? What are the recurring costs?
• What about other standard security features such as selinux policies, secure boot, CIS compliance, fido2 authentication etc.?

Thank you

[PieWol]

Hey @ajk-code , thanks for participating in the review process 🙏 .

[Ra33it0]

Hey @ajk-code , thank you for your questions. I am happy to clear things up a bit.

The idea behind Dotnix is to simplify the deployment and administration of secure Polkadot validators by including various helper services for monitoring, backupping, e.g. into a single Nix flake that can be deployed through simple means.
In this grant application, e.g. we are doing the groundwork to deploy monitoring for visualizing the collected metrics on a single machine.
Later stages of Dotnix will allow deploying the same configuration on a mesh of machines that will be connected to each other using VPN.

Copying and pasting from the Polkadot wiki won't implement Linux best practices like updating the operating system, setting up the firewall, running regular backups, etc.,
All these fall into the domain of Dotnix which either already does or will in future cover it by default, aiming to simplify running a secure Polkadot validator.

The system is scanned for CVEs using Vulnix; the public database is NVD
The operator can choose to be alerted via mail or Matrix

With Polkadot.js the frontend is meant.
The VPN is primarily used to connect Dotnix machines to each other enabling monitoring and allowing backups, e.g. to machines that are otherwise unreachable to the public Internet.

The actual deliverable is a Nix flake that exposes tooling to deploy Dotnix to generate images and deploy Dotnix to arbitrary targets like Docker or bare metal.
Docker is used only since it is the preferred way of testing for W3F.

All parts that make sense to be in Nixpkgs, will be upstreamed, the domain-specific parts will remain in Dotnix.
We have chosen a similar architecture to ethereum.nix or nix-bitcoin

Polkadot is built from source using andresilva's polkadot.nix flake

Secure validator mode is supported and active by default in the current release.

Session Key Management has been integrated as a part of our deliverables within our previous Grant.

We're going to maintain this project. A the very least we would need to follow the biannual release cycle of Nixpkgs stable in order to allow automatic updates of the system.
Within this scope, Sporyon will continue to cover the maintenance costs to ensure the security of Dotnix in the forceable future.

These items are planned for subsequent grants, although there is still work to be done upstream, particularly with SELINUX and CIS compliance.

In principle, SELinux and Secure boot are possible today and are planned for subsequent grants.
Regarding CIS compliance, as of today, a default Dotnix installation fails 46% of aquasecurity's linux-bench (which implements CIS Distribution Independent Linux Benchmark version 1.1.0).
As for FIDO2, we have not researched how it can be integrated best, it's not on the roadmap, yet.
If there are strong opinions about using FIDO2, we could consider prioritizing the implementation of this feature.

Hope this clarifies things a bit
I am happy to hear your thoughts

[PieWol]

Hey @Ra33it0 ,
before we start to discuss the application any further I would like to let you know that we only support one DOT address per application. Somehow your company would need to find a way to distribute the vested DOT internally. Otherwise the application looks good and I'll share it with the committee after this issue has been resolved. Thanks!

[PieWol]

Hey @Ra33it0 ,
as much as I like your work and agree with the goals of the application to further increase the security of nodes within the Polkadot ecosystem, I think it's crucial that you provide some evidence gathered from the ecosystem that node providers actually desire this project and want to use this. Especially as we are talking about a level 3 grant and having funded the MVP already.

[keeganquigley]

Hi @Ra33it0 your previous deliveries were all very highly rated, so I guess my main question is, are any validators currently utilizing Dotnix? What are your plans to help the tool gain traction? I see there haven't been any commits made in four months or so. How about the Nix community, have they given any feedback or engagement? Do you plan to publish the NixOS modules on the official community repo?

[Ra33it0]

Thank you @PieWol for your feedback. I completely agree that concrete evidence from the ecosystem is key to demonstrating the value of this project. We’re currently facing a bit of a "chicken and egg" problem with Dotnix, as it's still in an MVP phase and not feature complete yet, which makes it difficult to offer a strong selling point for administrators to transition their running validators to it at this stage.

We chose to first build an MVP to prove that Dotnix is feasible. In doing so, we implemented tests for every component, beyond the usual testing, to ensure quality and reliability. Regarding W3F's funding of the MVP, we intentionally made it a relatively small project to establish trust between W3F and Sporyon. We felt that it was important to first demonstrate that we could do good work, proving Dotnix's viability before pursuing further development.

Our intention was always to earn our "deed" and demonstrate that Dotnix works, and then, in subsequent grants, focus on adding unique selling points such as the vulnerability scanner and state management, which we believe will be particularly valuable to validator providers and administrators.

We don’t intend to monetize Dotnix but see it as a direct contribution of a high-quality product to the Dotsama ecosystem. That said, I’d appreciate your advice on how we can generate the necessary metrics to show interest from node providers. Should we consider splitting the grant into smaller portions, perhaps multiple Level 2 grants, if the Level 3 grant doesn’t seem suitable for the current state of the project? We initially chose a Level 3 grant as we believed it would be easier for both sides, allowing us to deliver more within each milestone.

Any advice or insights you could share on how we can best approach this challenge would be greatly appreciated!

[Ra33it0]

Hi @keeganquigley thanks for your questions!

Regarding your first question, we can't definitively say how many validators are currently utilizing Dotnix since anyone can download our repo or run a nix command to execute it locally. As of now, we don't have direct visibility into usage data, but we plan to gain more insight as we continue developing and promoting the tool.

Our main strategy to help Dotnix gain traction is to present it at upcoming ecosystem conferences such as Polkadot Decoded and NixCon 2025. However, we believe it only makes sense to present it publicly when it's feature complete — that is, when we can offer concrete unique selling points apart from simply using NixOS, like vulnerability scanning, state management, and integrated monitoring. We also hope to be mentioned in the upstream documentation, though we feel it might be a bit early for that, as we want to ensure that dotnix moved out of the MVP phase.
Regarding the Nix community we got positive feedback from our local nix community in Berlin and a similar project from the Bitcoin ecosystem was presented at NixCon2024 and got good feedback as well.
We plan to make it an official Nix community repo.

In the meantime, we've written a guide detailing how to create a Polkadot validator using Dotnix on a machine that only has a rescue shell. Using kexec, you can install NixOS and then Dotnix — a helpful approach for setting up validators on cost-effective servers, which are often available at a discounted price in the Hetzner server auction. We hope this tutorial helps make it easier for more users to get started with Dotnix and Polkadot validation.

Regarding the lack of commits in the last four months, our entire team has been involved in the planning and preparations for NixCon 2024, which has required a significant portion of our time. Additionally, it took W3F about a month to evaluate our last milestone, which further impacted our development pace. Given these factors, we chose to focus on testing our validator on Westend and preparing a detailed feature roadmap rather than pushing out new commits. We see Dotnix as a direct contribution to the Dotsama ecosystem, and we don’t intend to monetize it. Rather, we see it as a fundamentally better way of validating Polkadot by leveraging Nix primitives such as reproducibility, modularity, security, and ease of use. We believe this will add to the resilience of the Polkadot network in the long term.

We belive Web 3 should be a decentralized effort not just regarding clients but also regarding infrastructure like the validators of a network , if everything even critical infrastructure is heavily reliant on docker we are introducing a single point of failure which is missing the point of how we understand web3.
Other ecosystems like Bitcoin, Ethereum have realized this and are mitigating this fact using tools like NixBitcoin or Ethereum Nix we try to achieve the same for the Polkadot ecosystem with this Dotnix.

Thanks again for your questions, and we look forward to hear your thoughts.