Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/nix support #120

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .github/workflows/build_with_nix.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
name: build with nix
on:
pull_request:
branches:
- main
workflow_dispatch: {}
concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
jobs:
build_and_test:
strategy:
matrix:
os: ['ubuntu-22.04', 'macos-14']
runs-on: ${{matrix.os}}
steps:
- uses: actions/checkout@v4
- uses: cachix/install-nix-action@v25
- uses: cachix/cachix-action@v15
with:
name: tket
signingKey: ${{ secrets.CACHIX_AUTH_TOKEN }}
- name: Build and test pytket-qir
run: nix flake check -L
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,5 @@ obj
docs/extensions
.ipynb_checkpoints
pytket/qir/_metadata.py
# nix
result
116 changes: 116 additions & 0 deletions flake.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

38 changes: 38 additions & 0 deletions flake.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
{
description = "Pytket QIR Extension";
nixConfig.extra-substituters = "https://tket.cachix.org https://cache.nixos.org";
nixConfig.trusted-public-keys = ''
tket.cachix.org-1:ACdm5Zg19qPL0PpvUwTPPiIx8SEUy+D/uqa9vKJFwh0=
cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
'';
inputs.flake-utils.url = "github:numtide/flake-utils";
inputs.tket.url = "github:CQCL/tket";
inputs.nixpkgs = {
# url = "github:nixos/nixpkgs";
follows = "tket/nixpkgs";
};
outputs = { self, nixpkgs, flake-utils, tket }:
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs {
inherit system;
overlays = [
(self: super: {
inherit (tket.packages."${system}") tket pytket;
})
(import ./nix-support/pyqir.nix)
(import ./nix-support/pytket-qir.nix)
];
};
in {
packages = {
pytket-qir = pkgs.pytket-qir;
};
devShells = {
default = pkgs.mkShell { buildInputs = [ pkgs.pytket-qir ]; };
};
checks = {
pytket-qir-tests = pkgs.pytket-qir;
};
});
}
107 changes: 107 additions & 0 deletions nix-support/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# Nix support for pytket-qir

## Background

Tket now comes with Nix support. What this means is that the
steps required to launch into an environment with tket and pytket
available on Linux x86-64 and Mac Silicon machines is a single
invocation:

```
nix develop github:CQCL/tket
```

Currently this will build all of the necessary dependencies,
establish the environment, and enter into a shell where
tket and pytket are available for use. Soon we aim to add Cachix,
so that the dependencies are served pre-built to the user.

## Using pytket-qir

The nix.flake in this repository is able to access tket and pytket
from the CQCL/tket nix flake, pinned at the commit and hash noted
in flake.lock (see the Maintenance section). It then exposes
pytket-qir as a package, and provides a development shell containing
pytket-qir.

To launch into an environment with pytket-qir available, use
```
nix develop github:CQCL/pytket-qir
```

from which pytket-qir is now available
```
y$ nix develop github:CQCL/pytket-qir/feature/nix-support

$ python3
Python 3.11.7 (main, Dec 4 2023, 18:10:11) [GCC 13.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pytket.circuit import Circuit
>>> from pytket.qir.conversion.api import QIRFormat, pytket_to_qir
>>> c = Circuit(2)
>>> c.H(0)
[H q[0]; ]
>>> c.H(1)
[H q[0]; H q[1]; ]
>>> c.CX(0, 1)
[H q[0]; H q[1]; CX q[0], q[1]; ]
>>> c.H(0)
[H q[0]; H q[1]; CX q[0], q[1]; H q[0]; ]
>>> c.H(1)
[H q[0]; H q[1]; CX q[0], q[1]; H q[0]; H q[1]; ]
>>> print(pytket_to_qir(c, name="example", qir_format=QIRFormat.STRING))
; ModuleID = 'example'
source_filename = "example"

%Qubit = type opaque
%Result = type opaque
; ... and so on
```

This will take some time if tket has not been built on the target machine yet,
as the necessary downloading and building will take place before the environment
is ready. Once we get Cachix it will be much faster, as the builds will be available
from our cache.

## Quirks

At the time of writing, mypy checks for `warn_unused_ignores` are disabled for
nix-only builds, as they lead to false-positives that fail the flake checks.

These are only disabled for the nix version of this repository. This is done
in nix-support/pytket-qir.nix, and is accomplished by modifying the mypy.ini file
at the time of copying it to the Nix store.

## Maintenance

When changes are made, run `nix flake check` to ensure that the nix build works.
If errors are present, run `nix flake check -L` to see the full logs.

The three nix dependencies for this repository are nixpkgs, flake-utils, and tket.

To update all three, simply run:
```
nix flake update
```
and commit the updated flake.lock after verifying with `nix flake check`.

Dependencies can also be updated selectively with e.g.

```
nix flake lock --update-input tket
```

pyqir is not present on the nix store, so we build it manually in nix-support/pyqir.nix.

If the version needs to be updated, follow this procedure:
1. Unset the resulting hashes with `pyqir-hash = "";` and `pytket-cargo-hash = "";`
2. Update pyqir-version to an available tag on the qir-alliance/pyqir github
3. Run `nix flake check`
4. The build will fail with a hash mismatch. Use the hash it provides to replace the blank `pyqir-hash`.
5. Run `nix flake check`
6. The build will fail with a hash mismatch. Use the hash it provides to replace the blank `pyqir-cargo-hash`.
7. Run `nix flake check` again. This time it should succeed.

If the build fails for another reason, further investigation is required.
It is likely down to required change in build steps for pyqir, or could be
a breaking change in pyqir that needs addressing in pytket-qir.
45 changes: 45 additions & 0 deletions nix-support/pyqir.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
self: super:
let
lib = super.lib;
llvm = super.llvm_14;
llvm-v-major = lib.versions.major llvm.version;
llvm-v-minor = builtins.substring 0 1 (lib.versions.minor llvm.version);
pyqir-version = "0.10.0";
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to import the version from the python packages here, or do we now need to update this in two places?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case it is customary to provide the exact git revision to pull, then specify the build process, as pyqir isn't available on nixpkgs.

I can update it to scan setup.py for a specific version, though if it changes to using a loose version match then the process will fail.

Though I will add that I am committed to supporting the nix support for this repository and others, so if a change breaks the nix support, or if a version update is required, I'm absolutely happy to action it.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I case I am doing the update, is there anything else I need to do besides updating the version in this file as well?

pyqir-hash = sha256:dZd+U3vyHb9rrNB90XiLn6fAbsg3Xk9Htnw5Ce/vra4=;
pyqir-cargo-hash = sha256:U964/0ekTVgl5CCU4xgExgFhSIP1RKocNbjScWw4BTM=;
in
{
pyqir = super.python3Packages.buildPythonPackage rec {
pname = "pyqir";
version = pyqir-version;
format = "pyproject";
src = super.fetchFromGitHub {
owner = "qir-alliance";
repo = "pyqir";
rev = "v${pyqir-version}";
sha256 = pyqir-hash;
};

cargoDeps = super.rustPlatform.fetchCargoTarball {
inherit src;
name = "pyqir-${pyqir-version}";
hash = pyqir-cargo-hash;
};

buildAndTestSubdir = "pyqir";

nativeBuildInputs = with super.rustPlatform; [ cargoSetupHook maturinBuildHook ];

buildInputs = [ llvm super.libxml2.dev ];

maturinBuildFlags = "-F llvm${llvm-v-major}-${llvm-v-minor}";

preConfigure = ''
export LLVM_SYS_${llvm-v-major}${llvm-v-minor}_PREFIX=${llvm.dev}
'';

pythonImportsCheck = [ "pyqir" ];

passthru.llvm = llvm;
};
}
54 changes: 54 additions & 0 deletions nix-support/pytket-qir.nix
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
self: super:
let
metadata = builtins.readFile ../_metadata.py;
versions =
builtins.match ''.*_version__ *= *["']([^"']+)["'].*'' metadata;
version = if builtins.length versions > 0 then
builtins.elemAt versions 0
else
builtins.trace "Warning: unable to find version. Defaulting to 0.0.0" "0.0.0";
in {
pytket-qir = super.python3.pkgs.buildPythonPackage {
pname = "pytket-qir";
version = version;
src = super.stdenv.mkDerivation{
name = "pytket-qir-sources";
phases = [ "installPhase" ];
installPhase = ''
mkdir -p $out;
cp -r ${../pytket} $out/pytket;
cp -r ${../tests} $out/tests;

cp ${../setup.py} $out/setup.py;
cp ${../README.md} $out/README.md; # required for setup's long description
cp ${../pytest.ini} $out/pytest.ini;
cp ${../_metadata.py} $out/_metadata.py;

# on nix versions of scipy and ipython, stubs are missing.
# adjust mypy.ini to ignore these errors.
(
cat ${../mypy.ini};
cat <<EOF
[mypy-scipy.*]
ignore_missing_imports = True
ignore_errors = True

[mypy-IPython.display.*]
ignore_missing_imports = True
ignore_errors = True
EOF
) >> $out/mypy.ini;
'';
};
propagatedBuildInputs = [ super.pytket self.pyqir ];
checkInputs = with super.python3Packages; [ pytest mypy ];
checkPhase = ''
export HOME=$TMPDIR;

python -m mypy --config-file=mypy.ini --no-incremental -p pytket -p tests

cd tests;
python -m pytest -s .
'';
};
}