-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/nix support #120
Open
jake-arkinstall
wants to merge
8
commits into
main
Choose a base branch
from
feature/nix-support
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Feature/nix support #120
Changes from all commits
Commits
Show all changes
8 commits
Select commit
Hold shift + click to select a range
fe10a52
Added nix support (WIP)
jake-arkinstall 1964d31
Updated flake.lock and pyqir version. Enabled mypy
jake-arkinstall 2195b7d
Revert change on mypy.ini and move it to nix derivation for nix-only …
jake-arkinstall f0bdfa5
Added nix build workflow
jake-arkinstall 30e7a85
Added Nix README
jake-arkinstall e38e68e
Updated nix flake to reflect TKET's latest commit
jake-arkinstall ab0a4c2
Updated workflow - NOTE: CACHIX_AUTH_TOKEN env is now required
jake-arkinstall 19d8ec2
Update .github/workflows/build_with_nix.yml
cqc-alec File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,24 @@ | ||
name: build with nix | ||
on: | ||
pull_request: | ||
branches: | ||
- main | ||
workflow_dispatch: {} | ||
concurrency: | ||
group: ${{ github.workflow }}-${{ github.ref }} | ||
cancel-in-progress: true | ||
jobs: | ||
build_and_test: | ||
strategy: | ||
matrix: | ||
os: ['ubuntu-22.04', 'macos-14'] | ||
runs-on: ${{matrix.os}} | ||
steps: | ||
- uses: actions/checkout@v4 | ||
- uses: cachix/install-nix-action@v25 | ||
- uses: cachix/cachix-action@v15 | ||
with: | ||
name: tket | ||
signingKey: ${{ secrets.CACHIX_AUTH_TOKEN }} | ||
- name: Build and test pytket-qir | ||
run: nix flake check -L |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -11,3 +11,5 @@ obj | |
docs/extensions | ||
.ipynb_checkpoints | ||
pytket/qir/_metadata.py | ||
# nix | ||
result |
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,38 @@ | ||
{ | ||
description = "Pytket QIR Extension"; | ||
nixConfig.extra-substituters = "https://tket.cachix.org https://cache.nixos.org"; | ||
nixConfig.trusted-public-keys = '' | ||
tket.cachix.org-1:ACdm5Zg19qPL0PpvUwTPPiIx8SEUy+D/uqa9vKJFwh0= | ||
cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= | ||
''; | ||
inputs.flake-utils.url = "github:numtide/flake-utils"; | ||
inputs.tket.url = "github:CQCL/tket"; | ||
inputs.nixpkgs = { | ||
# url = "github:nixos/nixpkgs"; | ||
follows = "tket/nixpkgs"; | ||
}; | ||
outputs = { self, nixpkgs, flake-utils, tket }: | ||
flake-utils.lib.eachDefaultSystem (system: | ||
let | ||
pkgs = import nixpkgs { | ||
inherit system; | ||
overlays = [ | ||
(self: super: { | ||
inherit (tket.packages."${system}") tket pytket; | ||
}) | ||
(import ./nix-support/pyqir.nix) | ||
(import ./nix-support/pytket-qir.nix) | ||
]; | ||
}; | ||
in { | ||
packages = { | ||
pytket-qir = pkgs.pytket-qir; | ||
}; | ||
devShells = { | ||
default = pkgs.mkShell { buildInputs = [ pkgs.pytket-qir ]; }; | ||
}; | ||
checks = { | ||
pytket-qir-tests = pkgs.pytket-qir; | ||
}; | ||
}); | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,107 @@ | ||
# Nix support for pytket-qir | ||
|
||
## Background | ||
|
||
Tket now comes with Nix support. What this means is that the | ||
steps required to launch into an environment with tket and pytket | ||
available on Linux x86-64 and Mac Silicon machines is a single | ||
invocation: | ||
|
||
``` | ||
nix develop github:CQCL/tket | ||
``` | ||
|
||
Currently this will build all of the necessary dependencies, | ||
establish the environment, and enter into a shell where | ||
tket and pytket are available for use. Soon we aim to add Cachix, | ||
so that the dependencies are served pre-built to the user. | ||
|
||
## Using pytket-qir | ||
|
||
The nix.flake in this repository is able to access tket and pytket | ||
from the CQCL/tket nix flake, pinned at the commit and hash noted | ||
in flake.lock (see the Maintenance section). It then exposes | ||
pytket-qir as a package, and provides a development shell containing | ||
pytket-qir. | ||
|
||
To launch into an environment with pytket-qir available, use | ||
``` | ||
nix develop github:CQCL/pytket-qir | ||
``` | ||
|
||
from which pytket-qir is now available | ||
``` | ||
y$ nix develop github:CQCL/pytket-qir/feature/nix-support | ||
|
||
$ python3 | ||
Python 3.11.7 (main, Dec 4 2023, 18:10:11) [GCC 13.2.0] on linux | ||
Type "help", "copyright", "credits" or "license" for more information. | ||
>>> from pytket.circuit import Circuit | ||
>>> from pytket.qir.conversion.api import QIRFormat, pytket_to_qir | ||
>>> c = Circuit(2) | ||
>>> c.H(0) | ||
[H q[0]; ] | ||
>>> c.H(1) | ||
[H q[0]; H q[1]; ] | ||
>>> c.CX(0, 1) | ||
[H q[0]; H q[1]; CX q[0], q[1]; ] | ||
>>> c.H(0) | ||
[H q[0]; H q[1]; CX q[0], q[1]; H q[0]; ] | ||
>>> c.H(1) | ||
[H q[0]; H q[1]; CX q[0], q[1]; H q[0]; H q[1]; ] | ||
>>> print(pytket_to_qir(c, name="example", qir_format=QIRFormat.STRING)) | ||
; ModuleID = 'example' | ||
source_filename = "example" | ||
|
||
%Qubit = type opaque | ||
%Result = type opaque | ||
; ... and so on | ||
``` | ||
|
||
This will take some time if tket has not been built on the target machine yet, | ||
as the necessary downloading and building will take place before the environment | ||
is ready. Once we get Cachix it will be much faster, as the builds will be available | ||
from our cache. | ||
|
||
## Quirks | ||
|
||
At the time of writing, mypy checks for `warn_unused_ignores` are disabled for | ||
nix-only builds, as they lead to false-positives that fail the flake checks. | ||
|
||
These are only disabled for the nix version of this repository. This is done | ||
in nix-support/pytket-qir.nix, and is accomplished by modifying the mypy.ini file | ||
at the time of copying it to the Nix store. | ||
|
||
## Maintenance | ||
|
||
When changes are made, run `nix flake check` to ensure that the nix build works. | ||
If errors are present, run `nix flake check -L` to see the full logs. | ||
|
||
The three nix dependencies for this repository are nixpkgs, flake-utils, and tket. | ||
|
||
To update all three, simply run: | ||
``` | ||
nix flake update | ||
``` | ||
and commit the updated flake.lock after verifying with `nix flake check`. | ||
|
||
Dependencies can also be updated selectively with e.g. | ||
|
||
``` | ||
nix flake lock --update-input tket | ||
``` | ||
|
||
pyqir is not present on the nix store, so we build it manually in nix-support/pyqir.nix. | ||
|
||
If the version needs to be updated, follow this procedure: | ||
1. Unset the resulting hashes with `pyqir-hash = "";` and `pytket-cargo-hash = "";` | ||
2. Update pyqir-version to an available tag on the qir-alliance/pyqir github | ||
3. Run `nix flake check` | ||
4. The build will fail with a hash mismatch. Use the hash it provides to replace the blank `pyqir-hash`. | ||
5. Run `nix flake check` | ||
6. The build will fail with a hash mismatch. Use the hash it provides to replace the blank `pyqir-cargo-hash`. | ||
7. Run `nix flake check` again. This time it should succeed. | ||
|
||
If the build fails for another reason, further investigation is required. | ||
It is likely down to required change in build steps for pyqir, or could be | ||
a breaking change in pyqir that needs addressing in pytket-qir. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,45 @@ | ||
self: super: | ||
let | ||
lib = super.lib; | ||
llvm = super.llvm_14; | ||
llvm-v-major = lib.versions.major llvm.version; | ||
llvm-v-minor = builtins.substring 0 1 (lib.versions.minor llvm.version); | ||
pyqir-version = "0.10.0"; | ||
pyqir-hash = sha256:dZd+U3vyHb9rrNB90XiLn6fAbsg3Xk9Htnw5Ce/vra4=; | ||
pyqir-cargo-hash = sha256:U964/0ekTVgl5CCU4xgExgFhSIP1RKocNbjScWw4BTM=; | ||
in | ||
{ | ||
pyqir = super.python3Packages.buildPythonPackage rec { | ||
pname = "pyqir"; | ||
version = pyqir-version; | ||
format = "pyproject"; | ||
src = super.fetchFromGitHub { | ||
owner = "qir-alliance"; | ||
repo = "pyqir"; | ||
rev = "v${pyqir-version}"; | ||
sha256 = pyqir-hash; | ||
}; | ||
|
||
cargoDeps = super.rustPlatform.fetchCargoTarball { | ||
inherit src; | ||
name = "pyqir-${pyqir-version}"; | ||
hash = pyqir-cargo-hash; | ||
}; | ||
|
||
buildAndTestSubdir = "pyqir"; | ||
|
||
nativeBuildInputs = with super.rustPlatform; [ cargoSetupHook maturinBuildHook ]; | ||
|
||
buildInputs = [ llvm super.libxml2.dev ]; | ||
|
||
maturinBuildFlags = "-F llvm${llvm-v-major}-${llvm-v-minor}"; | ||
|
||
preConfigure = '' | ||
export LLVM_SYS_${llvm-v-major}${llvm-v-minor}_PREFIX=${llvm.dev} | ||
''; | ||
|
||
pythonImportsCheck = [ "pyqir" ]; | ||
|
||
passthru.llvm = llvm; | ||
}; | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,54 @@ | ||
self: super: | ||
let | ||
metadata = builtins.readFile ../_metadata.py; | ||
versions = | ||
builtins.match ''.*_version__ *= *["']([^"']+)["'].*'' metadata; | ||
version = if builtins.length versions > 0 then | ||
builtins.elemAt versions 0 | ||
else | ||
builtins.trace "Warning: unable to find version. Defaulting to 0.0.0" "0.0.0"; | ||
in { | ||
pytket-qir = super.python3.pkgs.buildPythonPackage { | ||
pname = "pytket-qir"; | ||
version = version; | ||
src = super.stdenv.mkDerivation{ | ||
name = "pytket-qir-sources"; | ||
phases = [ "installPhase" ]; | ||
installPhase = '' | ||
mkdir -p $out; | ||
cp -r ${../pytket} $out/pytket; | ||
cp -r ${../tests} $out/tests; | ||
|
||
cp ${../setup.py} $out/setup.py; | ||
cp ${../README.md} $out/README.md; # required for setup's long description | ||
cp ${../pytest.ini} $out/pytest.ini; | ||
cp ${../_metadata.py} $out/_metadata.py; | ||
|
||
# on nix versions of scipy and ipython, stubs are missing. | ||
# adjust mypy.ini to ignore these errors. | ||
( | ||
cat ${../mypy.ini}; | ||
cat <<EOF | ||
[mypy-scipy.*] | ||
ignore_missing_imports = True | ||
ignore_errors = True | ||
|
||
[mypy-IPython.display.*] | ||
ignore_missing_imports = True | ||
ignore_errors = True | ||
EOF | ||
) >> $out/mypy.ini; | ||
''; | ||
}; | ||
propagatedBuildInputs = [ super.pytket self.pyqir ]; | ||
checkInputs = with super.python3Packages; [ pytest mypy ]; | ||
checkPhase = '' | ||
export HOME=$TMPDIR; | ||
|
||
python -m mypy --config-file=mypy.ini --no-incremental -p pytket -p tests | ||
|
||
cd tests; | ||
python -m pytest -s . | ||
''; | ||
}; | ||
} |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it possible to import the version from the python packages here, or do we now need to update this in two places?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In this case it is customary to provide the exact git revision to pull, then specify the build process, as pyqir isn't available on nixpkgs.
I can update it to scan setup.py for a specific version, though if it changes to using a loose version match then the process will fail.
Though I will add that I am committed to supporting the nix support for this repository and others, so if a change breaks the nix support, or if a version update is required, I'm absolutely happy to action it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I case I am doing the update, is there anything else I need to do besides updating the version in this file as well?