diff --git a/README.md b/README.md index 48802ce..e438f67 100644 --- a/README.md +++ b/README.md @@ -1,16 +1,40 @@ -# Circuit benchmark task +# Qiskit compilation benchmark for Quantum circuits -Execute quantum circuit compilation [benchmark task 25](https://metriq.info/Task/25) and submit performance results for each version of Qiskit to metriq.info. +Execute quantum circuit compilation [benchmark task 25](https://metriq.info/Task/25) and submit performance results for each version of Qiskit to [metriq.info](https://metriq.info/), a community-driven platform for hosting quantum benchmarks. ## Archictecture -Tox environments, with different Qiskit versions installed, execute [circuit benchmark tasks](https://github.com/qiskit-community/submit-metriq/blob/main/scripts/circuit_depth_and_gate_count.py), compiled for the `IBM-Q Rochester` and `Rigetti 16Q Aspen-1` architecture platforms. +This project creates a tox environment for each qiskit-terra version, starting from v0.13.0 to [latest](https://github.com/Qiskit/qiskit/releases). + +The benchmark circuit compilation is batch processed, and the results are submitted to [metriq.info](https://metriq.info/). + +Benchmark tasks for quantum computer compilers: +- ex1_226.qasm benchmark circuit + - [Task 25](https://metriq.info/Task/25) + - [Task 26](https://metriq.info/Task/26) - Aspen architecture + - [Task 27](https://metriq.info/Task/27) - IBMQ Rochester architecture + +## Results +**Circuit depth distribution** + +Compiled for the Rigetti 16Q Aspen architecture +![Circuit depth-Aspen](benchmarking/results/visualizations/boxplot-aspen_depth.png) + +Compiled for the IBMQ Rochester architecture +![Circuit depth-Rochester](benchmarking/results/visualizations/boxplot-rochester_depth.png) + +**Gate count distribution** + +Compiled for the Rigetti 16Q Aspen architecture +![Circuit depth-Aspen](benchmarking/results/visualizations/boxplot-aspen_gates.png) + +Compiled for the IBMQ Rochester architecture +![Circuit depth-Rochester](benchmarking/results/visualizations/boxplot-rochester_gates.png) ## Requirements * [tox](https://pypi.org/project/tox/) * Python 3.8+ - ## Run locally ### To run benchmark task using the current stable version of `qiskit-terra`: ```bash @@ -22,9 +46,3 @@ Versions >=0.13,<=0.15 require numpy<1.20. You can run the tox environments `ter ```bash tox -e py8-terra13 ``` - -### Create historical data using previous versions of `qiskit-terra` since last update to Metriq: - -```bash -python scripts/run_tox_envs.py -``` diff --git a/benchmarking/qiskit_versions.json b/benchmarking/qiskit_versions.json new file mode 100644 index 0000000..63813ab --- /dev/null +++ b/benchmarking/qiskit_versions.json @@ -0,0 +1,67 @@ +[ + { + "version": "0.13.0", + "date": "2020-04-09T21:22:39", + "python_version": ">=3.5" + }, + { + "version": "0.14.2", + "date": "2020-06-15T21:02:04", + "python_version": ">=3.5" + }, + { + "version": "0.15.2", + "date": "2020-09-08T18:01:48", + "python_version": ">=3.5" + }, + { + "version": "0.16.4", + "date": "2021-02-08T17:16:48", + "python_version": ">=3.6" + }, + { + "version": "0.17.4", + "date": "2021-05-18T21:24:27", + "python_version": ">=3.6" + }, + { + "version": "0.18.3", + "date": "2021-09-29T11:57:07", + "python_version": ">=3.6" + }, + { + "version": "0.19.2", + "date": "2022-02-02T14:03:42", + "python_version": ">=3.6" + }, + { + "version": "0.20.2", + "date": "2022-05-18T15:29:49", + "python_version": ">=3.7" + }, + { + "version": "0.21.2", + "date": "2022-08-23T21:07:29", + "python_version": ">=3.7" + }, + { + "version": "0.22.4", + "date": "2023-01-17T13:01:47", + "python_version": ">=3.7" + }, + { + "version": "0.23.3", + "date": "2023-03-21T19:16:16", + "python_version": ">=3.7" + }, + { + "version": "0.24.2", + "date": "2023-07-19T18:23:22", + "python_version": ">=3.7" + }, + { + "version": "0.25.0", + "date": "2023-07-27T22:01:11", + "python_version": ">=3.8" + } +] \ No newline at end of file diff --git a/benchmarking/results/ex1_226-qiskit0.24.2-ibm_rochester.csv b/benchmarking/results/ex1_226-qiskit0.24.2-ibm_rochester.csv new file mode 100644 index 0000000..8266ff4 --- /dev/null +++ b/benchmarking/results/ex1_226-qiskit0.24.2-ibm_rochester.csv @@ -0,0 +1,101 @@ +|Qasm file|Method|Date|Opt level|Platform|Seed|Circuit depth|Gate count +0|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|0|11|15 +1|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|1|11|16 +2|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|2|9|17 +3|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|3|8|12 +4|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|4|11|16 +5|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|5|10|15 +6|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|6|11|15 +7|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|7|13|20 +8|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|8|9|13 +9|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|9|10|17 +10|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|10|11|15 +11|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|11|10|17 +12|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|12|9|13 +13|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|13|9|12 +14|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|14|11|17 +15|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|15|12|22 +16|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|16|11|17 +17|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|17|11|16 +18|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|18|12|20 +19|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|19|10|16 +20|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|20|11|18 +21|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|21|8|12 +22|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|22|9|17 +23|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|23|10|17 +24|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|24|11|18 +25|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|25|11|16 +26|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|26|10|17 +27|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|27|10|16 +28|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|28|13|19 +29|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|29|13|19 +30|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|30|11|17 +31|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|31|11|15 +32|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|32|9|13 +33|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|33|11|16 +34|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|34|10|17 +35|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|35|13|21 +36|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|36|11|16 +37|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|37|10|16 +38|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|38|13|22 +39|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|39|11|17 +40|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|40|11|16 +41|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|41|10|17 +42|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|42|10|17 +43|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|43|9|13 +44|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|44|13|18 +45|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|45|8|13 +46|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|46|13|18 +47|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|47|11|17 +48|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|48|9|17 +49|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|49|11|18 +50|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|50|8|12 +51|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|51|11|15 +52|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|52|8|12 +53|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|53|10|16 +54|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|54|7|13 +55|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|55|10|17 +56|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|56|11|16 +57|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|57|10|16 +58|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|58|9|17 +59|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|59|13|19 +60|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|60|9|13 +61|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|61|11|15 +62|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|62|10|17 +63|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|63|10|17 +64|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|64|10|17 +65|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|65|9|12 +66|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|66|11|15 +67|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|67|10|15 +68|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|68|9|17 +69|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|69|10|17 +70|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|70|8|12 +71|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|71|9|17 +72|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|72|11|16 +73|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|73|10|17 +74|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|74|9|12 +75|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|75|11|17 +76|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|76|11|17 +77|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|77|8|12 +78|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|78|11|16 +79|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|79|11|15 +80|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|80|9|17 +81|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|81|12|21 +82|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|82|10|15 +83|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|83|9|17 +84|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|84|11|17 +85|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|85|13|20 +86|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|86|12|21 +87|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|87|11|16 +88|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|88|13|19 +89|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|89|11|16 +90|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|90|11|16 +91|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|91|9|17 +92|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|92|13|19 +93|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|93|13|22 +94|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|94|13|19 +95|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|95|9|12 +96|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|96|9|17 +97|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|97|9|17 +98|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|98|7|12 +99|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|ibm_rochester|99|11|15 diff --git a/benchmarking/results/ex1_226-qiskit0.24.2-rigetti_16q_aspen.csv b/benchmarking/results/ex1_226-qiskit0.24.2-rigetti_16q_aspen.csv new file mode 100644 index 0000000..aef4c24 --- /dev/null +++ b/benchmarking/results/ex1_226-qiskit0.24.2-rigetti_16q_aspen.csv @@ -0,0 +1,101 @@ +|Qasm file|Method|Date|Opt level|Platform|Seed|Circuit depth|Gate count +0|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|0|8|16 +1|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|1|10|16 +2|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|2|9|13 +3|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|3|10|15 +4|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|4|8|14 +5|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|5|9|13 +6|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|6|8|16 +7|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|7|8|13 +8|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|8|8|12 +9|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|9|10|18 +10|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|10|8|14 +11|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|11|9|14 +12|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|12|8|15 +13|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|13|10|19 +14|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|14|8|14 +15|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|15|10|16 +16|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|16|8|11 +17|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|17|10|18 +18|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|18|8|13 +19|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|19|8|11 +20|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|20|8|11 +21|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|21|10|16 +22|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|22|8|14 +23|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|23|8|11 +24|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|24|10|16 +25|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|25|10|16 +26|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|26|9|17 +27|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|27|8|16 +28|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|28|10|19 +29|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|29|6|9 +30|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|30|10|19 +31|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|31|8|11 +32|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|32|13|19 +33|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|33|13|22 +34|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|34|10|17 +35|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|35|10|16 +36|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|36|8|11 +37|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|37|8|12 +38|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|38|11|20 +39|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|39|10|16 +40|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|40|10|15 +41|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|41|13|20 +42|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|42|9|19 +43|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|43|10|18 +44|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|44|10|17 +45|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|45|8|13 +46|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|46|8|12 +47|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|47|10|16 +48|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|48|8|12 +49|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|49|10|17 +50|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|50|10|16 +51|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|51|10|16 +52|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|52|10|19 +53|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|53|8|16 +54|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|54|10|19 +55|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|55|10|16 +56|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|56|8|14 +57|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|57|10|19 +58|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|58|8|11 +59|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|59|8|15 +60|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|60|10|19 +61|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|61|10|18 +62|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|62|10|17 +63|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|63|10|19 +64|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|64|10|19 +65|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|65|8|11 +66|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|66|11|20 +67|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|67|8|15 +68|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|68|8|11 +69|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|69|10|19 +70|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|70|8|12 +71|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|71|10|16 +72|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|72|8|14 +73|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|73|8|12 +74|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|74|11|16 +75|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|75|10|16 +76|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|76|11|16 +77|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|77|10|16 +78|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|78|8|15 +79|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|79|10|18 +80|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|80|8|15 +81|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|81|8|13 +82|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|82|10|19 +83|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|83|8|14 +84|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|84|8|11 +85|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|85|10|19 +86|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|86|8|14 +87|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|87|8|11 +88|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|88|8|14 +89|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|89|10|16 +90|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|90|8|14 +91|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|91|8|15 +92|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|92|10|18 +93|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|93|10|15 +94|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|94|10|16 +95|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|95|8|11 +96|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|96|10|16 +97|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|97|8|11 +98|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|98|8|15 +99|ex1_226.qasm|Qiskit 0.24.2 compilation|2023-07-19|3|rigetti_16q_aspen|99|8|15 diff --git a/benchmarking/results/ex1_226-qiskit0.25.0-ibm_rochester.csv b/benchmarking/results/ex1_226-qiskit0.25.0-ibm_rochester.csv new file mode 100644 index 0000000..a2e6482 --- /dev/null +++ b/benchmarking/results/ex1_226-qiskit0.25.0-ibm_rochester.csv @@ -0,0 +1,101 @@ +|Qasm file|Method|Date|Opt level|Platform|Seed|Circuit depth|Gate count +0|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|0|11|15 +1|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|1|11|16 +2|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|2|9|17 +3|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|3|8|12 +4|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|4|11|16 +5|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|5|10|15 +6|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|6|11|15 +7|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|7|13|20 +8|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|8|9|13 +9|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|9|10|17 +10|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|10|11|15 +11|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|11|10|17 +12|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|12|9|13 +13|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|13|9|12 +14|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|14|11|17 +15|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|15|12|22 +16|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|16|11|17 +17|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|17|11|16 +18|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|18|12|20 +19|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|19|10|16 +20|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|20|11|18 +21|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|21|8|12 +22|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|22|9|17 +23|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|23|10|17 +24|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|24|11|18 +25|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|25|11|16 +26|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|26|10|17 +27|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|27|10|16 +28|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|28|13|19 +29|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|29|13|19 +30|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|30|11|17 +31|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|31|11|15 +32|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|32|9|13 +33|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|33|11|16 +34|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|34|10|17 +35|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|35|13|21 +36|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|36|11|16 +37|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|37|10|16 +38|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|38|13|22 +39|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|39|11|17 +40|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|40|11|16 +41|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|41|10|17 +42|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|42|10|17 +43|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|43|9|13 +44|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|44|13|18 +45|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|45|8|13 +46|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|46|13|18 +47|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|47|11|17 +48|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|48|9|17 +49|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|49|11|18 +50|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|50|8|12 +51|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|51|11|15 +52|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|52|8|12 +53|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|53|10|16 +54|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|54|7|13 +55|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|55|10|17 +56|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|56|11|16 +57|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|57|10|16 +58|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|58|9|17 +59|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|59|13|19 +60|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|60|9|13 +61|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|61|11|15 +62|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|62|10|17 +63|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|63|10|17 +64|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|64|10|17 +65|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|65|9|12 +66|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|66|11|15 +67|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|67|10|15 +68|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|68|9|17 +69|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|69|10|17 +70|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|70|8|12 +71|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|71|9|17 +72|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|72|11|16 +73|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|73|10|17 +74|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|74|9|12 +75|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|75|11|17 +76|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|76|11|17 +77|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|77|8|12 +78|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|78|11|16 +79|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|79|11|15 +80|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|80|9|17 +81|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|81|12|21 +82|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|82|10|15 +83|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|83|9|17 +84|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|84|11|17 +85|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|85|13|20 +86|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|86|12|21 +87|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|87|11|16 +88|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|88|13|19 +89|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|89|11|16 +90|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|90|11|16 +91|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|91|9|17 +92|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|92|13|19 +93|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|93|13|22 +94|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|94|13|19 +95|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|95|9|12 +96|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|96|9|17 +97|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|97|9|17 +98|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|98|7|12 +99|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|ibm_rochester|99|11|15 diff --git a/benchmarking/results/ex1_226-qiskit0.25.0-rigetti_16q_aspen.csv b/benchmarking/results/ex1_226-qiskit0.25.0-rigetti_16q_aspen.csv new file mode 100644 index 0000000..81eb37b --- /dev/null +++ b/benchmarking/results/ex1_226-qiskit0.25.0-rigetti_16q_aspen.csv @@ -0,0 +1,101 @@ +|Qasm file|Method|Date|Opt level|Platform|Seed|Circuit depth|Gate count +0|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|0|8|16 +1|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|1|10|16 +2|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|2|9|13 +3|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|3|10|15 +4|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|4|8|14 +5|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|5|9|13 +6|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|6|8|16 +7|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|7|8|13 +8|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|8|8|12 +9|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|9|10|18 +10|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|10|8|14 +11|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|11|9|14 +12|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|12|8|15 +13|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|13|10|19 +14|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|14|8|14 +15|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|15|10|16 +16|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|16|8|11 +17|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|17|10|18 +18|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|18|8|13 +19|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|19|8|11 +20|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|20|8|11 +21|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|21|10|16 +22|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|22|8|14 +23|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|23|8|11 +24|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|24|10|16 +25|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|25|10|16 +26|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|26|9|17 +27|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|27|8|16 +28|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|28|10|19 +29|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|29|6|9 +30|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|30|10|19 +31|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|31|8|11 +32|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|32|13|19 +33|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|33|13|22 +34|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|34|10|17 +35|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|35|10|16 +36|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|36|8|11 +37|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|37|8|12 +38|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|38|11|20 +39|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|39|10|16 +40|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|40|10|15 +41|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|41|13|20 +42|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|42|9|19 +43|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|43|10|18 +44|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|44|10|17 +45|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|45|8|13 +46|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|46|8|12 +47|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|47|10|16 +48|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|48|8|12 +49|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|49|10|17 +50|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|50|10|16 +51|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|51|10|16 +52|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|52|10|19 +53|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|53|8|16 +54|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|54|10|19 +55|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|55|10|16 +56|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|56|8|14 +57|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|57|10|19 +58|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|58|8|11 +59|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|59|8|15 +60|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|60|10|19 +61|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|61|10|18 +62|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|62|10|17 +63|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|63|10|19 +64|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|64|10|19 +65|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|65|8|11 +66|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|66|11|20 +67|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|67|8|15 +68|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|68|8|11 +69|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|69|10|19 +70|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|70|8|12 +71|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|71|10|16 +72|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|72|8|14 +73|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|73|8|12 +74|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|74|11|16 +75|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|75|10|16 +76|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|76|11|16 +77|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|77|10|16 +78|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|78|8|15 +79|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|79|10|18 +80|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|80|8|15 +81|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|81|8|13 +82|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|82|10|19 +83|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|83|8|14 +84|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|84|8|11 +85|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|85|10|19 +86|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|86|8|14 +87|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|87|8|11 +88|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|88|8|14 +89|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|89|10|16 +90|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|90|8|14 +91|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|91|8|15 +92|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|92|10|18 +93|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|93|10|15 +94|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|94|10|16 +95|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|95|8|11 +96|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|96|10|16 +97|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|97|8|11 +98|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|98|8|15 +99|ex1_226.qasm|Qiskit 0.25.0 compilation|2023-07-27|3|rigetti_16q_aspen|99|8|15 diff --git a/benchmarking/results/visualizations/boxplot-aspen_depth.png b/benchmarking/results/visualizations/boxplot-aspen_depth.png new file mode 100644 index 0000000..c66565e Binary files /dev/null and b/benchmarking/results/visualizations/boxplot-aspen_depth.png differ diff --git a/benchmarking/results/visualizations/boxplot-aspen_gates.png b/benchmarking/results/visualizations/boxplot-aspen_gates.png new file mode 100644 index 0000000..0479b19 Binary files /dev/null and b/benchmarking/results/visualizations/boxplot-aspen_gates.png differ diff --git a/benchmarking/results/visualizations/boxplot-rochester_depth.png b/benchmarking/results/visualizations/boxplot-rochester_depth.png new file mode 100644 index 0000000..270f8bb Binary files /dev/null and b/benchmarking/results/visualizations/boxplot-rochester_depth.png differ diff --git a/benchmarking/results/visualizations/boxplot-rochester_gates.png b/benchmarking/results/visualizations/boxplot-rochester_gates.png new file mode 100644 index 0000000..5db8a14 Binary files /dev/null and b/benchmarking/results/visualizations/boxplot-rochester_gates.png differ diff --git a/scripts/run_tox_envs.py b/scripts/run_tox_envs.py deleted file mode 100644 index 9219460..0000000 --- a/scripts/run_tox_envs.py +++ /dev/null @@ -1,48 +0,0 @@ -import os -from qiskit_versions import * - -versions_info = get_qiskit_versions_info() - -# Create a tox env for each qiskit version -for info in versions_info: - # TODO Pass python version to tox env setup - # Using 3.8 for now as it is compatible with all qiskit-terra versions - python_version = "3.8" - qiskit_version = info["version"] - env_name = "q_v" + qiskit_version - run_task_command = f"python {{toxinidir}}/scripts/circuit_depth_and_gate_count.py" - numpy_version = "numpy<1.20" - tox_config = f"""[tox] -minversion = {python_version} -envlist = {env_name} -[testenv] -usedevelop = True -deps = - qiskit-terra=={qiskit_version} - %s - requests - pyzx - pandas -commands = - {run_task_command} -""" %(numpy_version if qiskit_version in ["0.13.0", "0.14.2", "0.15.2"] else "") - absolute_path = os.path.abspath(os.path.join(os.path.dirname( __file__ ), "..")) - filename = absolute_path + "/tox.ini" - run_tox_command = "tox -vre " + env_name - - # Create tox config file - with open(filename, 'w') as f: - f.write(tox_config) - - os.system(run_tox_command) - -""" - Ideally, instead of overriding tox.ini content, only override qiskit version using tox CLI. - For some reason it always installs the latest, defined in tox.ini - - # command = "tox -e "+ env_name + " -- pip install --qiskit-terra==" + qiskit_version - # CLI commands tried directly on terminal: - # tox -e q_0.20.0 -- pip install qiskit-terra==0.20.0 - # tox -e q_0.20.0 --force-dep qiskit-terra==0.20.0 - # tox -e q_0.20.0 --force-dep qiskit-terra==0.20.0 -""" \ No newline at end of file diff --git a/scripts/circuit_depth_and_gate_count.py b/src/circuit_depth_and_gate_count.py similarity index 63% rename from scripts/circuit_depth_and_gate_count.py rename to src/circuit_depth_and_gate_count.py index 9875cbc..6089569 100644 --- a/scripts/circuit_depth_and_gate_count.py +++ b/src/circuit_depth_and_gate_count.py @@ -2,7 +2,7 @@ import os import statistics from pyzx import routing -from qiskit import qiskit +import qiskit from qiskit import QuantumCircuit from qiskit import transpile from qiskit.transpiler import CouplingMap, TranspilerError @@ -16,12 +16,10 @@ METHOD = f"Qiskit {VERSION} compilation" def run_task(qasm_id: str): + print(f"\nRunning {METHOD} batch on {qasm_id}\n") + qasm_file_path = os.path.abspath(os.path.join( os.path.dirname( __file__ ),"..", "benchmarking",f"{qasm_id}.qasm")) circuit = QuantumCircuit.from_qasm_file(qasm_file_path) - - print(METHOD) - print("Sample Size: ",SAMPLE_SIZE) - # Transpile for each architecture using pyzx for arch in ARCHITECTURES: architecture = routing.create_architecture(arch) @@ -42,28 +40,6 @@ def run_task(qasm_id: str): output_path = os.path.abspath(os.path.join(os.path.dirname( __file__ ),"..","benchmarking","results",f"{qasm_id}-qiskit{VERSION}-{arch}.csv")) df.to_csv(output_path, sep="|") - # Calc ave + stdev using pandas and send results for metriq submission - # TODO: Move it to a post processing script that reads from generated csv files - # metriq_results1 = { - # "Name":[f"{qasm_id}.qasm circuit benchmark"], - # "Method" :[METHOD], - # "Date":[DATE], - # "Sample Size":[SAMPLE_SIZE], - # "Platform":[arch], - # "Metric Name":["Circuit depth"], - # "Metric Value":[round(df["Circuit depth"].mean())], - # "Notes":[f"Seed:{i}, Version:{VERSION}, Opt Level:{OPTIMIZATION_LEVEL}, Stdev: {round(df['Circuit depth'].std(),3)}"] - # } - # metriq_results2 = { - # "Name":[f"{qasm_id}.qasm circuit benchmark"], - # "Method" :[METHOD], - # "Date":[DATE], - # "Sample Size":[SAMPLE_SIZE], - # "Platform":[arch], - # "Metric Name":["Gate count"], - # "Metric Value":[round(df["Gate count"].mean())], - # "Notes":[f"Seed:{i}, Version:{VERSION}, Opt Level:{OPTIMIZATION_LEVEL}, Stdev: {round(df['Gate count'].std(),3)}"] - # } print(f"{arch}\n", f"- Circuit depth - ave: {round(df['Circuit depth'].mean())} | stdev: {round(df['Circuit depth'].std(),3)}\n", f"- Gate count - ave: {round(df['Gate count'].mean())} | stdev: {round(df['Gate count'].std(),3)}") diff --git a/src/data_visualization_example.py b/src/data_visualization_example.py new file mode 100644 index 0000000..973af6c --- /dev/null +++ b/src/data_visualization_example.py @@ -0,0 +1,77 @@ +import os +import pandas as pd +import seaborn as sns +import matplotlib.pyplot as plt +import glob + +# All results +csv_files = glob.glob(os.path.abspath(os.path.join( os.path.dirname( __file__ ),"..", "benchmarking", "results", "*.csv"))) + +# Filter by architecture +architectures = {"aspen" : "rigetti_16q_aspen", "rochester" : "ibm_rochester"} +aspen_files = [file for file in csv_files if architectures["aspen"] in file] +rochester_files = [file for file in csv_files if architectures["rochester"] in file] + +# Store dataframes by architecture +aspen_dataframes = [] +rochester_dataframes = [] + +# For each architecture, read each file and store its dataframe +for csv_file_path in aspen_files: + df = pd.read_csv(csv_file_path, sep="|") + aspen_dataframes.append(df) + +for csv_file_path in rochester_files: + df = pd.read_csv(csv_file_path, sep="|") + rochester_dataframes.append(df) + +# Concatenate all dataframes by architecture +aspen_combined_df = pd.concat(aspen_dataframes, ignore_index=True) +rochester_combined_df = pd.concat(rochester_dataframes, ignore_index=True) + +# Get data columns for plotting chart +date_col = "Date" +circuit_depth_col = "Circuit depth" +gate_count_col = "Gate count" + +# Sort dataframes by date +aspen_combined_df.sort_values(by=date_col, inplace=True) +rochester_combined_df.sort_values(by=date_col, inplace=True) + +# Enable grid lines +sns.set(style="whitegrid") + +# Create boxplots for aspen architecture +# Circuit depth +plt.figure(figsize=(10,6)) +sns.boxplot(data=aspen_combined_df,x=date_col,y=circuit_depth_col) +plt.title(f"{circuit_depth_col} distribution of Qiskit compilation for {architectures['aspen']}") +plt.xticks(rotation=90) +plt.tight_layout() +plt.show() +# Gate count +plt.figure(figsize=(10,6)) +sns.boxplot(data=aspen_combined_df,x=date_col,y=gate_count_col) +plt.title(f"{gate_count_col} distribution of Qiskit compilation for {architectures['aspen']}") +plt.xticks(rotation=90) +plt.tight_layout() +plt.show() + +# Create boxplots for rochester architecture +# Circuit depth +plt.figure(figsize=(10,6)) +sns.boxplot(data=rochester_combined_df,x=date_col,y=circuit_depth_col) +plt.title(f"{circuit_depth_col} distribution of Qiskit compilation for {architectures['rochester']}") +plt.xticks(rotation=90) +plt.tight_layout() +plt.show() +# Gate count +plt.figure(figsize=(10,6)) +sns.boxplot(data=rochester_combined_df,x=date_col,y=gate_count_col) +plt.title(f"{gate_count_col} distribution of Qiskit compilation for {architectures['rochester']}") +plt.xticks(rotation=90) +plt.tight_layout() +plt.show() + + + diff --git a/src/postprocessing.py b/src/postprocessing.py new file mode 100644 index 0000000..f9adf6c --- /dev/null +++ b/src/postprocessing.py @@ -0,0 +1,106 @@ + +import os +import pandas as pd +from metriq import MetriqClient +from metriq.models.result import ResultCreateRequest +from metriq.models.submission import (Submission, SubmissionCreateRequest) + +METRIQ_TOKEN = os.getenv("METRIQ_TOKEN") +RESULTS_PATH = os.path.abspath(os.path.join( os.path.dirname( __file__ ),"..", "benchmarking", "results")) + +CONTENT_URL = "https://github.com/qiskit-community/submit-metriq" +THUMBNAIL_URL = "https://avatars.githubusercontent.com/u/30696987?s=200&v=4" + +# Metriq API parameters and associated ids +METHOD = {"8": "Qiskit compilation"} +PLATFORMS = {"64": "Rigetti 16Q Aspen-1 ", "69": "ibmq-rochester"} +TAGS = ["quantum circuits", "compiler", "compilation", "ibm qiskit"] +TASKS = {"25": "ex1_226.qasm", "26": "ex1_226.qasm (Aspen)", "27": "ex1_226.qasm (Rochester)"} + +def get_id(param_list: dict, param_name: str) -> str: + # Return key from value + return list(param_list.keys())[list(param_list.values()).index(param_name)] + +def get_platform_id(keywork: str) -> str: + norm_keyword = keywork.lower() + for key, value in PLATFORMS.items(): + norm_value = value.lower() + if norm_keyword in norm_value: + return key + return None + +# TODO Keep track of results already submitted and prepare automation pipeline +def submit_all(task_name: str, submission_id: str = None): + client = MetriqClient(token=METRIQ_TOKEN) + + if not submission_id: + create_new_submission(client,task_name) + + # Populate submission parameters + # Task + task_id = get_id(TASKS, task_name) + client.submission_add_task(submission_id, task_id) + + # Method + method_id = get_id(METHOD, "Qiskit compilation") + client.submission_add_method(submission_id, method_id) + + # Tags + for tag in TAGS: + client.submission_add_tag(submission_id, tag) + + # TODO Find a way to to update params below using the API + # submission.codeUrl + # submission.platform + + # Process results and add them to submission + filenames = os.listdir(RESULTS_PATH) + for filename in filenames: + architectures = ["aspen", "rochester"] + for arch in architectures: + if arch in filename and arch in task_name.lower(): + print(f"*** Processing {arch} results from {filename}") + file_path = os.path.join(RESULTS_PATH, f"{filename}") + df = pd.read_csv(file_path, sep='|') + process_results(df, client, task_id, method_id, submission_id) + +def create_new_submission(client: MetriqClient, task_name: str): + submission_req = SubmissionCreateRequest() + submission_req.name = task_name + submission_req.contentUrl = CONTENT_URL + submission_req.thumbnailUrl = THUMBNAIL_URL + submission_req.description = f"Qiskit compilation for {task_name} benchmark circuit" + client.submission_add(submission_req) + +def process_results(dataframe, client: MetriqClient, task_id: str, method_id: str, submission_id: str): + metrics = ["Circuit depth", "Gate count"] + for metric in metrics: + result_item = ResultCreateRequest() + result_item.task = task_id # Must be id + result_item.method = method_id # Must be id + result_item.metricName = metric + result_item.metricValue = str(round(dataframe[metric].mean())) # Must be a string + result_item.evaluatedAt = dataframe["Date"].iloc[0] + result_item.isHigherBetter = "false" + + platform_name = dataframe["Platform"].iloc[0] # This is not an exact match with metriq platform names + # Get last word and find it in PLATFORMS + platform_keyword = platform_name.rsplit('_', 1)[-1] + platform_id = get_platform_id(platform_keyword) + result_item.platform = platform_id # Must be id + + # TODO: Update sample size + # sample_size = len(dataframe.index) + # result_item.sampleSize = sample_size # ERROR: object has no field "sampleSize" + + # Get extra info and add to notes + metric_std = dataframe[metric].std() + opt_level = dataframe["Opt level"].iloc[0] + version = dataframe["Method"].iloc[0].split(" ")[1] + result_item.notes = f"Stdev: {round(metric_std,3)}, Optimization level:{opt_level}, qiskit-terra version:{version}" + + client.result_add(result_item, submission_id) + +# TODO Get submission ids from client +# submit_all(TASKS["26"], "661") +# submit_all(TASKS["27"], "662") diff --git a/scripts/qiskit_versions.py b/src/qiskit_versions.py similarity index 59% rename from scripts/qiskit_versions.py rename to src/qiskit_versions.py index bb83729..cff1f17 100644 --- a/scripts/qiskit_versions.py +++ b/src/qiskit_versions.py @@ -1,11 +1,18 @@ +import os import requests import json from datetime import datetime +PACKAGE_NAME = "qiskit-terra" +VERSIONS_PATH = os.path.abspath(os.path.join( os.path.dirname( __file__ ),"..", "benchmarking", "qiskit_versions.json")) + def get_qiskit_releases_data() -> dict: - response = requests.get("https://pypi.org/pypi/qiskit-terra/json") - data = response.json() - return data["releases"].items() + response = requests.get(f"https://pypi.org/pypi/{PACKAGE_NAME}/json") + + if response.status_code == 200: + data = response.json() + return data["releases"].items() + return None def get_qiskit_versions_info() -> []: data_items = get_qiskit_releases_data() @@ -19,22 +26,21 @@ def get_qiskit_versions_info() -> []: if "rc" in release: continue - # Filter releases starting from 2020-03 date_str = release_info[0]["upload_time"] date_format = "%Y-%m-%dT%H:%M:%S" date_time_obj = datetime.strptime(date_str, date_format) year = date_time_obj.year month = date_time_obj.month + # Filter releases starting from 2020-03 if (year == 2020 and month < 3) or year < 2020: continue python_version = release_info[0]["requires_python"] # Parse the release string of format "x.y.z" into a list of "x","y","z" - dot_char = "." - major_minor_patch_list = release.split(dot_char) - major_minor = dot_char.join(major_minor_patch_list[:2]) + major_minor_patch_list = release.split(".") + major_minor = ".".join(major_minor_patch_list[:2]) # Get latest patch patch_number = int(major_minor_patch_list[2]) @@ -63,18 +69,11 @@ def get_release_date(input_version:str) -> str: return date_time.split('T', 1)[0] return "Invalid version" -""" -Qiskit Terra versions: -{'version': '0.13.0', 'date': '2020-04-09T21:22:39', 'python_version': '>=3.5'} -{'version': '0.14.2', 'date': '2020-06-15T21:02:04', 'python_version': '>=3.5'} -{'version': '0.15.2', 'date': '2020-09-08T18:01:48', 'python_version': '>=3.5'} -{'version': '0.16.4', 'date': '2021-02-08T17:16:48', 'python_version': '>=3.6'} -{'version': '0.17.4', 'date': '2021-05-18T21:24:27', 'python_version': '>=3.6'} -{'version': '0.18.3', 'date': '2021-09-29T11:57:07', 'python_version': '>=3.6'} -{'version': '0.19.2', 'date': '2022-02-02T14:03:42', 'python_version': '>=3.6'} -{'version': '0.20.2', 'date': '2022-05-18T15:29:49', 'python_version': '>=3.7'} -{'version': '0.21.2', 'date': '2022-08-23T21:07:29', 'python_version': '>=3.7'} -{'version': '0.22.4', 'date': '2023-01-17T13:01:47', 'python_version': '>=3.7'} -{'version': '0.23.3', 'date': '2023-03-21T19:16:16', 'python_version': '>=3.7'} -{'version': '0.24.0', 'date': '2023-05-04T19:14:05', 'python_version': '>=3.7'} -""" +def write_versions_to_file(versions, filename=VERSIONS_PATH): + with open(filename,"w") as file: + json.dump(versions, file, indent=4) + +# Uncomment lines alone to test this file output on its own +# versions_info = get_qiskit_versions_info() +# write_versions_to_file(versions_info) +# print(*versions_info, sep='\n') \ No newline at end of file diff --git a/src/run_tox_envs.py b/src/run_tox_envs.py new file mode 100644 index 0000000..bcdcdb8 --- /dev/null +++ b/src/run_tox_envs.py @@ -0,0 +1,96 @@ +import os +from qiskit_versions import * + +RESULTS_PATH = os.path.abspath(os.path.join(os.path.dirname( __file__ ),"..", "benchmarking", "results")) +METRIQ_CLIENT_URL = "https://github.com/unitaryfund/metriq-client/tarball/development" +EXPERIMENT = "circuit_depth_and_gate_count" +versions_info = get_qiskit_versions_info() + +# Check if qiskit_version already has a result +def search_results(qiskit_version: str): + for filename in os.listdir(RESULTS_PATH): + if qiskit_version in filename: + return True + return False + +# Check for matching major-minor version results to be replaced with latest +def replace_with_latest(qiskit_version: str) -> [str]: + replace_files = [] + for filename in os.listdir(RESULTS_PATH): + if "csv" in filename: + filename_major_minor_patch = filename.split("-")[1].replace("qiskit","").split(".") + input_major_minor_patch = qiskit_version.split(".") + + if (filename_major_minor_patch[0] == input_major_minor_patch[0] + and filename_major_minor_patch[1] == input_major_minor_patch[1] + and filename_major_minor_patch[2] < input_major_minor_patch[2]): + replace_files.append(filename) + return replace_files + +# Create a tox env for each qiskit version +for info in versions_info: + qiskit_version = info["version"] + + # Skip running the experiment if there are alredy results associated with that version + if search_results(qiskit_version): + continue + + replace_results = replace_with_latest(qiskit_version) + if replace_results: + for filename in replace_results: + # Remove from folder + os.remove(os.path.join(RESULTS_PATH,filename)) + # Update json file + write_versions_to_file(versions_info) + + print("\nRun experiment for qiskit-terra version ", qiskit_version,"...\n") + + # TODO Pass python version and qasm file to tox env setup + + # Using 3.8 for now as it is compatible with all qiskit-terra versions + python_version = "3.8" + env_name = "q_v" + qiskit_version + run_experiment_command = f"python {{toxinidir}}/src/{{EXPERIMENT}}.py" + install_metriq_client_command = f"pip install --upgrade {{METRIQ_CLIENT_URL}}" + submit_data_command = f"python {{toxinidir}}/src/postprocessing.py" + numpy_version = "numpy<1.20" + tox_config = f"""[tox] +minversion = {python_version} +envlist = {env_name} +[testenv] +usedevelop = True +deps = + qiskit-terra=={qiskit_version} + %s + pandas + pyzx + requests + tea-client==0.0.7 + tea-console==0.0.6 + typer==0.3.2 +passenv = METRIQ_TOKEN +commands = + {run_experiment_command} + ;{install_metriq_client_command} + ;{submit_data_command} +""" %(numpy_version if qiskit_version in ["0.13.0", "0.14.2", "0.15.2"] else "") + absolute_path = os.path.abspath(os.path.join(os.path.dirname( __file__ ), "..")) + filename = absolute_path + "/tox.ini" + run_tox_command = "tox -vre " + env_name + + #Create tox config file + with open(filename, 'w') as f: + f.write(tox_config) + + os.system(run_tox_command) + +""" + Ideally, instead of overriding tox.ini content, only override qiskit version using tox CLI. + For some reason it always installs the latest, defined in tox.ini + + # command = "tox -e "+ env_name + " -- pip install --qiskit-terra==" + qiskit_version + # CLI commands tried directly on terminal: + # tox -e q_0.20.0 -- pip install qiskit-terra==0.20.0 + # tox -e q_0.20.0 --force-dep qiskit-terra==0.20.0 + # tox -e q_0.20.0 --force-dep qiskit-terra==0.20.0 +""" diff --git a/tox.ini b/tox.ini index ec75586..42beccb 100644 --- a/tox.ini +++ b/tox.ini @@ -1,21 +1,24 @@ [tox] minversion = 3.8 -envlist = py38,{py38}-terra{13,14,15} +envlist = py38,{py38}-terra{13,14,15,25} [testenv] description = run circuit benchmark deps = terra13: qiskit-terra==0.13.0 terra14: qiskit-terra==0.14.2 terra15: qiskit-terra==0.15.2 + terra25: qiskit-terra==0.25.0 terra13,terra14,terra15: numpy<1.20 + terra25: qiskit==0.44.0 qiskit-terra requests pyzx pandas tea-client==0.0.7 - tea-console==0.0.5 + tea-console==0.0.6 typer==0.3.2 +passenv = METRIQ_TOKEN commands = - pip install --upgrade https://github.com/unitaryfund/metriq-client/tarball/master - python {toxinidir}/scripts/circuit_depth_and_gate_count.py - ; python {toxinidir}/scripts/postprocessing.py + python {toxinidir}/src/circuit_depth_and_gate_count.py + pip install --upgrade https://github.com/unitaryfund/metriq-client/tarball/development + python {toxinidir}/src/postprocessing.py \ No newline at end of file