Skip to content

Commit

Permalink
API: configure Cython build via environment variables
Browse files Browse the repository at this point in the history
Previously configuration via `pip` used `--install-option`,
but this parameter is removed in `pip >= 23.1`.

- REL: use `pip`, instead of `setup.py install`,
  which is deprecated in `setuptools == 58.3.0`.

- REL: require `setuptools >= 65.6.0` in `install_requires`,
  to use `pip install . --use-pep517`

- CI: use `pip` to install `dd`

- API: warn about `extern/` license files when running
  `bdist_wheel`, to enable `pip install .` for local builds.
  • Loading branch information
johnyf committed Nov 27, 2023
1 parent 8fc3193 commit d25ee69
Show file tree
Hide file tree
Showing 11 changed files with 299 additions and 172 deletions.
10 changes: 9 additions & 1 deletion .github/workflows/main.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,15 @@ jobs:
set -o posix
echo "Exported environment variables:"
export -p
python setup.py install --fetch --cudd --cudd_zdd --sylvan
export \
DD_FETCH=1 \
DD_CUDD=1 \
DD_CUDD_ZDD=1 \
DD_SYLVAN=1
pip install . \
--verbose \
--use-pep517 \
--no-build-isolation
- name: Install test dependencies
run: |
pip install pytest
Expand Down
3 changes: 2 additions & 1 deletion .github/workflows/setup_build_env.sh
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ sudo apt install \
dot -V
pip install --upgrade \
pip \
setuptools
setuptools \
wheel
# note that installing from `requirements.txt`
# would also install packages that
# may be absent from where `dd` will be installed
Expand Down
80 changes: 48 additions & 32 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -3,74 +3,90 @@

SHELL := bash
wheel_file := $(wildcard dist/*.whl)
temp_file := _temp.txt

.PHONY: cudd install test
.PHONY: clean clean_all clean_cudd
.PHONY: clean clean_all clean_cudd wheel_deps


build_cudd: clean cudd install test

build_sylvan: clean
build_sylvan: clean wheel_deps
-pip uninstall -y dd
python setup.py install --sylvan
export DD_SYLVAN=1; \
pip install . -vvv --use-pep517 --no-build-isolation
pip install pytest
make test

sdist_test: clean
python setup.py sdist --cudd --buddy
cd dist; \
sdist_test: clean wheel_deps
pip install -U build cython
export DD_CUDD=1 DD_BUDDY=1; \
python -m build --sdist --no-isolation
pushd dist; \
pip install dd*.tar.gz; \
tar -zxf dd*.tar.gz
tar -zxf dd*.tar.gz && \
popd
pip install pytest
make -C dist/dd*/ -f ../../Makefile test

sdist_test_cudd: clean
pip install cython ply
python setup.py sdist --cudd --buddy
sdist_test_cudd: clean wheel_deps
pip install build cython ply
export DD_CUDD=1 DD_BUDDY=1; \
python -m build --sdist --no-isolation
yes | pip uninstall cython ply
cd dist; \
pushd dist; \
tar -zxf dd*.tar.gz; \
cd dd*; \
python setup.py install --fetch --cudd
pushd dd*/; \
export DD_FETCH=1 DD_CUDD=1; \
pip install . -vvv --use-pep517 --no-build-isolation && \
popd && popd
pip install pytest
make -C dist/dd*/ -f ../../Makefile test

# use to create source distributions for PyPI
sdist: clean
sdist: clean wheel_deps
-rm dist/*.tar.gz
python setup.py sdist --cudd --buddy --sylvan
pip install -U build cython
export DD_CUDD=1 DD_BUDDY=1 DD_SYLVAN=1; \
python -m build --sdist --no-isolation

wheel_deps:
pip install --upgrade pip setuptools wheel

# use to create binary distributions for PyPI
wheel: clean
wheel: clean wheel_deps
-rm dist/*.whl
-rm wheelhouse/*.whl
python setup.py bdist_wheel --cudd --cudd_zdd
export DD_CUDD=1 DD_CUDD_ZDD=1; \
pip wheel . \
-vvv \
--wheel-dir dist \
--no-deps
@echo "-------------"
auditwheel show dist/*.whl
@echo "-------------"
auditwheel repair --plat manylinux_2_17_x86_64 dist/*.whl
@echo "-------------"
auditwheel show wheelhouse/*.whl

install:
python setup.py install --cudd
install: wheel_deps
export DD_CUDD=1; \
pip install . -vvv --use-pep517 --no-build-isolation

reinstall: uninstall
python setup.py install --cudd --cudd_zdd --sylvan
reinstall: uninstall wheel_deps
export DD_CUDD=1 DD_CUDD_ZDD DD_SYLVAN; \
pip install . -vvv --use-pep517 --no-build-isolation

reinstall_buddy: uninstall
echo ". --install-option='--buddy'" \
> $(temp_file)
pip install -vvv -r $(temp_file)
reinstall_buddy: uninstall wheel_deps
export DD_BUDDY=1; \
pip install . -vvv --use-pep517 --no-build-isolation

reinstall_cudd: uninstall
python setup.py install --cudd --cudd_zdd
reinstall_cudd: uninstall wheel_deps
export DD_CUDD=1 DD_CUDD_ZDD=1; \
pip install . -vvv --use-pep517 --no-build-isolation

reinstall_sylvan: uninstall
echo ". --install-option='--sylvan'" \
> $(temp_file)
pip install -vvv -r $(temp_file)
reinstall_sylvan: uninstall wheel_deps
export DD_SYLVAN=1; \
pip install . -vvv --use-pep517 --no-build-isolation

uninstall:
pip uninstall -y dd
Expand Down
154 changes: 29 additions & 125 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,7 +176,7 @@ package installer [`pip`](https://pip.pypa.io):
pip install dd
```

Locally:
or from the directory of source files:

```shell
pip install .
Expand All @@ -190,103 +190,47 @@ For Python 2.7, use `dd == 0.5.7`.

## Cython bindings


### Wheel files with compiled CUDD


As of `dd` version 0.5.3, [`manylinux2014_x86_64`](
https://www.python.org/dev/peps/pep-0599/)
[wheel files](https://www.python.org/dev/peps/pep-0427/) are
[available from PyPI](https://pypi.org/project/dd/#files) for some Python
versions. These wheel files contain the module `dd.cudd` with the CUDD
library compiled and linked.
If you have a Linux system and Python version compatible with one of the
available wheels, then `pip install dd` will install `dd.cudd`, so you need
not compile CUDD. Otherwise, read below.


### `dd` fetching CUDD

By default, the package installs only the Python modules.
You can select to install any Cython extensions using
the `setup.py` options:

- `--cudd`: build module of CUDD BDDs
- `--cudd_zdd`: build module of CUDD ZDDs
- `--sylvan`: build module of Sylvan BDDs
- `--buddy`: build module of BuDDy BDDs

Pass `--fetch` to `setup.py` to tell it to download, unpack, and
`make` CUDD v3.0.0. For example:

```shell
pip download dd --no-deps
tar xzf dd-*.tar.gz
cd dd-*/
python setup.py install --fetch --cudd --cudd_zdd
```

The path to an existing CUDD build directory can be passed as an argument:
To compile also the module `dd.cudd` (which interfaces to CUDD)
when installing from PyPI, run:

```shell
python setup.py install --cudd="/home/user/cudd"
pip install --upgrade wheel cython
export DD_FETCH=1 DD_CUDD=1
pip install dd -vvv --use-pep517 --no-build-isolation
```

If you prefer defining installation directories, then follow [Cython's instructions](
https://cython.readthedocs.io/en/latest/src/tutorial/clibraries.html#compiling-and-linking)
to define `CFLAGS` and `LDFLAGS` before running `setup.py`.
You need to have copied `CuddInt.h` to the installation's include location
(CUDD omits it).
(`DD_FETCH=1 DD_CUDD=1 pip install dd` also works,
when the source tarball includes cythonized code.)

If building from the repository, then first install `cython`. For example:
To confirm that the installation succeeded:

```shell
git clone [email protected]:tulip-control/dd
cd dd
pip install cython # not needed if building from PyPI distro
python setup.py install --fetch --cudd
```

The above options can be passed to `pip` too, using the [`--install-option`](
https://pip.pypa.io/en/latest/cli/pip_install/#per-requirement-overrides)
in a requirements file, for example:

```
dd >= 0.1.1 --install-option="--fetch" --install-option="--cudd"
python -c 'import dd.cudd'
```

The command line behavior of `pip` [is currently different](
https://github.com/pypa/pip/issues/1883), so

```shell
pip install --install-option="--fetch" dd
```

will propagate option `--fetch` to dependencies, and so raise an error.


### User installing build dependencies
The [environment variables](
https://en.wikipedia.org/wiki/Environment_variable)
above mean:
- `DD_FETCH=1`: download CUDD v3.0.0 sources from the internet,
unpack the tarball (after checking its hash), and `make` CUDD.
- `DD_CUDD=1`: build the Cython module `dd.cudd`

If you build and install CUDD, Sylvan, or BuDDy yourself, then ensure that:
More about environment variables that configure the
C extensions of `dd` is described in the file [`doc.md`](
https://github.com/tulip-control/dd/blob/main/doc.md)

- the header files and libraries are present, and
- suitable compiler, include, linking, and library flags are passed,
either by setting [environment variables](
https://en.wikipedia.org/wiki/Environment_variable)
prior to calling `pip`, or by editing the file [`download.py`](
https://github.com/tulip-control/dd/blob/main/download.py).

Currently, `download.py` expects to find Sylvan under `dd/sylvan` and built with [Autotools](https://en.wikipedia.org/wiki/GNU_Build_System)
(for an example, read `.github/workflows/main.yml`).
If the path differs in your environment, remember to update it.
## Wheel files with compiled CUDD

Scripts are available that fetch, build, and install the Cython bindings:
- [`examples/install_dd_cudd.sh`](
https://github.com/tulip-control/dd/blob/main/examples/install_dd_cudd.sh)
- [`examples/install_dd_sylvan.sh`](
https://github.com/tulip-control/dd/blob/main/examples/install_dd_sylvan.sh)
- [`examples/install_dd_buddy.sh`](
https://github.com/tulip-control/dd/blob/main/examples/install_dd_buddy.sh)
[Wheel files](
https://www.python.org/dev/peps/pep-0427/)
are [available from PyPI](
https://pypi.org/project/dd/#files),
which contain the module `dd.cudd`,
with the CUDD library compiled and linked.
If you have a Linux system and Python version compatible with
one of the PyPI wheels,
then `pip install dd` will install also `dd.cudd`.


### Licensing of the compiled modules `dd.cudd` and `dd.cudd_zdd` in the wheel
Expand Down Expand Up @@ -335,46 +279,6 @@ The modules `dd.cudd` and `dd.cudd_zdd` in the wheel dynamically link to the:
that is dynamically linked.


## Installing the development version

For installing the development version of `dd` from the `git` repository,
an alternative to cloning the repository and installing from the cloned
repository is to [use `pip` for doing so](
https://pip.pypa.io/en/stable/cli/pip_install/#argument-handling):

```shell
pip install https://github.com/tulip-control/dd/archive/main.tar.gz
```

or with [`pip` using `git`](
https://pip.pypa.io/en/stable/topics/vcs-support/#git)
(this alternative requires that `git` be installed):

```shell
pip install git+https://github.com/tulip-control/dd
```

A `git` URL can be passed also to [`pip download`](
https://pip.pypa.io/en/stable/cli/pip_download/#overview),
for example:

```shell
pip download --no-deps https://github.com/tulip-control/dd/archive/main.tar.gz
```

The extension `.zip` too can be used for the name of the [archive file](
https://en.wikipedia.org/wiki/Archive_file)
in the URL. Analogously, with `pip` using `git`:

```shell
pip download --no-deps git+https://github.com/tulip-control/dd
```

Note that the naming of paths *within* the archive file downloaded from
GitHub in this way will differ, depending on whether `https://` or
`git+https://` is used.


Tests
=====

Expand Down
Loading

0 comments on commit d25ee69

Please sign in to comment.