Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#809 Switch from Boost Python to PyBind11 #950

Merged
merged 118 commits into from
Apr 16, 2018
Merged
Show file tree
Hide file tree
Changes from 103 commits
Commits
Show all changes
118 commits
Select commit Hold shift + click to select a range
83b95d9
First pass at implementing pybind11 wrapping
rmjarvis Dec 30, 2017
be6c282
Remove TMV from places that don't really need it
rmjarvis Dec 31, 2017
c4d8246
Add check for Eigen in SConstruct
rmjarvis Jan 2, 2018
eab89ae
Fix a stray use of installed boost header file
rmjarvis Jan 2, 2018
714ba52
Use Eigen rather than TMV in WCS.cpp, RealGalaxy.cpp
rmjarvis Jan 2, 2018
8162304
Use Thomas algorithm for LookupTable spline solver
rmjarvis Jan 2, 2018
e808425
Use Eigen rather than TMV for Shapelet class
rmjarvis Jan 2, 2018
eced2a1
Use Eigen in PSFCorr rather than TMV
rmjarvis Jan 2, 2018
baf65fa
Make TMV optional in SConstruct
rmjarvis Jan 2, 2018
2674501
Automatically get Eigen directory from eigency if possible
rmjarvis Jan 2, 2018
2948102
Automatically get fftw3 library from pyfftw3 if possible
rmjarvis Jan 3, 2018
d221e96
Don't do tgamma, lgamma if already in scope
rmjarvis Jan 3, 2018
ed4d247
First pass at setup.py
rmjarvis Jan 3, 2018
a04d582
Add requirements file
rmjarvis Jan 3, 2018
471225e
Remove mmgr.cpp file
rmjarvis Jan 3, 2018
10960dc
typo
rmjarvis Jan 3, 2018
ea38cdf
Make meta_data.py file using actual install_dir
rmjarvis Jan 3, 2018
93b7b1f
Switch to using entry_points rather than scripts for the executables
rmjarvis Jan 3, 2018
1205657
Add conda_requirements for those so inclined
rmjarvis Jan 3, 2018
38dd0db
Use a custom function to find fftw3 rather than pyfftw3, which doesn'…
rmjarvis Jan 4, 2018
f5cd94d
Add conda_requirements.txt to use conda for whatever it can do.
rmjarvis Jan 4, 2018
eefc527
gcc is not a pip package.
rmjarvis Jan 4, 2018
d6451ab
Write Version.h file
rmjarvis Jan 4, 2018
e525e2c
Fix share symlink, which wasn't working properly on linux
rmjarvis Jan 4, 2018
0c8e64e
Switch to using package_data for share, since seems to be more reliab…
rmjarvis Jan 4, 2018
e6c644d
Fix gcc compiler warnings
rmjarvis Jan 5, 2018
d8a8c42
Fix making meta_data.py file for pip installation
rmjarvis Jan 5, 2018
d32b077
Fix script_install_dir check for pip installation
rmjarvis Jan 5, 2018
6abd2c2
Merge branch '#809f' into #809-pybind11
rmjarvis Jan 5, 2018
a79281c
Use -O2 to avoid numerical inaccuracies from over optimization
rmjarvis Jan 6, 2018
497ed90
Regularize how test suite imports galsim
rmjarvis Jan 6, 2018
512e3f7
Use find_packages to include package subdirectories
rmjarvis Jan 6, 2018
6ec3c63
Make compatible with pybind11 < v2.2
rmjarvis Jan 6, 2018
7bdcff6
Try to use ccache if possible
rmjarvis Jan 6, 2018
a7dc3b1
Remove unnecessary excludes from manifest
rmjarvis Jan 6, 2018
f1aa716
Avoid clang warning and inaccuracy from over optimizing
rmjarvis Jan 6, 2018
34ffca1
Use mutiple processes for compiling if possible
rmjarvis Jan 6, 2018
6fb64da
Don't die if build_ext doesn't have parallel attribute.
rmjarvis Jan 8, 2018
4a5dcb5
Try adding the copt flags to the extensions
rmjarvis Jan 8, 2018
bf3804c
Use env python now for run_all_tests.
rmjarvis Jan 9, 2018
43b60d4
Merge branch '#809f' into #809-pybind11
rmjarvis Jan 9, 2018
3e4c90e
Make sure both env and pyenv get the USE* defines and the -std= flags
rmjarvis Jan 10, 2018
678bbb2
Fix for when USE_BOOST=true
rmjarvis Jan 10, 2018
deb9442
Fix #if not directive for tgamma, lgamma
rmjarvis Jan 10, 2018
414be68
Fix making executables when not using setup.py
rmjarvis Jan 10, 2018
892baad
Fix Eigen version of CRGCoefficients for nband > nsed
rmjarvis Jan 11, 2018
bcba044
Switch over completely to pybind11 v2.2 syntax
rmjarvis Jan 11, 2018
f6534ac
Clean up the PyBind11Helper macros to remove unnecessary ones and giv…
rmjarvis Jan 11, 2018
f417a0c
Make sure setup.py knows about the pybind11>=2.2 requirement
rmjarvis Jan 11, 2018
163e513
Remove some old deprecated test files
rmjarvis Jan 14, 2018
fb88271
Make python setup.py test run the unit tests
rmjarvis Jan 14, 2018
1c22aeb
Break up the sources into a library and extension
rmjarvis Jan 17, 2018
688a263
Don't require eigency. Just look for Eigen in various places
rmjarvis Feb 10, 2018
c8fea5e
Add C++ unit tests to python setup.py test
rmjarvis Feb 10, 2018
2e4624d
Don't use eigency for Eigen.
rmjarvis Feb 16, 2018
5518d38
Switch order of pybind11 include directories to have user path take p…
rmjarvis Feb 16, 2018
3e64dd5
Rework old INSTALL.md file to be about SCons installation.
rmjarvis Feb 16, 2018
a3ef511
Describe new installation method in INSTALL.md
rmjarvis Feb 16, 2018
acf6c7f
Update CHANGELOG
rmjarvis Feb 16, 2018
94a7e5d
Go back to using eigency for Eigen now that 1.77 is released.
rmjarvis Feb 19, 2018
d053cb0
Don't require pyyaml, pandas or starlink in setup.py. But leave in r…
rmjarvis Feb 19, 2018
d3bca56
Nicer output when lookign for FFTW and Eigen directories
rmjarvis Feb 19, 2018
f30351f
Minor cleanup.
rmjarvis Feb 19, 2018
4a466eb
Cleanup README
rmjarvis Feb 19, 2018
5e3ff5d
Add setuptools>=38 as a build_dep.
rmjarvis Feb 19, 2018
089214d
Don't add '' to library_dirs
rmjarvis Feb 20, 2018
af84529
Fix ThreadPool usage
rmjarvis Feb 20, 2018
e0cb9e0
Check for the pybind11 header files
rmjarvis Feb 20, 2018
97a81c6
Make it easier to change the number of jobs for pytest
rmjarvis Feb 20, 2018
7340089
Rework credo.txt into a developer's README
rmjarvis Feb 20, 2018
31b5871
Update instructions about best way to install prereqs
rmjarvis Feb 20, 2018
c7e55a4
Describe setup.py installation in demo1.yaml
rmjarvis Feb 20, 2018
703a488
Merge branch 'noboost' into #809-pybind11
rmjarvis Feb 26, 2018
655b2e6
Ignore .pytest_cache
rmjarvis Feb 26, 2018
80865e9
First try at setup.py travis runs
rmjarvis Feb 26, 2018
37381c4
Use ccache on Travis. And suggest usage in devel/README
rmjarvis Feb 27, 2018
23cead6
Use apt-get for eigen
rmjarvis Feb 26, 2018
1dfec5b
Report include directory for fftw3
rmjarvis Feb 27, 2018
9bd307b
Try a different path finder for fftw3 on unix
rmjarvis Feb 27, 2018
72d1d49
Make sure galsim comes before fftw3 in libraries list
rmjarvis Feb 27, 2018
f6534d7
List pip installed versions.
rmjarvis Feb 26, 2018
a7b98d9
Mention apt-get option in INSTALL.md
rmjarvis Feb 27, 2018
af18562
Fix error in libraries for test
rmjarvis Feb 27, 2018
49baef4
Name this version 2.0.0-alpha
rmjarvis Feb 27, 2018
587b408
Handle semantic versions with words after the release number
rmjarvis Feb 27, 2018
9aa5603
Max of 4 jobs for test by default
rmjarvis Feb 27, 2018
1c8b438
Try the user's home directory for possible fftw3 installation dir
rmjarvis Feb 27, 2018
6841502
Add enable-shared to fftw3 instructions
rmjarvis Feb 27, 2018
bf3851e
Add cython if we need to use eigency
rmjarvis Feb 27, 2018
39bce2e
Write -jN, not -j1
rmjarvis Feb 27, 2018
8bdbb77
Remove workaround for Python 2.6
rmjarvis Feb 27, 2018
cccc917
Don't try to cover the files for the executables, since nosetests doe…
rmjarvis Feb 27, 2018
125d82f
Fix version_info in SCons, which I messed up.
rmjarvis Feb 27, 2018
af4f977
Add * in front of omitted files
rmjarvis Feb 27, 2018
4414f46
Add matplotlib as test dependency for starlink
rmjarvis Feb 27, 2018
1f14359
Add nose as test_dep, so assert_raises, etc. works.
rmjarvis Feb 27, 2018
13b3dc7
Make sure candidate library exists before trying to load it.
rmjarvis Feb 28, 2018
f4705ad
Check EIGEN_DIR/include
rmjarvis Feb 28, 2018
06e0c09
Check if clang needs the -stdlib=libc++ flag
rmjarvis Feb 28, 2018
ed2b677
Fix -j flag to work even with older distutils
rmjarvis Feb 28, 2018
990b643
Fix -jN option to control number of jobs for compiling
rmjarvis Feb 28, 2018
9563445
Prefer libc++ for clang compiling and linking
jmeyers314 Mar 1, 2018
287ae0f
Merge branch 'noboost' into #809-pybind11
rmjarvis Mar 7, 2018
b783237
typos
jmeyers314 Mar 14, 2018
d95730f
Fix boost version of PYBIND11_MODULE definition
rmjarvis Mar 15, 2018
7f62305
Apply Josh's suggested changes in INSTALL files
rmjarvis Mar 15, 2018
0663061
Fix links
rmjarvis Mar 15, 2018
0f6d236
Try to fix aside spec
rmjarvis Mar 15, 2018
1600998
Fix indenting of toc
rmjarvis Mar 15, 2018
6ec0fa0
Try .note rather than aside
rmjarvis Mar 15, 2018
2b94689
I guess .note isn't supported. Remove it.
rmjarvis Mar 15, 2018
d56431b
Remove some files that got accidentally added in the merge
rmjarvis Mar 22, 2018
14018ac
Merge branch 'noboost' into #809-pybind11
rmjarvis Mar 22, 2018
375dd3c
Update codecov requirements to be a bit more strict
rmjarvis Mar 23, 2018
223a400
Merge branch 'noboost' into #809-pybind11
rmjarvis Apr 13, 2018
d1fac7f
Merge branch 'noboost' into #809-pybind11
rmjarvis Apr 13, 2018
9867ce2
Give better progress info about pybind11 header dirs
rmjarvis Apr 16, 2018
0b271ce
Limit to pip version 9.0, not 10.0
rmjarvis Apr 16, 2018
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,10 +15,14 @@ docs/doxygen_example_output/*
tests/nosetests.xml
examples/output/*
devutils/sizeof_SIFD
gs.error
gs_error.txt
*junk*
debug.out
tmp*
examples_bin
*~
.cache
*.egg*
build
dist
.pytest_cache
86 changes: 26 additions & 60 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,97 +5,63 @@ branches:

language: python
python:
#- 2.6 # disabled until Travis fixes their issue #6732
# https://github.com/travis-ci/travis-ci/issues/6732
- 2.7
- 3.4
- 3.5
#- 3.6 # disabled until Travis fixes their issue #4990
# https://github.com/travis-ci/travis-ci/issues/4990
# i.e. has python 3.6 pre-installed. Otherwise it can time out from having to install
# all the python 3.6 stuff from scratch.
- 3.6

compiler:
- g++

before_install:
- export PATH=$(echo $PATH | tr ':' "\n" | sed '/\/opt\/python/d' | tr "\n" ":" | sed "s|::|:|g")
- sudo apt-get -qq update
- sudo apt-get install -y python-dev libfftw3-dev scons libblas-dev liblapack-dev gfortran libav-tools
- sudo apt-get install -y python-dev libfftw3-dev libav-tools libeigen3-dev

# List current contents of directories that should be being cached.
- ls -l $HOME
- if test -d $HOME/tmv-0.73; then ls -l $HOME/tmv-0.73; fi
- if test -d $HOME/boost_1_61_0; then ls -l $HOME/boost_1_61_0; fi
- if test -d $HOME/des_data; then ls -l $HOME/des_data; fi

# Add ~/bin and ~/lib, etc. to the appropriate paths where scons install will put things.
- export PYHOME=$HOME/virtualenv/python${TRAVIS_PYTHON_VERSION}
- export PATH=$HOME/bin:$PATH
- export LD_LIBRARY_PATH=$HOME/lib:$LD_LIBRARY_PATH

# Fix a directory name in 3.x installations so boost can find it.
- if test -d $PYHOME/include/python${TRAVIS_PYTHON_VERSION}m; then ln -s $PYHOME/include/python${TRAVIS_PYTHON_VERSION}m $PYHOME/include/python${TRAVIS_PYTHON_VERSION}; fi

# To get coverage of the WcsToolsWCS class:
#- sudo add-apt-repository "deb http://archive.ubuntu.com/ubuntu $(lsb_release -sc) universe"
#- sudo apt-get -qq update
#- sudo apt-get install -y wcstools
# Hm. This didn't work, and I can't figure out why. I get the following error:
# Reading package lists... Done
# Building dependency tree
# Reading state information... Done
# E: Unable to locate package wcstools
# Perhaps someone with more familiarity with apt-get can figure this out, but for now, we'll
# live with lack of coverage of WcsToolsWCS.

# Only get TMV if not cached
- pushd $HOME
- if ! test -d tmv-0.73 || ! test -f tmv-0.73/SConstruct; then wget https://github.com/rmjarvis/tmv/archive/v0.73.tar.gz && tar -xf v0.73.tar.gz ; else echo Using cached TMV; fi
# But always install it to /usr/local
- cd tmv-0.73 && sudo scons install
- popd

# Only get Boost if not cached
- pushd $HOME
- if ! test -d boost_1_61_0 || ! test -f boost_1_61_0/bootstrap.sh; then wget https://sourceforge.net/projects/boost/files/boost/1.61.0/boost_1_61_0.tar.bz2 --no-check-certificate && tar --bzip2 -xf boost_1_61_0.tar.bz2 && cd boost_1_61_0 && ./bootstrap.sh --with-python=python$TRAVIS_PYTHON_VERSION && ./b2 link=shared && cd ..; else echo Using cached Boost; fi
- cd boost_1_61_0 && sudo ./b2 -d0 link=shared install
- popd

# Get the des data needed for the check_des test.
- if ! test -d $HOME/des_data || ! test -f $HOME/des_data/DECam_00154912_01.fits.fz; then wget http://www.sas.upenn.edu/~mjarvis/des_data.tar.gz && tar xfz des_data.tar.gz -C $HOME --wildcards *_01*; fi
- ln -s $HOME/des_data examples/des/

cache:
ccache: true
pip: true
directories:
- $HOME/tmv-0.73
- $HOME/boost_1_61_0
- $HOME/des_data

install:
# Travis doesn't always have the most up-to-date nupy already installed, so use -U
- travis_wait 30 pip install -U numpy
# Install the requirements
# Use -U to make sure we get the latest versions of everything so we notice any
# incompatibilities as soon as possible.
- pip install -U -r requirements.txt

# Also some things just required for tests
# (This includes scipy, which can take a while to install. So tell Travis to be patient.)
- travis_wait 30 pip install -U -r test_requirements.txt

# Note: matplotlib is only required because starlink has an `import matplotlib` in their
# code, despite that not being a dependency.
# Allow 30 minutes for this one, since sometimes scipy in particular can take a long time
# if it's not in the cache yet.
- travis_wait 30 pip install numpy astropy future lsstdesc.coord pyyaml starlink-pyast nose codecov coveralls matplotlib==1.5.0 scipy pandas coverage
- if [[ $TRAVIS_PYTHON_VERSION == 2.6 ]]; then pip install simplejson ordereddict; fi
- pip install matplotlib

# Finally, a few things for the code coverage
- pip install nose codecov coveralls coverage

- pip list

script:
# This lets scons work even on Python 3 builds
# cf. https://github.com/travis-ci/travis-ci/issues/5961
- source $HOME/virtualenv/python2.7/bin/activate
# But now we need to manually set the python, since it's not the default in this virtualenv.
- scons PREFIX=$HOME PYTHON=$PYHOME/bin/python PYPREFIX=$PYHOME/lib/python${TRAVIS_PYTHON_VERSION}/site-packages BOOST_DIR=$PYHOME && scons install
- if test -f gs.error; then cat gs.error; fi
# Go back to the regular python environment for the tests
- source $PYHOME/bin/activate
# Install GalSim
- python setup.py install

# If galsim_download_cosmos.py changed, then run it.
- if git --no-pager diff $TRAVIS_COMMIT_RANGE --name-only | grep -Fxq 'bin/galsim_download_cosmos.py'; then galsim_download_cosmos -s 23.5 -v1; fi
- echo $TRAVIS_COMMIT_RANGE
- if git --no-pager diff $TRAVIS_COMMIT_RANGE --name-only | grep -Fxq 'galsim/download_cosmos.py'; then galsim_download_cosmos -s 23.5 -v1; fi

- cd tests
# Use this rather than scons tests, so we can get the coverage options.

# Use this rather than setup.py test, so we can get the coverage options.
- "nosetests test*.py --with-coverage --cover-package=galsim --with-doctest --cover-erase"
# Without cover-erase, this will append to the .coverage file
- "nosetests run_examples.py --with-coverage --cover-package=galsim --with-doctest"
Expand Down
10 changes: 8 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,12 @@ Dependency Changes
GalSim as the Angle and CelestialCoord classes. We moved it to a separate
repo so people could more easily use this functionality without requiring all
of GalSim as a dependency. (#809b)
- Removed dependency on boost.
- Added dependency on (pybind11 or cffi...)
- Removed dependency on boost. (#809)
- Removed dependency on TMV. (#809)
- Added dependency on pybind11. (#809)
- Added dependency on Eigen. (#809)
- FFTW is now the only dependency that pip cannot handle automatically. (#809)
- Officially no longer support Python 2.6. (Pretty sure no one cares.)


API Changes
Expand Down Expand Up @@ -43,6 +47,8 @@ API Changes
InclinedSersic has been changed to disk_half_light_radius, since it does
not really correspond to the realized half-light radius of the inclined
profile (unless the inclination angle is 0 degrees). (#809f)
- Removed galsim_yaml and galsim_json scripts, which were essentially just
aliases for galsim -f yaml and galsim -f json respectively. (#809f)


Bug Fixes
Expand Down
Loading