Skip to content
GalSim CI

Fix #1324 and add more regression tests of maxk, stepk, etc. #1210

Sign in to view logs

Fix #1324 and add more regression tests of maxk, stepk, etc.

Fix #1324 and add more regression tests of maxk, stepk, etc. #1210

Workflow file for this run

.github/workflows/ci.yml at 11a7cac
name: GalSim CI
on:
push:
branches:
- main
- runci
- releases/*
pull_request:
branches:
- main
- runci
- releases/*
jobs:
build:
runs-on: ${{ matrix.os }}
strategy:
matrix:
# First all python versions in basic linux
os: [ ubuntu-latest ]
py: [ 3.7, 3.8, 3.9, "3.10", 3.11, 3.12 ]
CC: [ gcc ]
CXX: [ g++ ]
FFTW_DIR: [ "/usr/local/lib" ]
# Add some other particular combinations to test
include:
# Just one in MacOS, since GitHub actions seems to have some kind of
# system problem where seg faults happen randomly on their Mac machines.
# Usually just restarting will fix it, but that's annoying, so minimize
# that by only having one of these to fight with.
- os: macos-latest
py: "3.10"
CC: cc
CXX: c++
FFTW_DIR: "/opt/homebrew/lib/"
# Check one with clang compiler
- os: ubuntu-latest
py: "3.10"
CC: clang
CXX: clang++
FFTW_DIR: "/usr/local/lib/"
# And a pypy system.
# As of 12/1/2022, this is broken for astropy.
# pypy3 used to work, but that's Py 3.6, and GHA no longer supports it.
# Maybe revisit at some point. Or maybe don't care too much about pypy...
#- os: ubuntu-latest
#py: pypy-3.7
#CC: gcc
#CXX: g++
env:
FFTW_DIR: ${{ matrix.FFTW_DIR }}
steps:
- uses: actions/checkout@v4
with:
# Need this for git commands to work right.
# Also for codecov upload apparently.
fetch-depth: 0
- name: Print github context properties
run: |
echo 'event: ${{ github.event_name }}'
echo 'sha: ${{ github.sha }}'
echo 'ref: ${{ github.ref }}'
echo 'head_ref: ${{ github.head_ref }}'
echo 'base_ref: ${{ github.base_ref }}'
echo 'event.before: ${{ github.event.before }}'
echo 'event.after: ${{ github.event.after }}'
- name: Set up Python ${{ matrix.py }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.py }}
- name: Set up ccache
uses: hendrikmuhs/ccache-action@v1
with:
key: ${{ matrix.os }}-${{ matrix.py }}-${{ matrix.CXX }}
- name: Cache pip
uses: actions/cache@v4
with:
path: |
~/.cache/pip
$HOME/des_data
$HOME/Library/Caches/Homebrew
/usr/local/Cellar
key: ${{ runner.os }}-${{ matrix.py }}-pip-${{ hashFiles('requirements.txt') }}
restore-keys: |
${{ runner.os }}-${{ matrix.py }}-pip-
${{ runner.os }}-
- name: Install fftw, etc. on linux
if: matrix.os == 'ubuntu-latest'
run: |
echo ${{ matrix.os }}
sudo -H apt update -y
sudo -H apt install -y libfftw3-dev
ls $FFTW_DIR
# Easier if eigen is installed with apt-get, but on at least one system, check that
# it gets downloaded and installed properly if it isn't installed.
if ${{ matrix.py != 3.7 }}; then sudo -H apt install -y libeigen3-dev; fi
# Need this for the mpeg tests
sudo -H apt install -y ffmpeg
- name: Install fftw, etc. on MacOS
if: matrix.os == 'macos-latest'
# brew sometimes exits with 1 if things are already installed.
# continue-on-error should mean that this still counds as success for this step.
# but it doesn't seem to work right. Still counts as an error, it just moves
# on and finishes everything else before calling the whole job a failure. :(
# So we still need the || true bits below.
continue-on-error: true
run: |
echo ${{ matrix.os }}
# brew update takes forever to run. Hopefully won't need to enable this too often.
# Note: The || true bit here is because brew returns an error code if everything is
# already installed, which is goofy. So || true means these always give
# non-error values. If there really is an error, we should notice later
# when trying to use whatever package failed to install.
#brew update || true
brew install fftw wget || true
brew upgrade wget || true
brew link --overwrite fftw gcc wget || true
brew install eigen || true
brew link --overwrite eigen || true
ls $FFTW_DIR
- name: Download des_data
run: |
ls -l $HOME
if test -d $HOME/des_data; then ls -l $HOME/des_data; fi
if ! test -d $HOME/des_data || ! test -f $HOME/des_data/DECam_00154912_01.fits.fz; then wget --no-check-certificate http://www.sas.upenn.edu/~mjarvis/des_data.tar.gz; tar xfz des_data.tar.gz -C $HOME; fi
ln -s $HOME/des_data examples/des/
- name: Install basic dependencies
run: |
python -m pip install -U pip
# Do these three first to clarify potential conflicts
pip install -U numpy
pip install -U "setuptools<72"
pip install -U wheel
# Standard dependencies
pip install -U -r requirements.txt
# Extra packages needed for testing
pip install -U -r test_requirements.txt
pip install -U coverage
pip install -U matplotlib
- name: Install astroplan
# astroplan isn't yet numpy 2.0 compatible. So don't install that on 3.9+.
# I'm also getting errors with starlink, which I think may be related to
# numpy 2.0.
if: (matrix.py == 3.8) || (matrix.py == 3.9)
run: |
pip install -U astroplan
pip install -U starlink-pyast
- name: Install CPython-only dependencies
if: matrix.py != 'pypy-3.7'
run: |
# The only one left that doesn't seem to work right on pypy is starlink-pyast.
# (This is now above in numpy 1.x versions.)
#pip install -U starlink-pyast
# And now pandas is acting up on pypy.
# cf. https://github.com/pandas-dev/pandas/issues/44253
pip install -U pandas
- name: List all installed packages for reference
run: pip list
- name: Build GalSim
run: |
# The prefix is required for recent MacOS, because of System Integrity Protection.
# It's not necessary on Linux, but is harmless enough.
FFTW_DIR=$FFTW_DIR pip install -vvv .
- name: Check download_cosmos only if it changed. (And only on 1 runner)
if: matrix.py == 3.7 && github.base_ref != ''
run: |
git status
git --no-pager log --graph --pretty=oneline --abbrev-commit | head -50
echo git --no-pager diff origin/${{ github.base_ref }}...${{ github.event.after }} --name-only
git --no-pager diff origin/${{ github.base_ref }}...${{ github.event.after }} --name-only
if git --no-pager diff origin/${{ github.base_ref }}...${{ github.event.after }} --name-only | grep -Fxq 'galsim/download_cosmos.py'; then galsim_download_cosmos -s 23.5 -v1; fi
- name: Run unit tests
run: |
cd tests
coverage run -m pytest -v
pytest -v run_examples.py
cd .. # N.B. This seems to happen automatically if omitted.
# Less confusing to include it explicitly.
- name: Check shared library
if: matrix.py == 3.8
run: |
# On a few systems (arbitrarily py3.8, where we also check mac and clang),
# check the shared library creation and link.
GALSIM_BUILD_SHARED=1 python setup.py install
# These directories/files should now exist.
echo "Look for the shared library:"
test -d build/shared_clib
ls build/shared_clib
ls build/shared_clib/libgalsim.*
# For now, use `python setup.py test` to test linking to the shared library.
# We run some C++ tests there, and we use the shared library for linking.
# Caveats:
# 1. The C++ tests are not very comprehensive, so it won't catch all errors.
# 2. Apparently setup.py test is deprecated. I'm getting warnings when I run
# it that indicate it may go away at some point. So whenever that happens,
# we'll have to revisit this test to find another way to build and run
# the C++ tests.
GALSIM_TEST_PY=0 python setup.py test
- name: Upload coverage to codecov
if: matrix.py != 'pypy-3.7'
run: |
cd tests
pwd -P
ls -la
coverage combine || true # (Not necessary I think, but just in case.)
coverage report
ls -la
#codecov # This didn't work.
# cf. https://community.codecov.io/t/github-not-getting-codecov-report-after-switching-from-travis-to-github-actions-for-ci/
# The solution was to switch to the bash uploader line instead.
bash <(curl -s https://codecov.io/bash)
cd ..
- name: Pre-cache cleanup
continue-on-error: true
run: |
# This was helpful in Travis. Not sure how important it is in GHA.
rm -rfv $HOME/.cache/pip/log
rm -rfv $HOME/.cache/pip/http
if ${{ runner.os == 'macOS' }}; then brew cleanup || true; fi