numpy 2.0 fixes

[braingram]

The changes in this PR partially address: #737

numpy 2.0 changes the string format of scalars this format was relied on in gridded_library which this PR updates to first convert the values to builtin floats prior to converting to strings.

numpy 2.0 also removes float_ and complex_. This PR replaces their usage with double and cdouble. See: numpy/numpy#24376 for the numpy changes and the docs for a description of how float_ is an alias of double and complex_ an alias of cdouble: https://numpy.org/doc/stable/reference/arrays.scalars.html#numpy.float_

I attempted to update the CI to use numpy 2.0 and failed. I ran into many issues with versions conflicts that could be boiled down to contourpy (used by matplotlib) setting an upper pin on numpy 2.0. I was able to get things working locally (although this required building scipy from source as the nightly builds for m1 macs are very out-of-date). With the changes in this PR all tests pass except for 3:

FAILED webbpsf/tests/test_nircam.py::test_nircam_blc_wedge_0 - AttributeError: Module 'scipy' has no attribute 'poly1d'
FAILED webbpsf/tests/test_nircam.py::test_nircam_blc_wedge_45 - AttributeError: Module 'scipy' has no attribute 'poly1d'
FAILED webbpsf/tests/test_webbpsf.py::test_return_intermediates - AttributeError: Module 'scipy' has no attribute 'poly1d'

These are due to poly1d usage in poppy addressed here: spacetelescope/poppy#585

[codecov]

Codecov Report

Attention: 1 lines in your changes are missing coverage. Please review.

Files	Coverage Δ
webbpsf/gridded_library.py	76.92% <100.00%> (ø)
webbpsf/utils.py	57.93% <ø> (ø)
webbpsf/optics.py	60.66% <50.00%> (ø)

📢 Thoughts on this report? Let us know!.

[BradleySappington]

@braingram - looks good. Will this affect issue #710?

[braingram]

Thanks! I believe this should (eventually) fix #710

I just noticed 2 uses of scipy.poly1d in webbpsf. Mind if I include those changes (switching them to numpy.poly1d) in this PR?

[BradleySappington]

Thanks! I believe this should (eventually) fix #710

I just noticed 2 uses of scipy.poly1d in webbpsf. Mind if I include those changes (switching them to numpy.poly1d) in this PR?

Go for it

[braingram]

@BradleySappington Thanks!

I pushed b87ae31 updating the usage of poly1d.

EDIT: I was originally seeing errors testing this with the source branch for spacetelescope/poppy#585 The issue was that I had not pushed the git tags from the upstream spacetelescope fork so when I installed my branch it was reporting as an old version of poppy. This caused issues in webbpsf as it appears this package handles different poppy versions differently as seen here:

webbpsf/webbpsf/opds.py

Lines 1167 to 1171 in ca1fdf6

	if Version(poppy.__version__) < Version('1.0'):
	# For earlier poppy versions, fix IFM sign convention for consistency to WSS
	cnames = self._influence_fns.colnames
	for icol in cnames[3:]:
	self._influence_fns[icol] *= -1

The CI should test this with current poppy develop which didn't show the failures mentioned below (when I tested it locally).

I tested this locally using poppy installed from the source branch for spacetelescope/poppy#585 and 2 tests failed:

FAILED webbpsf/tests/test_opds.py::test_segment_tilt_signs - AssertionError: Expected A1:  +X rotation -> -Y pixels (DMS coords)
FAILED webbpsf/tests/test_opds.py::test_segment_tilt_signs_2048npix - AssertionError: Expected A1:  +X rotation -> -Y pixels (DMS coords)

I'm currently at a loss to describe how the single change in poppy (using poly1d directly from numpy instead of scipy) would lead to these failures. Does anything come to mind?

full traceback (click here)

============================= test session starts ============================== platform darwin -- Python 3.10.6, pytest-7.4.2, pluggy-1.3.0

Running tests with Webbpsf version 0.1.dev2196+g67177a7.d20230927.
Running tests in webbpsf/tests.

Date: 2023-09-27T16:13:10

Platform: macOS-12.6-arm64-arm-64bit

Executable: /Users/bgraham/.pyenv/versions/3.10.6/envs/webbpsf/bin/python3.10

Full Python Version:
3.10.6 (main, Oct 3 2022, 15:35:33) [Clang 14.0.0 (clang-1400.0.29.102)]

encodings: sys: utf-8, locale: UTF-8, filesystem: utf-8
byteorder: little
float info: dig: 15, mant_dig: 15

Package versions:
Numpy: 2.0.0.dev0+git20230926.5c46c11
Scipy: 1.12.0.dev0+1778.bf7256b
Matplotlib: 3.9.0.dev301+gd60de5ed4d
Astropy: 6.0.dev728+g0c73d13f6
Photutils: 1.9.1.dev79+g80364656
Poppy: 0.1.dev2100+g15c5893

Using Astropy options: remote_data: none.

rootdir: /Users/bgraham/projects/src/webbpsf
configfile: pyproject.toml
testpaths: webbpsf/tests
plugins: doctestplus-1.0.0, hypothesis-6.87.0, astropy-0.11.0, remotedata-0.4.1, asdf-2.15.1, cov-4.1.0, filter-subpackage-0.1.2, mock-3.11.1, astropy-header-0.2.2, arraydiff-0.5.0
collected 108 items / 106 deselected / 2 selected

webbpsf/tests/test_opds.py FF [100%]

=================================== FAILURES ===================================
___________________________ test_segment_tilt_signs ____________________________

fov_pix = 50, plot = False, npix = 1024

def test_segment_tilt_signs(fov_pix = 50, plot=False, npix=1024):
    """Test that segments move in the direction expected when tilted.

    The local coordinate systems are non-obvious, to say the least. This verifies
    sign conventions and coordinates are consistent in the linear optical model and
    optical propagation code.

    """

    if plot:
        fig, axs = plt.subplots(3, 5, figsize=(14,9))#, sharex = True, sharey = True)

    nrc = webbpsf.NIRCam()

    ote = webbpsf.opds.OTE_Linear_Model_WSS(npix=npix)
    nrc.include_si_wfe = False # not relevant for this test

    tilt = 1.0

    # We aim for relatively minimalist PSF calcs, to reduce test runtime
    psf_kwargs = {'monochromatic': 2e-6,
                  'fov_pixels': fov_pix,
                  'oversample': 1,
                  'add_distortion': False}

    # Which way are things expected to move?
    #
    # A1:  +X rotation -> -Y pixels (DMS), +Y rotation -> -X pixels
    # B1: +X rotation -> +Y pixels, +Y rotation -> +X pixels
    # C1: +X rotation -> +X/+Y pixels, +Y rotation -> -Y/+X pixels
    # (for C1, A/B means A is the sqrt(3)/2 component, B is the 1/2 component)
    #
    # The above derived from Code V models by R. Telfer, subsequently cross checked by Perrin

    for i, iseg in enumerate(['A1', 'B1', 'C1']):
        ote.zero()

        pupil = webbpsf.webbpsf_core.one_segment_pupil(iseg, npix=npix)

        ote.amplitude = pupil[0].data
        nrc.pupil = ote

        # CENTERED PSF:
        psf = nrc.calc_psf(**psf_kwargs)
        cen_ref = webbpsf.measure_centroid(psf, boxsize=10, threshold=1)

        ote.move_seg_local(iseg, xtilt=tilt)
        # XTILT PSF:
        psfx = nrc.calc_psf(**psf_kwargs)
        cen_xtilt = webbpsf.measure_centroid(psfx, boxsize=10, threshold=1)

        if iseg.startswith("A"):

          assert cen_xtilt[0] < cen_ref[0], "Expected A1:  +X rotation -> -Y pixels (DMS coords)"

E AssertionError: Expected A1: +X rotation -> -Y pixels (DMS coords)
E assert np.float64(37.740464181168946) < np.float64(24.499406806161154)

webbpsf/tests/test_opds.py:481: AssertionError
_______________________ test_segment_tilt_signs_2048npix _______________________

def test_segment_tilt_signs_2048npix():
    """ Re-run same test as above, but with a different value for npix

    This verifies the LOM works as expected for a size other than 1024 pixels
    """

  test_segment_tilt_signs(npix=2048)

webbpsf/tests/test_opds.py:534:

---

fov_pix = 50, plot = False, npix = 2048

def test_segment_tilt_signs(fov_pix = 50, plot=False, npix=1024):
    """Test that segments move in the direction expected when tilted.

    The local coordinate systems are non-obvious, to say the least. This verifies
    sign conventions and coordinates are consistent in the linear optical model and
    optical propagation code.

    """

    if plot:
        fig, axs = plt.subplots(3, 5, figsize=(14,9))#, sharex = True, sharey = True)

    nrc = webbpsf.NIRCam()

    ote = webbpsf.opds.OTE_Linear_Model_WSS(npix=npix)
    nrc.include_si_wfe = False # not relevant for this test

    tilt = 1.0

    # We aim for relatively minimalist PSF calcs, to reduce test runtime
    psf_kwargs = {'monochromatic': 2e-6,
                  'fov_pixels': fov_pix,
                  'oversample': 1,
                  'add_distortion': False}

    # Which way are things expected to move?
    #
    # A1:  +X rotation -> -Y pixels (DMS), +Y rotation -> -X pixels
    # B1: +X rotation -> +Y pixels, +Y rotation -> +X pixels
    # C1: +X rotation -> +X/+Y pixels, +Y rotation -> -Y/+X pixels
    # (for C1, A/B means A is the sqrt(3)/2 component, B is the 1/2 component)
    #
    # The above derived from Code V models by R. Telfer, subsequently cross checked by Perrin

    for i, iseg in enumerate(['A1', 'B1', 'C1']):
        ote.zero()

        pupil = webbpsf.webbpsf_core.one_segment_pupil(iseg, npix=npix)

        ote.amplitude = pupil[0].data
        nrc.pupil = ote

        # CENTERED PSF:
        psf = nrc.calc_psf(**psf_kwargs)
        cen_ref = webbpsf.measure_centroid(psf, boxsize=10, threshold=1)

        ote.move_seg_local(iseg, xtilt=tilt)
        # XTILT PSF:
        psfx = nrc.calc_psf(**psf_kwargs)
        cen_xtilt = webbpsf.measure_centroid(psfx, boxsize=10, threshold=1)

        if iseg.startswith("A"):

          assert cen_xtilt[0] < cen_ref[0], "Expected A1:  +X rotation -> -Y pixels (DMS coords)"

E AssertionError: Expected A1: +X rotation -> -Y pixels (DMS coords)
E assert np.float64(37.712403907317764) < np.float64(24.49939316768443)

webbpsf/tests/test_opds.py:481: AssertionError

[braingram]

Running the tests locally with numpy 2.0 and poppy installed from spacetelescope/poppy#585 I now see no errors or warnings.

Fixes: #710
and combined with spacetelescope/poppy#585 this should allow the romancal devdeps tests to run with numpy 2.0.