ENH: Update CI and scikit-build-core

[cookpa]

No description provided.

[coveralls]

coverage: 80.836% (+0.3%) from 80.524%
when pulling e05d2e4 on nano_wheels
into 0903c42 on master.

[cookpa]

It looks like we need to set the MACOSX_DEPLOYMENT_TARGET to the runner version, because brew is used to get libpng, and brew always targets the current OS.

[cookpa]

Success - I think Python 3.12 might still fail but most versions should work now

[cookpa]

Here's the current state of things

Python 3.12 fails on all platforms with some compile errors.

Python 3.8-3.11 works on Windows (there's a test failure on one but these are known to happen randomly)

Python 3.8-3.11 works on Linux

Python 3.9-3.11 works on Mac OS (x86). I think Python 3.8 fails because Python 3.8 can't handle a deployment target of 12.0.

Python 3.10-3.11 works on Mac OS (arm64).

The MACOSX_DEPLOYMENT_TARGET has to be set to the runner OS, because homebrew uses that as its target. I don't think this can be fixed unless we avoid brew and install libpng by some other (probably slower) method.

[ncullen93]

Ok the python3.12 error looks to be due to itkPyBuffer.h which I manually copied from itk a long time ago... it could be outdated so I will try to update it or just include the file header from the actual itk build. I think I couldn't find the flag to have the itk numpy bridge code be included in the build a long time ago.

edit: I believe this is an issue with itk as they dont yet have wheels for 3.12. See InsightSoftwareConsortium/ITK#4590

[cookpa]

Awesome that would be great to fix.

[cookpa]

The Mac wheels build but won't install on my machine. This is because the tag is wrong, it comes out as cp311-abi3-macosx_12_0_x86_64 but pip wants cp311-abi3-macosx_10_12_x86_64. Looking into a fix

[cookpa]

The suggested fix of SYSTEM_VERSION_COMPAT=0 didn't change anything

[cookpa]

Actually

SYSTEM_VERSION_COMPAT=0 pip install <wheel> 

works for me, it doesn't help to build the right tags, but it tells my local pip to let me install. A workaround at least.

[cookpa]

So I guess this is a known problem, and Intel Macs running Mac OS 12 won't install packages built for target mac os 12

pypa/pip#10670

[ncullen93]

How do you download the wheels? I'd also like to try that.
edit: never mind, I figured it out via the github api. The 3.11 mac wheel installs and runs for me, so that's good.

e.g.

curl -L \
  -H "Accept: application/vnd.github+json" \
  -H "Authorization: Bearer <token>" \
  -H "X-GitHub-Api-Version: 2022-11-28" \
  https://api.github.com/repos/ANTsX/ANTsPy/actions/artifacts/1503134086/zip --output test.zip

[ncullen93]

Ok, I think I fixed the python3.12 issue, or at least got it to build locally on my mac. I replaced any instances of PySequence_FAST_GET_ITEM with PySequence_GetItem in itkPyBuffer.hxx and itkPyVnl.hxx.

Ideally, we could have an if statement in those files checking the python version and then only redefine PySequence_FAST_GET_ITEM to PySequence_GetItem if the python version is 3.12 or above.

Unfortunately, the idea of including the ITKBridgeNumPy module in the ITK build did not work because they use an even more outdated version that SWIG - their (bad) wrapper - requires or something.

EDIT: now all the 3.12 wheels build but all the < 3.11 builds for windows fail... so I am trying to conditionally redefine that function with the stable api version.

[ncullen93]

Sorry I think the final solution is here: https://github.com/ANTsX/ANTsPy/pull/632/files#diff-1e7de1ae2d059d21e1dd75d5812d5a34b0222cef273b7c3a2af62eb747f9d20a

Basically, remove the STABLE_ABI statement as this allows a smaller subset of the python c library on 3.12+. Now all the wheels except for 3.8 macos seemingly build correctly.

Here is the build https://github.com/ANTsX/ANTsPy/actions/runs/9094885938

[cookpa]

Great, thanks. I think 3.8 is a lost cause, I have seen many issues on other repos.

[ncullen93]

Ok! If you can just make that small change to your PR then I think it's good to merge.

And I think it's good to remove these lines from the pyproject.toml, as I believe it could be messing up the tags:

# Build stable ABI wheels for CPython 3.12+
wheel.py-api = "cp311"

[ncullen93]

I see the 3.8 windows build failed with this same weird error I was getting earlier:

  Windows fatal exception: access violation
  
  Windows fatal exception: Thread 0xaccess violation000008d4

Could that be related to the different ptrstr function for windows that is not currently used? I don't know how to conditionally include functions like that.. See

ANTsPy/src/antsImageWin.cxx

Line 52 in 9266f2f

std::string ptrstr(void * c)

[cookpa]

How do you download the wheels? I'd also like to try that. edit: never mind, I figured it out via the github api. The 3.11 mac wheel installs and runs for me, so that's good.

e.g.

curl -L \
  -H "Accept: application/vnd.github+json" \
  -H "Authorization: Bearer <token>" \
  -H "X-GitHub-Api-Version: 2022-11-28" \
  https://api.github.com/repos/ANTsX/ANTsPy/actions/artifacts/1503134086/zip --output test.zip

I didn't know the API trick, very useful. On the web, once the action finishes all jobs, artifacts appear at the bottom of the page for download.

[cookpa]

I don't understand how ANTsPy/src/antsImageWin.cxx, formally ants/lib/LOCAL_antsImageWin.cxx was used. Was it copied to LOCAL_antsImage.cxx at some point during installation if the platform is windows? Maybe @stnava knows.

[cookpa]

Also, if the pointer string is wrong somehow, shouldn't all Windows builds fail consistently?

[ncullen93]

Yeah good point. It is probably not necessary. I think getting 3.9 - 3.12 to build consistently is very good. Feel free to merge whenever.