Issues with pip install proposal for Windows / VS

[Jean1995]

There is a report of installation issues when trying to install PROPOSAL with pip install proposal on Windows machines.

Firstly, CMake fails with the error

      CMake Error at build/conan_toolchain.cmake:32 (message):

        The CMake policy CMP0091 must be NEW, but is ''

Might be related to conan-io/conan#12458
This can be fixed by explicitly setting DCMAKE_POLICY_DEFAULT_CMP0091.

However, then there is another problem with CMake:

      MSBUILD : error MSB1001: Unknown switch.

          Full command line: '"C:/Program Files (x86)/Microsoft Visual Studio/2022/BuildTools/MSBuild/Current/Bin/amd64/MSBuild.exe" ALL_BUILD.vcxproj /p:Configuration=Release /p:Platform=x64 /p:VisualStudioVersion=17.0 /v:m -j2'

        Switches appended by response files:

      '' came from 'C:\Program Files (x86)\Microsoft Visual Studio\2022\BuildTools\MSBuild\Current\Bin\amd64\MSBuild.rsp'

      Switch: -j2

Apparently, there is a problem with the way -j2 is passed from setup.py to CMake. This can temporarily be fixed by disabling -j.

For now, I have created the branch disable_multicore_build which fixes these two issues. Until this if fixed on the master, installation is possible with

git clone https://github.com/tudo-astroparticlephysics/PROPOSAL.git
cd PROPOSAL
git switch disable_multicore_build
pip install .

[maxnoe]

-j is an option for make, not cmake. On windows, make is not used, so the build system (the actual build system, not cmake, which is the build system generator) complains.

One solution would be to use Ninja as the build system, which is supported on all platforms and available from PyPI, so it can be put into the build-dependencies in pyproject.toml.

(scikit-build-core also uses ninja)

Ninja parallizes the build by default to all available cores

[Jean1995]

A new issue that seemed to have emerged on Windows:

pip install does not copy the created *.pyd file to the site-packages folder, so the proposal library can not be found...

The verbose output from pip install . --verbose looks like:

  -- Build files have been written to: C:/Users/Jean-Marco/software/PROPOSAL/build/temp.win-amd64-cpython-311/Release
  MSBuild version 17.5.1+f6fdcf537 for .NET Framework

    1>Checking Build System
    Building Custom Rule C:/Users/Jean-Marco/software/PROPOSAL/src/PROPOSAL/CMakeLists.txt
    PROPOSAL.vcxproj -> C:\Users\Jean-Marco\software\PROPOSAL\build\temp.win-amd64-cpython-311\Release\src\PROPOSAL\Release\PROPOSAL.lib
    Building Custom Rule C:/Users/Jean-Marco/software/PROPOSAL/src/pyPROPOSAL/CMakeLists.txt
    pyPROPOSAL.vcxproj -> C:\Users\Jean-Marco\software\PROPOSAL\build\temp.win-amd64-cpython-311\Release\src\pyPROPOSAL\Release\proposal.cp311-win_amd64.pyd
    Building Custom Rule C:/Users/Jean-Marco/software/PROPOSAL/CMakeLists.txt
  Using conan to install dependencies. Set environment variable NO_CONAN to skip conan.
  installing to build\bdist.win-amd64\wheel
  running install
  running install_lib
  creating build\bdist.win-amd64\wheel
  running install_egg_info
  running egg_info
  writing proposal.egg-info\PKG-INFO
  writing dependency_links to proposal.egg-info\dependency_links.txt
  writing top-level names to proposal.egg-info\top_level.txt
  reading manifest file 'proposal.egg-info\SOURCES.txt'
  reading manifest template 'MANIFEST.in'
  warning: no previously-included files matching '.git*' found anywhere in distribution
  adding license file 'LICENSE.md'
  writing manifest file 'proposal.egg-info\SOURCES.txt'
  Copying proposal.egg-info to build\bdist.win-amd64\wheel\.\proposal-7.6.2-py3.11.egg-info
  running install_scripts
  C:\Users\Jean-Marco\AppData\Local\Temp\pip-build-env-bvlxvnbd\overlay\Lib\site-packages\wheel\bdist_wheel.py:108: RuntimeWarning: Config variable 'Py_DEBUG' is unset, Python ABI tag may be incorrect
    if get_flag("Py_DEBUG", hasattr(sys, "gettotalrefcount"), warn=(impl == "cp")):
  creating build\bdist.win-amd64\wheel\proposal-7.6.2.dist-info\WHEEL
  creating 'C:\Users\Jean-Marco\AppData\Local\Temp\pip-wheel-3meb__xf\.tmp-rg4drv7_\proposal-7.6.2-cp311-cp311-win_amd64.whl' and adding 'build\bdist.win-amd64\wheel' to it
  adding 'proposal-7.6.2.dist-info/LICENSE.md'
  adding 'proposal-7.6.2.dist-info/METADATA'
  adding 'proposal-7.6.2.dist-info/WHEEL'
  adding 'proposal-7.6.2.dist-info/top_level.txt'
  adding 'proposal-7.6.2.dist-info/RECORD'
  removing build\bdist.win-amd64\wheel
  Building wheel for proposal (pyproject.toml) ... done
  Created wheel for proposal: filename=proposal-7.6.2-cp311-cp311-win_amd64.whl size=7504 sha256=195294b9012e9d3e811bfac390894d5aa004ee424977897413925f83b9f873a4
  Stored in directory: C:\Users\Jean-Marco\AppData\Local\Temp\pip-ephem-wheel-cache-a2pwwtdf\wheels\aa\e1\b6\7dbacbe9e10eaae3d5cb7dff8b96127564cbc96c6dbd6ba76d
Successfully built proposal
Installing collected packages: proposal
Successfully installed proposal-7.6.2

My workaround so far was to copy the PROPOSAL\build\temp.win-amd64-cpython-311\Release\src\pyPROPOSAL\Release\proposal.cp311-win_amd64.pyd file manually to the site-packages folder, but this is not a viable solution... 😅

I don't know how to tell setup.py to add this file -- and I also believed that this has worked earlier, so something about Windows must have changed...?

[maxnoe]

I would suggest to try switching to scikit-build-core and not apply further hacks to the current cmake code in setup.py