Spectre IO H5 _Pybindings not found after cmake

[thisismeamir]

It's been a couple of weeks since I am continuously trying to get all the binaries of spectre available on my computer. I have followed the instructions on spectre-code website and I'm using the released version of spectre.

inside the directory I do:

mkdir build && cd build

and then

cmake ..

Since all the bugs previously has been worked out I recieve the following messages (which I assume it means that everything is fine)

-- SpECTRE release version: 2024.09.29
-- The CXX compiler identification is GNU 13.2.0
-- The C compiler identification is GNU 13.2.0
-- The Fortran compiler identification is GNU 13.2.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting Fortran compiler ABI info
-- Detecting Fortran compiler ABI info - done
-- Check for working Fortran compiler: /usr/bin/gfortran - skipped
-- Not running in a git repository. Some features are disabled.
-- Found Python: /usr/bin/python3 (found suitable version "3.12.3", minimum required is "3.8") found components: Interpreter 
-- PYTHONPATH: /home/kid-a/Downloads/spectre-2024.09.29/build/bin/python:/home/kid-a/Downloads/spectre-2024.09.29/build/lib/python3.12/site-packages
-- Found Git: /usr/bin/git (found version "2.43.0") 
-- CMAKE_BUILD_TYPE not specified, setting to 'Debug'
-- Link-time optimizations disabled (no LTO)
-- Found Boost: /usr/lib/x86_64-linux-gnu/cmake/Boost-1.83.0/BoostConfig.cmake (found suitable version "1.83.0", minimum required is "1.60.0") found components: program_options 
-- Boost libraries: Boost::program_options
-- Boost include: /usr/include
-- Boost version: 1.83.0
-- Could NOT find ccache (missing: CCACHE_EXEC) 
-- Charm++ 'charmc' is invoked as: /home/kid-a/Documents/other-people-projects/spack/opt/spack/linux-ubuntu24.04-skylake/gcc-13.2.0/charmpp-8.0.0-dzsjsr5vdg25v54dpsl7golk6vxc23xk/bin/charmc -print-building-blocks -nomain -nomain-module -modules EveryLB
-- Charm++ lib names: moduleEveryLB, ckmetis, ck, memory-default, threads-default, converse, m, memory-default, threads-default, ldb-rand, converse, pthread, ckqt, ckqt, z, dl, moduleEveryLB, moduleNDMeshStreamer, modulecompletion, z, m
-- These Charm++ compiler flags will be ignored: -DCMK_GFORTRAN -DCMK_GFORTRAN -D_REENTRANT -D__CHARMC__=1 -U_FORTIFY_SOURCE -fno-stack-protector -fno-lifetime-dse -ftls-model=initial-exec
-- These Charm++ linker flags will be ignored: -D_REENTRANT -ftls-model=initial-exec
-- Found Charm: /home/kid-a/Documents/other-people-projects/spack/opt/spack/linux-ubuntu24.04-skylake/gcc-13.2.0/charmpp-8.0.0-dzsjsr5vdg25v54dpsl7golk6vxc23xk/bin/charmc (found suitable version "8.0.0", minimum required is "7.0.0") 
CMake Warning at cmake/SetupCharm.cmake:42 (message):
  Builds with Charm++ versions greater than 7.0.1 are considered
  experimental.  Please file any issues you encounter.
Call Stack (most recent call first):
  CMakeLists.txt:126 (include)


-- Checking for broken std::array<..., 0>
-- Checking for broken std::array<..., 0> -- works
-- Looking for Fortran sgemm
-- Looking for Fortran sgemm - not found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE  
-- Looking for Fortran sgemm
-- Looking for Fortran sgemm - found
-- Found BLAS: /usr/lib/x86_64-linux-gnu/libblas.so  
-- BLAS libs: /usr/lib/x86_64-linux-gnu/libblas.so
-- BLAS vendor is probably not OpenBLAS. Make sure it doesn't try to do multithreading that might conflict with Charm++ parallelism.
-- Looking for Fortran cheev
-- Looking for Fortran cheev - not found
-- Looking for Fortran cheev
-- Looking for Fortran cheev - found
-- Found LAPACK: /usr/lib/x86_64-linux-gnu/liblapack.so;/usr/lib/x86_64-linux-gnu/libblas.so  
-- LAPACK libs: /usr/lib/x86_64-linux-gnu/liblapack.so/usr/lib/x86_64-linux-gnu/libblas.so
-- Found LIBXSMM: /usr/include (found suitable version "1.17.0", minimum required is "1.16.1") 
-- Found GSL: /usr/include (found version "2.7.1") 
-- GSL libs: /usr/lib/x86_64-linux-gnu/libgsl.so;/usr/lib/x86_64-linux-gnu/libgslcblas.so
-- GSL incl: /usr/include
-- GSL vers: 2.7.1
-- Found Blaze: /usr/local/include (found suitable version "3.9", minimum required is "3.8") 
-- Blaze incl: /usr/local/include
-- Blaze vers: 3.9
-- Found MPI_C: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so (found version "3.1") 
-- Found MPI: TRUE (found version "3.1") found components: C 
-- Could NOT find FUKA (missing: FUKA_LIB) 
-- Found HDF5: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so;/usr/lib/x86_64-linux-gnu/libcrypto.so;/usr/lib/x86_64-linux-gnu/libcurl.so;/usr/lib/x86_64-linux-gnu/libpthread.a;/usr/lib/x86_64-linux-gnu/libsz.so;/usr/lib/x86_64-linux-gnu/libz.so;/usr/lib/x86_64-linux-gnu/libdl.a;/usr/lib/x86_64-linux-gnu/libm.so (found version "1.10.10") found components: C 
-- HDF5 libs: /usr/lib/x86_64-linux-gnu/hdf5/serial/libhdf5.so/usr/lib/x86_64-linux-gnu/libcrypto.so/usr/lib/x86_64-linux-gnu/libcurl.so/usr/lib/x86_64-linux-gnu/libpthread.a/usr/lib/x86_64-linux-gnu/libsz.so/usr/lib/x86_64-linux-gnu/libz.so/usr/lib/x86_64-linux-gnu/libdl.a/usr/lib/x86_64-linux-gnu/libm.so
-- HDF5 incl: /usr/include/hdf5/serial
-- HDF5 vers: 1.10.10
-- Performing Test HDF5_SUPPORTS_SET_FILE_LOCKING
-- Performing Test HDF5_SUPPORTS_SET_FILE_LOCKING - Success
-- Performing Test HDF5_INIT_WITHOUT_FPES
-- Performing Test HDF5_INIT_WITHOUT_FPES - Success
-- Found JEMALLOC: /usr/include (found version "5.3.0") 
-- jemalloc libs: /usr/lib/x86_64-linux-gnu/libjemalloc.so
-- jemalloc incl: /usr/include
-- jemalloc vers: 5.3.0
-- Found Python: /usr/bin/python3 (found version "3.12.3") found components: Interpreter Development Development.Module Development.Embed 
-- Found pybind11-config tool (/usr/bin/pybind11-config) and determined CMake dir: /usr/lib/python3/dist-packages/pybind11/share/cmake/pybind11
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- Found pybind11: /usr/lib/python3/dist-packages/pybind11/include (found version "2.11.1")
-- Pybind11 include: /usr/lib/python3/dist-packages/pybind11/include
-- Could NOT find SpEC (missing: SPEC_PACKAGED_EXPORTER_LIB SPEC_EXPORTER_FACTORY_OBJECTS SPEC_EXPORTER_INCLUDE_DIR) 
-- Brigand include: /home/kid-a/Downloads/spectre-2024.09.29/external/brigand/include
-- Found ClangFormat: /usr/bin/clang-format (found version "Ubuntu 18.1.3 (1ubuntu1)") 
-- clang-tidy: Not using clang or couldn't find clang-tidy.
-- Found jupyter-nbconvert: /usr/bin/jupyter-nbconvert
-- Found Doxygen: /usr/bin/doxygen (found version "1.9.8") found components: doxygen dot 
-- Found Perl: /usr/bin/perl (found version "5.38.2") 
-- Found PY_bs4: /usr/lib/python3/dist-packages/bs4  
-- Found PY_pybtex: /usr/lib/python3/dist-packages/pybtex  
-- Could NOT find PY_coverxygen (missing: PY_coverxygen_LOCATION) 
-- Found NumPy: /usr/lib/python3/dist-packages/numpy/core/include (found suitable version "1.26.4", minimum required is "1.10") 
-- NumPy ver. 1.26.4 found (include: /usr/lib/python3/dist-packages/numpy/core/include)
-- NumPy incl: /usr/lib/python3/dist-packages/numpy/core/include
-- NumPy vers: 1.26.4
-- Found PY_scipy: /usr/lib/python3/dist-packages/scipy  
-- Using libbacktrace for stack traces: /usr/lib/gcc/x86_64-linux-gnu/13/libbacktrace.a
-- Could NOT find PY_paraview (missing: PY_paraview_LOCATION) 
-- ParaView not found, skipping 'Render3D' tests
-- 
Useful Information:
-- Git description: 
-- Git branch: 
-- Git hash: 
-- Build directory: /home/kid-a/Downloads/spectre-2024.09.29/build
-- Source directory: /home/kid-a/Downloads/spectre-2024.09.29
-- Bin directory: /home/kid-a/Downloads/spectre-2024.09.29/build/bin/
-- CMake modules path: /home/kid-a/Downloads/spectre-2024.09.29/cmake;/usr/local/lib/cmake/Catch2
-- CMAKE_CXX_FLAGS: 
-- CMAKE_CXX_LINK_FLAGS:  -fuse-ld=gold
-- CMAKE_CXX_FLAGS_DEBUG: -g -DSPECTRE_DEBUG
-- CMAKE_CXX_FLAGS_RELEASE: -O3 -DNDEBUG
-- CMAKE_EXE_LINKER_FLAGS: 
-- CMAKE_BUILD_TYPE: Debug
-- CMAKE_CXX_COMPILER: /usr/bin/c++
-- BUILD_SHARED_LIBS: 
-- USE_PCH: ON
-- ASAN: OFF
-- UBSAN_UNDEFINED: OFF
-- UBSAN_INTEGER: OFF
-- Python: /usr/bin/python3
-- Python version: 3.12.3
-- BUILD_PYTHON_BINDINGS: ON
-- Doxygen: /usr/bin/doxygen
-- When using the python bindings you must run python as:
   LD_PRELOAD=/usr/lib/x86_64-linux-gnu/libjemalloc.so python ...
Alternatively, use the system allocator by setting 
-D MEMORY_ALLOCATOR=SYSTEM
-- Configuring done (8.5s)
-- Generating done (1.3s)
-- Build files have been written to: /home/kid-a/Downloads/spectre-2024.09.29/build

But after I start the spectre in bin folder:

cd bin && ./spectre

I get:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/kid-a/Downloads/spectre-2024.09.29/build/bin/python/spectre/__main__.py", line 340, in <module>
    cli(prog_name="spectre")
  File "/usr/lib/python3/dist-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/click/core.py", line 1077, in main
    with self.make_context(prog_name, args, **extra) as ctx:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/click/core.py", line 943, in make_context
    self.parse_args(ctx, args)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1641, in parse_args
    echo(ctx.get_help(), color=ctx.color)
         ^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/click/core.py", line 704, in get_help
    return self.command.get_help(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/click/core.py", line 1325, in get_help
    self.format_help(ctx, formatter)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1358, in format_help
    self.format_options(ctx, formatter)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1564, in format_options
    self.format_commands(ctx, formatter)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1617, in format_commands
    cmd = self.get_command(ctx, subcommand)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kid-a/Downloads/spectre-2024.09.29/build/bin/python/spectre/__main__.py", line 58, in get_command
    from spectre.IO.H5.CombineH5 import combine_h5_command
  File "/home/kid-a/Downloads/spectre-2024.09.29/build/bin/python/spectre/IO/H5/__init__.py", line 4, in <module>
    from ._Pybindings import *
ModuleNotFoundError: No module named 'spectre.IO.H5._Pybindings'

[kidder]

did you build any targets after running cmake? For example make cli or make all-pybindings

[thisismeamir]

@kidder Yes I specially used make cli the result is the same. The installation apparently needs site-packages but there's no such thing after git cloning in the whole directories and not even after cmake I guess this can be a bug.

In the build folder I have checked:

tree | grep site

returning nothing.

Apart from that there were many warnings for different python packages and I installed them by hand.

[nilsvu]

From an earlier issue you said you build without Python bindings. In that case you won't be able to use the Python features.

[thisismeamir]

@nilsvu True, but even with the flag to shutdown pythonbindings I still get this error message. So I guess it worth having bindings on as well. So:

cmake -DBUILD_PYTHON_BINDINGS=OFF ..

and we get:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/kid-a/Downloads/spectre-2024.09.29/build/bin/python/spectre/__main__.py", line 340, in <module>
    cli(prog_name="spectre")
  File "/usr/lib/python3/dist-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/click/core.py", line 1077, in main
    with self.make_context(prog_name, args, **extra) as ctx:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/click/core.py", line 943, in make_context
    self.parse_args(ctx, args)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1641, in parse_args
    echo(ctx.get_help(), color=ctx.color)
         ^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/click/core.py", line 704, in get_help
    return self.command.get_help(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/click/core.py", line 1325, in get_help
    self.format_help(ctx, formatter)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1358, in format_help
    self.format_options(ctx, formatter)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1564, in format_options
    self.format_commands(ctx, formatter)
  File "/usr/lib/python3/dist-packages/click/core.py", line 1617, in format_commands
    cmd = self.get_command(ctx, subcommand)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/kid-a/Downloads/spectre-2024.09.29/build/bin/python/spectre/__main__.py", line 58, in get_command
    from spectre.IO.H5.CombineH5 import combine_h5_command
  File "/home/kid-a/Downloads/spectre-2024.09.29/build/bin/python/spectre/IO/H5/__init__.py", line 4, in <module>
    from ._Pybindings import *
ImportError: /home/kid-a/Downloads/spectre-2024.09.29/build/bin/python/spectre/IO/H5/_Pybindings.so: cannot allocate memory in static TLS block

Honestly I don't know what makes the builds on my computer fail. I'm just trying every possible options... (It's two weeks that I'm building this project lol).

[nilsvu]

You disabled Python bindings, so you're getting this error because Python bindings are not available.

Please follow these installation instructions: https://spectre-code.org/installation.html#quick_start_install
If you find an error following these instructions, please open an issue with the error you get and the files mentioned in the issue template.

[thisismeamir]

You disabled Python bindings, so you're getting this error because Python bindings are not available.

Please follow these installation instructions: https://spectre-code.org/installation.html#quick_start_install
If you find an error following these instructions, please open an issue with the error you get and the files mentioned in the issue template.

In the original text ( the issue content) it is on.
But either way i get this error.

I will try installing it tomorrow on a better computer. But i have read the documentation and did exactly as i was told.