Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add heFFTe #26633

Merged
merged 12 commits into from
Aug 30, 2024
Merged

Add heFFTe #26633

merged 12 commits into from
Aug 30, 2024

Conversation

ax3l
Copy link
Member

@ax3l ax3l commented Jun 11, 2024

Add the heFFTe library, a Highly Efficient FFT for Exascale.

Checklist

  • Title of this PR is meaningful: e.g. "Adding my_nifty_package", not "updated meta.yaml".
  • License file is packaged (see here for an example).
  • Source is from official source.
  • Package does not vendor other packages. (If a package uses the source of another package, they should be separate packages or the licenses of all packages need to be packaged).
  • If static libraries are linked in, the license of the static library is packaged.
  • Package does not ship static libraries. If static libraries are needed, follow CFEP-18.
  • Build number is 0.
  • A tarball (url) rather than a repo (e.g. git_url) is used in your recipe (see here for more details).
  • GitHub users listed in the maintainer section have posted a comment confirming they are willing to be listed there.
  • When in trouble, please check our knowledge base documentation before pinging a team.

@conda-forge-webservices
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipes/heffte) and found some lint.

Here's what I've got...

For recipes/heffte:

  • The recipe must have some tests.
  • There are 1 too many lines. There should be one empty line at the end of the file.

@conda-forge-webservices
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipes/heffte) and found some lint.

Here's what I've got...

For recipes/heffte:

  • The recipe must have some tests.

@conda-forge-webservices
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/heffte) and found it was in an excellent condition.

@ax3l
Copy link
Member Author

ax3l commented Jun 12, 2024

@conda-forge/help-c-cpp this new package for heFFTe is ready for review.

In the Linux test, I do not understand the linker missing the libpthread_nonshared.a library. This might be an MPI or FFTW package bug in CF... or a missing package? I cannot say for sure.

[ 20%] Linking CXX executable heffte_example_fftw
/home/conda/staged-recipes/build_artifacts/heffte_1718162756113/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/bin/../lib/gcc/x86_64-conda-linux-gnu/12.3.0/../../../../x86_64-conda-linux-gnu/bin/ld: cannot find /usr/lib64/libpthread_nonshared.a: No such file or directory
collect2: error: ld returned 1 exit status
make[2]: *** [examples/CMakeFiles/heffte_example_fftw.dir/build.make:107: examples/heffte_example_fftw] Error 1

@conda-forge-webservices
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipes/heffte) and found some lint.

Here's what I've got...

For recipes/heffte:

  • The following maintainers have not yet confirmed that they are willing to be listed here: mkstoyanov. Please ask them to comment on this PR if they are.

@ax3l
Copy link
Member Author

ax3l commented Jun 13, 2024

ping @conda-forge/help-c-cpp any idea on this pthread issue in tests? :)

@mkstoyanov
Copy link

I agree to be listed as feedstock maintainer.

@egpbos
Copy link
Contributor

egpbos commented Jun 26, 2024

Is this possibly an issue with FFTW's CMake support itself? Because that is not officially supported, see e.g. FFTW/fftw3#307, so you may need to fix it there.

@ax3l
Copy link
Member Author

ax3l commented Jul 20, 2024

That is possible, I myself do some quite elaborate back-and-forth when searching for FFTW downstream in other codes: https://github.com/ECP-WarpX/WarpX/blob/development/cmake/dependencies/FFT.cmake

@mkstoyanov do you have any recommendations on how to build heFFTe against FFTW on Linux or have seen this issue before?

@mkstoyanov
Copy link

@ax3l I don't know why I missed the notification on this mention. Feel free to email me at work and I seem to be getting the comments on the heFFTe issues.

I've only ever build heFFTe with FFTW on Linux using either default installed packages from apt or the system modules on the HPC clusters. The modules lead only one set of libraries at a time (usually) and I've had no conflicts with apt.

The long list of "overlinking" was a work-around when I don't know what will be installed. The hope is that the compiler will just pick the first one it finds and ignores the rest. I don't know what is the right logic to handle this if we have multiple libraries (e.g., openmp and threads) and how to detect whether a compiler supports OpenMP (clang can be iffy with OpenMP).

The last-resort solution that heFFTe supports is that the user can manually specify:

-D FFTW_LIBRARIES=...
-D FFTW_INCLUDES=...

If those two are present, heFFTe will not do any "guessing" or try to find or enable anything else.

I don't know how/if you can get the list of libraries and includes from conda but if you can, this should just work.

@conda-forge-webservices
Copy link

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/heffte/meta.yaml) and found it was in an excellent condition.

@ax3l
Copy link
Member Author

ax3l commented Aug 29, 2024

Thanks, @mkstoyanov!

I now started a patch to make the FFTW threads search in heFFTe's CMake logic more robust:
icl-utk-edu/heffte#55

I am trying to find out: is FFTW thread support (which is optional) are requirement for heFFTe to build? I see a few calls in a benchmark like this that suggest this:
https://github.com/icl-utk-edu/heffte/blob/a6236b06f321e06c23f2615a26da1ee82c63b962/benchmarks/speed3d.h#L49-L53
Those lines assumes FFTW presence is equal to thread or OMP support?

@ax3l ax3l force-pushed the heffte branch 2 times, most recently from f91595e to 370b92a Compare August 29, 2024 05:25
One test, `example_r2c`, likes to run with 4 processes.
We have 2 cores on Azure.
@ax3l
Copy link
Member Author

ax3l commented Aug 29, 2024

@egpbos @jakirkham we made it - heFFTe ready for review & merge :)

@egpbos
Copy link
Contributor

egpbos commented Aug 29, 2024

Awesome @ax3l! I'm not a reviewer here, but it LGTM nonetheless ;)

@ax3l
Copy link
Member Author

ax3l commented Aug 29, 2024

Oops, sorry 😅

@conda-forge/help-c-cpp heFFTe package is ready to gooo 🚀

recipes/heffte/meta.yaml Outdated Show resolved Hide resolved
recipes/heffte/meta.yaml Outdated Show resolved Hide resolved
Co-authored-by: Isuru Fernando <[email protected]>
@isuruf isuruf merged commit 1e14078 into conda-forge:main Aug 30, 2024
5 checks passed
@ax3l ax3l deleted the heffte branch August 30, 2024 01:04
@ax3l
Copy link
Member Author

ax3l commented Aug 30, 2024

Thank you, Isuru!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

5 participants