macOS build failure due to use of ld -r and injected rpaths by mpi compiler wrapper

[haampie]

This is not a Spack bug, it's an issue with openmpi configured with --enable-wrapper-rpath --with-wrapper-ldflags=/path/to/something, which I think is common enough. It tells openmpi to add certain default rpaths (e.g. to gcc runtime libraries).

The issue is

mpic++ -r ...

invokes

clang++ -r -Wl,-rpath,/path/to/something

which calls the linker

ld -r -rpath /path/to/something

which pedantically errors out because -rpath does not apply when creating a new object file with -r.

---

So:

1. The issue is caused by OpenMPI's compiler wrapper adding -rpath even if -r is passed.
2. Maybe parallel-netcdf should just create a static archive with ar instead of combining into an object file with ld -r, since there's tons of other compiler wrappers that could add rpaths and are unaware of ld -r.

Originally posted by @haampie in #155

[haampie]

Question is why parallel-netcdf adds -r at all, and it looks like this is due to a bug in libtool, fixed in libtool v2.5.4:

commit 82f7f52123e4e7e50721049f7fa6f9b870e09c9d
Author: Joshua Root <[email protected]>
Date:   Thu Sep 28 09:02:11 2023 +0300

    libtool.m4: Update '-single_module' detection
    
    '-single_module' detection is broken with Xcode 15, where a message to
    stderr indicates the flag is deprecated, not unsupported.
    
    * m4/libtool.m4: Check macOS versions to see if '-single_module' flag is
      unnecessary.

diff --git a/NEWS b/NEWS
index 561122c6..4c76075f 100644
--- a/NEWS
+++ b/NEWS
@@ -2,6 +2,11 @@ NEWS - list of user-visible changes between releases of GNU Libtool
 
 * Noteworthy changes in release ?.? (????-??-??) [?]
 
+** Bug fixes:
+
+  - Fix incorrect use of workarounds designed for Darwin versions that
+    don't have -single_module support.
+
 
 * Noteworthy changes in release 2.5.3 (2024-09-25) [stable]
 
diff --git a/m4/libtool.m4 b/m4/libtool.m4
index e5ddacee..5f701b70 100644
--- a/m4/libtool.m4
+++ b/m4/libtool.m4
@@ -1100,6 +1100,21 @@ _LT_EOF
     if test yes = "$lt_cv_apple_cc_single_mod"; then
       _lt_dar_single_mod='$single_module'
     fi
+    _lt_dar_needs_single_mod=no
+    case $host_os in
+    rhapsody* | darwin1.*)
+      _lt_dar_needs_single_mod=yes ;;
+    darwin*)
+      # When targeting Mac OS X 10.4 (darwin 8) or later,
+      # -single_module is the default and -multi_module is unsupported.
+      # The toolchain on macOS 10.14 (darwin 18) and later cannot
+      # target any OS version that needs -single_module.
+      case ${MACOSX_DEPLOYMENT_TARGET-10.0},$host in
+      10.0,*-darwin[[567]].*|10.[[0-3]],*-darwin[[5-9]].*|10.[[0-3]],*-darwin1[[0-7]].*)
+        _lt_dar_needs_single_mod=yes ;;
+      esac
+    ;;
+    esac
     if test yes = "$lt_cv_ld_exported_symbols_list"; then
       _lt_dar_export_syms=' $wl-exported_symbols_list,$output_objdir/$libname-symbols.expsym'
     else
@@ -1145,7 +1160,7 @@ m4_defun([_LT_DARWIN_LINKER_FEATURES],
     _LT_TAGVAR(archive_expsym_cmds, $1)="$SED 's|^|_|' < \$export_symbols > \$output_objdir/\$libname-symbols.expsym~\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$libobjs \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring $_lt_dar_single_mod$_lt_dar_export_syms$_lt_dsymutil"
     _LT_TAGVAR(module_expsym_cmds, $1)="$SED -e 's|^|_|' < \$export_symbols > \$output_objdir/\$libname-symbols.expsym~\$CC \$allow_undefined_flag -o \$lib -bundle \$libobjs \$deplibs \$compiler_flags$_lt_dar_export_syms$_lt_dsymutil"
     m4_if([$1], [CXX],
-[   if test yes != "$lt_cv_apple_cc_single_mod"; then
+[   if test yes = "$_lt_dar_needs_single_mod" -a yes != "$lt_cv_apple_cc_single_mod"; then
       _LT_TAGVAR(archive_cmds, $1)="\$CC -r -keep_private_externs -nostdlib -o \$lib-master.o \$libobjs~\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$lib-master.o \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring$_lt_dsymutil"
       _LT_TAGVAR(archive_expsym_cmds, $1)="$SED 's|^|_|' < \$export_symbols > \$output_objdir/\$libname-symbols.expsym~\$CC -r -keep_private_externs -nostdlib -o \$lib-master.o \$libobjs~\$CC -dynamiclib \$allow_undefined_flag -o \$lib \$lib-master.o \$deplibs \$compiler_flags -install_name \$rpath/\$soname \$verstring$_lt_dar_export_syms$_lt_dsymutil"
     fi

[wkliao]

Hi, @haampie
Thank for reporting and providing the tip for the fix.
I wonder if you can give PR #164 a try to see if it works for your test environment.

[haampie]

That should work. We're setting lt_cv_apple_cc_single_mod=yes explicitly now in Spack instead, so haven't tested.