make not linking #2

ceheitsch · 2021-02-17T21:24:58Z

/usr/local/Cellar/cmake/3.19.4/bin/cmake -E cmake_link_script CMakeFiles/gtmfe.dir/link.txt --verbose=1
/usr/local/opt/llvm/bin/clang++  -Wall -mlinker-version=450 -std=c++11 -fPIC -fopenmp -g -O2 -fno-lto         -I/usr/local/include -I/usr/local/opt/llvm/include -Igtfold-mfe/include/         -I/usr/local/Cellar/gmp/6.2.1/include -I/usr/local/Cellar/gmp/6.2.1/include         -DGTFOLD_PACKAGE_VERSION="1.0.0"  -Wall -mlinker-version=450 -std=c++11 -fPIC -fopenmp -g -O2 -fno-lto         -I/usr/local/include -I/usr/local/opt/llvm/include -Igtfold-mfe/include/         -I/usr/local/Cellar/gmp/6.2.1/include -I/usr/local/Cellar/gmp/6.2.1/include         -DGTFOLD_PACKAGE_VERSION="1.0.0" -L/usr/local/opt/llvm/lib          -mlinker-version=450 -L/usr/local/lib -lgmp -lgmpxx -lm -lpthread         -L/usr/local/Cellar/gmp/6.2.1/lib -lgmp -L/usr/local/Cellar/gmp/6.2.1/lib -lgmpxx -lgmp -flto=thin -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk -mmacosx-version-min=10.14 -Wl,-search_paths_first -Wl,-headerpad_max_install_names  -Wall -mlinker-version=450 -std=c++11 -fPIC -fopenmp -g -O2 -fno-lto         -I/usr/local/include -I/usr/local/opt/llvm/include -Igtfold-mfe/include/         -I/usr/local/Cellar/gmp/6.2.1/include -I/usr/local/Cellar/gmp/6.2.1/include         -DGTFOLD_PACKAGE_VERSION="1.0.0" -L/usr/local/opt/llvm/lib          -mlinker-version=450 -L/usr/local/lib -lgmp -lgmpxx -lm -lpthread         -L/usr/local/Cellar/gmp/6.2.1/lib -lgmp -L/usr/local/Cellar/gmp/6.2.1/lib -lgmpxx -lgmp CMakeFiles/gtmfe.dir/gtfold-mfe/src/AdvancedDouble.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/algorithms-partition.c.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/algorithms.c.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/boltzmann_main.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/constraints.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/energy.c.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/global.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/key.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/loader.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/mfe_main.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/options.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/partition-dangle.c.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/partition-func-d2.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/partition-func.c.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/pf-shel-check.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/random-sample.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/shapereader.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/stochastic-sampling-d2.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/stochastic-sampling.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/subopt_main.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/subopt_traceback.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/traceback.c.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/utils.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/binexe-utils/main.cc.o -o bin/gtmfe 
undef: _myExp
Undefined symbols for architecture x86_64:
  "_myExp", referenced from:
      _f in lto.o
      _calc_up_parallel in lto.o
      _calc_s1 in lto.o
      _calc_s2 in lto.o
      _calc_upm in lto.o
      _calc_up in lto.o
      _calc_s3 in lto.o
      ...
ld: symbol(s) not found for architecture x86_64
clang-11: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [bin/gtmfe] Error 1
make[1]: *** [CMakeFiles/gtmfe.dir/all] Error 2
make: *** [all] Error 2```

The text was updated successfully, but these errors were encountered:

ceheitsch · 2021-02-17T21:25:18Z

@(#)PROGRAM:ld  PROJECT:ld64-450.3
BUILD 18:01:43 Mar 13 2019
configured to support archs: armv6 armv7 armv7s arm64 arm64e arm64_32 i386 x86_64 x86_64h armv6m armv7k armv7m armv7em
LTO support using: LLVM version 10.0.1, (clang-1001.0.46.3) (static support for 22, runtime is 22)
TAPI support using: Apple TAPI version 10.0.1 (tapi-1001.0.4.1)```

maxieds · 2021-02-17T21:49:45Z

@ceheitsch
I updated the C++ standard to V14. Please try git pull, then retry the cmake followed by make procedure.

ceheitsch · 2021-02-18T13:33:18Z

Alas; no luck.

[  1%] Linking CXX executable bin/gtmfe
/usr/local/Cellar/cmake/3.19.4/bin/cmake -E cmake_link_script CMakeFiles/gtmfe.dir/link.txt --verbose=1
/usr/local/opt/llvm/bin/clang++  -Wall -mlinker-version=450 -std=c++14 -fPIC -fopenmp -g -O2 -fno-lto         -I/usr/local/include -I/usr/local/opt/llvm/include -Igtfold-mfe/include/         -I/usr/local/Cellar/gmp/6.2.1/include -I/usr/local/Cellar/gmp/6.2.1/include         -DGTFOLD_PACKAGE_VERSION="1.0.0"  -Wall -mlinker-version=450 -std=c++14 -fPIC -fopenmp -g -O2 -fno-lto         -I/usr/local/include -I/usr/local/opt/llvm/include -Igtfold-mfe/include/         -I/usr/local/Cellar/gmp/6.2.1/include -I/usr/local/Cellar/gmp/6.2.1/include         -DGTFOLD_PACKAGE_VERSION="1.0.0" -L/usr/local/opt/llvm/lib          -mlinker-version=450 -L/usr/local/lib -lgmp -lgmpxx -lm -lpthread         -L/usr/local/Cellar/gmp/6.2.1/lib -lgmp -L/usr/local/Cellar/gmp/6.2.1/lib -lgmpxx -lgmp -flto=thin -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk -mmacosx-version-min=10.14 -Wl,-search_paths_first -Wl,-headerpad_max_install_names  -Wall -mlinker-version=450 -std=c++14 -fPIC -fopenmp -g -O2 -fno-lto         -I/usr/local/include -I/usr/local/opt/llvm/include -Igtfold-mfe/include/         -I/usr/local/Cellar/gmp/6.2.1/include -I/usr/local/Cellar/gmp/6.2.1/include         -DGTFOLD_PACKAGE_VERSION="1.0.0" -L/usr/local/opt/llvm/lib          -mlinker-version=450 -L/usr/local/lib -lgmp -lgmpxx -lm -lpthread         -L/usr/local/Cellar/gmp/6.2.1/lib -lgmp -L/usr/local/Cellar/gmp/6.2.1/lib -lgmpxx -lgmp CMakeFiles/gtmfe.dir/gtfold-mfe/src/AdvancedDouble.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/algorithms-partition.c.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/algorithms.c.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/boltzmann_main.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/constraints.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/energy.c.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/global.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/key.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/loader.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/mfe_main.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/options.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/partition-dangle.c.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/partition-func-d2.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/partition-func.c.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/pf-shel-check.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/random-sample.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/shapereader.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/stochastic-sampling-d2.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/stochastic-sampling.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/subopt_main.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/subopt_traceback.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/traceback.c.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/utils.cc.o CMakeFiles/gtmfe.dir/gtfold-mfe/src/binexe-utils/main.cc.o -o bin/gtmfe 
undef: _myExp
Undefined symbols for architecture x86_64:
  "_myExp", referenced from:
      _f in lto.o
      _calc_up_parallel in lto.o
      _calc_s1 in lto.o
      _calc_s2 in lto.o
      _calc_upm in lto.o
      _calc_up in lto.o
      _calc_s3 in lto.o
      ...
ld: symbol(s) not found for architecture x86_64
clang-11: error: linker command failed with exit code 1 (use -v to see invocation)
make[2]: *** [bin/gtmfe] Error 1
make[1]: *** [CMakeFiles/gtmfe.dir/all] Error 2
make: *** [all] Error 2

maxieds · 2021-02-18T21:30:20Z

@ceheitsch
Ok. It looks like the templated code defined in a header needs to get moved to a .cc file for the symbols to get recognized. I will work on this for the week.

maxieds · 2021-02-22T21:52:38Z

@ceheitsch
If you still get errors, try running the following command first (see reference):

$ export CPATH=/Library/Developer/CommandLineTools/usr/include/c++/v1

Another option seems to be trying (we can walk through this on on Thursday if it comes down to it):

$ sudo xcode-select -s /Applications/Xcode.app/Contents/Developer

Please let me know what if any of these options works.

ceheitsch · 2021-02-23T20:00:06Z

The 'export' option did not seem to make a difference. FWIW:

 xcode-select -print-path
/Library/Developer/CommandLineTools

There is no Xcode.app directory under Applications:

% xcode-select -s /Applications/Xcode.app/Contents/Developer
xcode-select: error: invalid developer directory '/Applications/Xcode.app/Contents/Developer'

FYI, the account where the code is being compiled doesn't have sudo privileges. The machine admin is handled by a different account.

maxieds · 2021-02-23T20:14:14Z

@ceheitsch
It's actually preferable IMHO not to deal with the full XCode install anyway. Apple has this lovely tendency of selectively supporting, and then breaking their developer tools. This issue should boil down to a picky templates issue with changes in the C++ standards. It should not be as burdensome for GTFold users to install this on a Mac with cmake. Let me take some time to pick apart the code on another Mojave system and see if I can fix things without needing a critical upgrade to the vendor sanctioned toolchain.

ceheitsch · 2021-02-24T15:32:05Z

@maxieds Your approach makes a lot of sense. Unfortunately, the fresh 'make' didn't work, but it failed in a. different. way (so I'm counting that as progress). I'm attaching the whole output so you have all that info.
output_feb24.txt

maxieds · 2021-02-24T17:01:53Z

@ceheitsch
This reminds me of a longstanding fix for the non-standard cstring support on MacOS from RNAStructViz. Let me look at the errors more closely today. I will see if I can find a patch for the issue. This is actually encouraging in so much as it will probably just require adding in the include definitions for libc/libstdc++ from within the newly installed llvm sources.

Update:

This link on StackOverflow (for Catalina/10.15) should give you more perspective on how painful it is to maintain the compiler tools on Mac vendor systems without external software.

maxieds · 2021-02-24T17:49:03Z

@ceheitsch
Please try git pull && cmake [...] && make again where the arguments to cmake are documented as usual here.

I tried the following two fixes, both of which still generate a correctly compiling and linking build process on my Mac:

Added -isystem /usr/local/opt/llvm/include to the CMakeLists.txt configuration. The idea is to make sure that the compiler knows where to find its most up to date expected libc++ headers. This proposed fix should take into account some of my previous best guesses at the build problems related to problematic, or incorrectly symlinked, c++ headers on the system.
According to the suggestions noted here, it appears that recent clang++ compilers do not take well to the older style C header spec with #include <string.h>. I updated these references in the gtfold c++/*.cc/*.h sources to now point to #include <cstring>, a more modern standard way of getting to the same string functions. This seems to address the specific build errors you were having on Mojave (referenced in the link from my last reply).

Please let me know if this works. Otherwise, we can iterate if new errors are invoked after fixing these ones.

ceheitsch · 2021-02-25T01:41:38Z

Attaching output.
output_feb24_2nd.txt

maxieds · 2021-02-26T03:23:36Z

@ceheitsch
Please rerun git pull followed by the build commands. The latest commit removes the libstdc++ includes from the default search path, and then carefully re-appends the corresponding header directories to include these files, e.g., #include <iostream>, back to the working search path. It's not pretty, but it continues to build on my Mojave box. I suspect you should minimally at least encounter new errors output by the build with this fix. Let me know if it works.

My (truncated after the gtmfe build and test cases) output from compiling is attached as a reference point:
issue2-working-output-2021-02-25.out.txt

ceheitsch · 2021-02-26T20:23:26Z

@maxieds Ran through the process. Output attached.
output_feb26.txt

maxieds · 2021-02-26T21:00:56Z

@ceheitsch
I have another round of proposed fixes for these weird build errors in the most recent commit.

ceheitsch · 2021-03-11T00:16:43Z

@maxieds I tried on a different mac that I have access to at home, and it too failed. Interestingly, it failed in the same way as the original problem reported at the beginning of this issue, i.e.

Undefined symbols for architecture x86_64:
  "_myExp", referenced from:
      _f in lto.o
      _calc_up_parallel in lto.o
      _calc_s1 in lto.o
      _calc_s2 in lto.o
      _calc_upm in lto.o
      _calc_up in lto.o
      _calc_s3 in lto.o
      ...
ld: symbol(s) not found for architecture x86_64

FWIW, cmake was updated to 3.19.6, llvm 11.1.0 is now installed, and libomp was already updated at some point to 11.1.0.
I also note that it got about 20% through the make, which is considerably farther that we were getting on the other system.

ceheitsch · 2021-03-11T15:13:24Z

@maxieds On the original system (OSX 10.14.4), I tried a refresh install of llvm (11.1.0), cmake (3.19.6) and libomp (now 11.1.0). I also recloned the repo, just in case. Errors attached.
output10.14.4_mar11.txt

ceheitsch · 2021-03-11T15:15:51Z

@maxieds Here is the output on the other system (OSX 10.14.6).
output10.14.6_mar10.txt

ceheitsch · 2021-03-11T18:08:58Z

SoM IT installed cmake on my office machine, and gtfold builds under my account there. FWIW, I noted that clang-9 is being used, and not clang-11 as it is on the home machine(s).

maxieds · 2021-03-12T00:57:18Z

@ceheitsch
Unless I actually introduced new errors when we were debugging yesterday online, this error should have been fixed long ago. Please make sure you have recently run git pull, followed by the cmake (...) command, and then finally followed by make.

I had to go in and change some of the includes in some of the C/C++ source files so that they are now compatible with a modern clang++ compiler. It has been several weeks since this showed up last. If that doesn't work, make sure that the paths to llvm are correct. I recall that if the stock MacOS system ones get loaded before the updated ones, there will be issues.

ceheitsch · 2021-03-17T17:15:38Z

@maxieds That is exactly the point. I get the same error on a different system so clearly whatever was "fixed" previously was too localized.

ceheitsch · 2021-03-17T17:18:07Z

Is this relevant? Homebrew/homebrew-core#45061

ceheitsch · 2021-03-17T17:19:42Z

@maxieds I seem to recall asking about paths before. My memory is that you assured me that this wouldn't matter since they would be call be absolute paths.

maxieds · 2021-03-22T19:50:03Z

@ceheitsch
I looked at the CMakeLists.txt configuration after I noticed that the gtfold files were building with the custom Makefile I wrote last year for the Python bindings. I found something interesting:

$ python3-config --embed --cflags
-I/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.9/include/python3.9 
-I/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.9/include/python3.9 
-Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common 
-dynamic -DNDEBUG -g -fwrapv -O3 -Wall 
-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk

Note that if we do not call python3-config --libs --ldflags (as a convenience since it knows where to find things better on the system that I do, not because gtfold needs python), then this gets appended and everything goes to he-double-hockey-sticks:

-isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk 
-mmacosx-version-min=10.14

I am led to conclude that some of the issues we have been having on the different Mac Mojave systems is related to which version of the platform SDK is installed, and where it is located. (Working, but good intuitive hypothesis)

maxieds · 2021-03-22T19:51:31Z

Note also, that 10.15 is notably broken. So if the problem is that some installs point to a bad SDK, that could be a way to pinpoint build problems.

maxieds · 2021-03-23T04:59:04Z

@ceheitsch
It appears that the following procedure should fix things:

$ sudo xcode-select -s /Library/Developer/CommandLineTools
# Then add the following to the CMake command: 
#-DMAC_SYSROOT_PATH="/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk"

The updated docs are at the usual location.

maxieds added a commit that referenced this issue Feb 26, 2021

Proposed fix to issue #2 for Mac Mojave

2d01f0a

maxieds added a commit that referenced this issue Mar 23, 2021

Changing the CMake builds back to using clang (cf. issue #2)

7148374

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

make not linking #2

make not linking #2

ceheitsch commented Feb 17, 2021

ceheitsch commented Feb 17, 2021

maxieds commented Feb 17, 2021

ceheitsch commented Feb 18, 2021 •

edited

Loading

maxieds commented Feb 18, 2021

maxieds commented Feb 22, 2021

ceheitsch commented Feb 23, 2021

maxieds commented Feb 23, 2021 •

edited

Loading

ceheitsch commented Feb 24, 2021

maxieds commented Feb 24, 2021 •

edited

Loading

maxieds commented Feb 24, 2021 •

edited

Loading

ceheitsch commented Feb 25, 2021

maxieds commented Feb 26, 2021 •

edited

Loading

ceheitsch commented Feb 26, 2021

maxieds commented Feb 26, 2021

ceheitsch commented Mar 11, 2021

ceheitsch commented Mar 11, 2021

ceheitsch commented Mar 11, 2021

ceheitsch commented Mar 11, 2021

maxieds commented Mar 12, 2021

ceheitsch commented Mar 17, 2021

ceheitsch commented Mar 17, 2021

ceheitsch commented Mar 17, 2021

maxieds commented Mar 22, 2021

maxieds commented Mar 22, 2021

maxieds commented Mar 23, 2021

make not linking #2

make not linking #2

Comments

ceheitsch commented Feb 17, 2021

ceheitsch commented Feb 17, 2021

maxieds commented Feb 17, 2021

ceheitsch commented Feb 18, 2021 • edited Loading

maxieds commented Feb 18, 2021

maxieds commented Feb 22, 2021

ceheitsch commented Feb 23, 2021

maxieds commented Feb 23, 2021 • edited Loading

ceheitsch commented Feb 24, 2021

maxieds commented Feb 24, 2021 • edited Loading

Update:

maxieds commented Feb 24, 2021 • edited Loading

ceheitsch commented Feb 25, 2021

maxieds commented Feb 26, 2021 • edited Loading

ceheitsch commented Feb 26, 2021

maxieds commented Feb 26, 2021

ceheitsch commented Mar 11, 2021

ceheitsch commented Mar 11, 2021

ceheitsch commented Mar 11, 2021

ceheitsch commented Mar 11, 2021

maxieds commented Mar 12, 2021

ceheitsch commented Mar 17, 2021

ceheitsch commented Mar 17, 2021

ceheitsch commented Mar 17, 2021

maxieds commented Mar 22, 2021

maxieds commented Mar 22, 2021

maxieds commented Mar 23, 2021

ceheitsch commented Feb 18, 2021 •

edited

Loading

maxieds commented Feb 23, 2021 •

edited

Loading

maxieds commented Feb 24, 2021 •

edited

Loading

maxieds commented Feb 24, 2021 •

edited

Loading

maxieds commented Feb 26, 2021 •

edited

Loading