lc0 0.29.0 failed to build on osx

[chenrui333]

👋 trying to build the latest release, but run into some build issue. The error log is as below:

build failure

  clang++ -Ilc0.p -I. -I../.. -I../../src -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk/System/Library/Frameworks/Accelerate.framework/Headers -I/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk/System/Library/Frameworks/OpenCL.framework/Headers -flto -fcolor-diagnostics -DNDEBUG -Wall -Winvalid-pch -Wnon-virtual-dtor -Wextra -Wpedantic -std=c++17 -O3 -fobjc-arc -F/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk/System/Library/Frameworks/OpenCL.framework -isystem/opt/homebrew/Cellar/eigen/3.4.0_1/include/eigen3 -F/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk/System/Library/Frameworks/Accelerate.framework -isystem../../third_party -MD -MQ lc0.p/src_neural_metal_mps_NetworkGraph.mm.o -MF lc0.p/src_neural_metal_mps_NetworkGraph.mm.o.d -o lc0.p/src_neural_metal_mps_NetworkGraph.mm.o -c ../../src/neural/metal/mps/NetworkGraph.mm
  ../../src/neural/metal/mps/NetworkGraph.mm:709:51: error: no visible @interface for 'MPSGraph' declares the selector 'splitTensor:splitSizes:axis:name:'
      NSArray<MPSGraphTensor *> * offsets = [_graph splitTensor:keys
                                             ~~~~~~ ^~~~~~~~~~~~~~~~
  ../../src/neural/metal/mps/NetworkGraph.mm:723:21: error: no visible @interface for 'MPSGraph' declares the selector 'stackTensors:axis:name:'
      promo = [_graph stackTensors:stack axis:3 name:[NSString stringWithFormat:@"%@/offset_broadcast", label]];
               ~~~~~~ ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  2 errors generated.

full build log, https://github.com/Homebrew/homebrew-core/actions/runs/3696962365/jobs/6261461141
relates to Homebrew/homebrew-core#118155

[borg323]

It seems splitTensor is only available in macos 12.3 and newer. Until we have a solution you can build with -Dmetal=false -Dmetal=disabled to disable building the metal backend.

[Syahrilquadrat]

Please is not me, can cancel it for me, im syahrilquadrat or syahril.

[rschwiebert]

@borg323 I'm having that same traceback on mac 12.6.7. In what way is it available?

One thing that seemed relevant in my investigation was this in the build output:
Run-time dependency tensorflow_cc found: NO (tried pkgconfig, framework and cmake)

I'm not sure what to do about this. I brew installed tensorflow, locally built and installed the tensorflow_cc project, and still it says that.

BUT your suggestion to add -Dmetal=disabled works like a charm! Thank you!

[borg323]

This has nothing to do with tensorflow, which is optional for lc0 (and quite complicated to build). I have no idea why it fails for you on 12.6.7, but the code in question has changed in the latest lc0 release, v0.30.0 - can you try that?

[rschwiebert]

@borg323 Just changing to that tag and running the same build command, I get 20 errors now:

../../src/lc0ctl/describenet.cc:104:18: error: no member named 'has_smolgen_activation' in 'pblczero::NetworkFormat'
  if (net_format.has_smolgen_activation()) {
      ~~~~~~~~~~ ^
../../src/lc0ctl/describenet.cc:107:28: error: no member named 'smolgen_activation' in 'pblczero::NetworkFormat'
                net_format.smolgen_activation());
                ~~~~~~~~~~ ^
../../src/lc0ctl/describenet.cc:106:28: error: no member named 'ActivationFunction_Name' in 'pblczero::NetworkFormat'
         << NetworkFormat::ActivationFunction_Name(
            ~~~~~~~~~~~~~~~^
../../src/lc0ctl/describenet.cc:109:18: error: no member named 'has_ffn_activation' in 'pblczero::NetworkFormat'
  if (net_format.has_ffn_activation()) {
      ~~~~~~~~~~ ^
../../src/lc0ctl/describenet.cc:111:63: error: no member named 'ffn_activation' in 'pblczero::NetworkFormat'
         << NetworkFormat::ActivationFunction_Name(net_format.ffn_activation());
                                                   ~~~~~~~~~~ ^
../../src/lc0ctl/describenet.cc:111:28: error: no member named 'ActivationFunction_Name' in 'pblczero::NetworkFormat'
         << NetworkFormat::ActivationFunction_Name(net_format.ffn_activation());
            ~~~~~~~~~~~~~~~^
../../src/lc0ctl/describenet.cc:143:9: error: no member named 'encoder_size' in 'pblczero::Weights'
  if (w.encoder_size() > 0) {
      ~ ^
../../src/lc0ctl/describenet.cc:144:38: error: no member named 'encoder_size' in 'pblczero::Weights'
    COUT << Justify("Encoders") << w.encoder_size();
                                   ~ ^
../../src/lc0ctl/describenet.cc:145:43: error: no member named 'headcount' in 'pblczero::Weights'
    COUT << Justify("Encoder heads") << w.headcount();
                                        ~ ^
../../src/lc0ctl/describenet.cc:146:44: error: no member named 'ip_emb_b' in 'pblczero::Weights'
    COUT << Justify("Embedding size") << w.ip_emb_b().params().size() / 2;
                                         ~ ^
../../src/lc0ctl/describenet.cc:147:36: error: no member named 'encoder' in 'pblczero::Weights'
    COUT << Justify("Dmodel") << w.encoder(0).mha().q_b().params().size() / 2;
                                 ~ ^
../../src/lc0ctl/describenet.cc:149:15: error: no member named 'encoder' in 'pblczero::Weights'
         << w.encoder(0).ffn().dense1_b().params().size() / 2;
            ~ ^
../../src/lc0ctl/describenet.cc:168:40: error: no member named 'ACTIVATION_DEFAULT' in 'pblczero::NetworkFormat'
  auto pol_activation = NetworkFormat::ACTIVATION_DEFAULT;
                        ~~~~~~~~~~~~~~~^
../../src/lc0ctl/describenet.cc:172:34: error: no member named 'ffn_activation' in 'pblczero::NetworkFormat'
    auto ffn_activation = format.ffn_activation();
                          ~~~~~~ ^
../../src/lc0ctl/describenet.cc:173:11: error: no member named 'encoder_size' in 'pblczero::Weights'
    if (w.encoder_size() == 0) {
        ~ ^
../../src/lc0ctl/describenet.cc:174:39: error: no member named 'ACTIVATION_SELU' in 'pblczero::NetworkFormat'
      pol_activation = NetworkFormat::ACTIVATION_SELU;
                       ~~~~~~~~~~~~~~~^
../../src/lc0ctl/describenet.cc:175:39: error: no member named 'ACTIVATION_SELU' in 'pblczero::NetworkFormat'
      ffn_activation = NetworkFormat::ACTIVATION_SELU;
                       ~~~~~~~~~~~~~~~^
../../src/lc0ctl/describenet.cc:180:28: error: no member named 'ActivationFunction_Name' in 'pblczero::NetworkFormat'
         << NetworkFormat::ActivationFunction_Name(pol_activation);
            ~~~~~~~~~~~~~~~^
../../src/lc0ctl/describenet.cc:190:30: error: no member named 'ActivationFunction_Name' in 'pblczero::NetworkFormat'
           << NetworkFormat::ActivationFunction_Name(ffn_activation);
              ~~~~~~~~~~~~~~~^
fatal error: too many errors emitted, stopping now [-ferror-limit=]

[rschwiebert]

@borg323 What is supposed to define those two missing things (splitTensor/stacktensors)?

[borg323]

To fix those build errors with v0.30.0, you need to do git submodule update --remote. For the other errors I have no macos experience, but I expect they need a header files update, maybe a newer xcode will help.

[almaudoh]

@rschwiebert you need MacOs 12.3+ to have splitTensor/stackTensors. It should work without issue on MacOs 12.6. Once you've cleared the other errors by cloning the repo afresh or running git submodule update --remote, let me know if you get other errors.

[almaudoh]

Also, @rschwiebert if you upgraded your compute to macos 12.6 from an earlier version, ensure you have also upgraded XCode.

[rschwiebert]

After upgrading to Xcode 14.2 from 13 (I wasn't aware my current os supported it: there was no software update prompt) and being sure to export CPATH=\xcrun --show-sdk-path`/usr/includeand running with./build.sh -Dgtest=false`, I got a successful build. Thanks for the help.