Boost_Tests in develop_approvaltests_fetch_content fails on all Linux gcc CI builds

[claremacrae]

Example output:

https://github.com/claremacrae/ApprovalTests.cpp.CMakeSamples/runs/1635245806?check_suite_focus=true

The output is:

Test project /home/runner/work/ApprovalTests.cpp.CMakeSamples/ApprovalTests.cpp.CMakeSamples/develop_approvaltests_fetch_content/build
      Start  1: Boost_Tests
 1/10 Test  #1: Boost_Tests ......................***Failed    0.00 sec
Running 2 test cases...

*** No errors detected
Test setup error: 

[claremacrae]

Comparing the logs of the clang10 and gcc10 builds on Linux, the following stands out, in gcc10 (failing) build:

CC and CXX: gcc-10, g++-10 
Found gcc 10
gcc>=5, using the major as version

************************* WARNING: GCC OLD ABI COMPATIBILITY ***********************
 
Conan detected a GCC version > 5 but has adjusted the 'compiler.libcxx' setting to
'libstdc++' for backwards compatibility.
Your compiler is likely using the new CXX11 ABI by default (libstdc++11).

If you want Conan to use the new ABI for the default profile, run:

    $ conan profile update settings.compiler.libcxx=libstdc++11 default

Or edit '/home/runner/.conan/profiles/default' and set compiler.libcxx=libstdc++11

************************************************************************************

The test outputs are:

clang 10

1: Test timeout computed to be: 10000000
1: Running 2 test cases...
1:
1: ESC[1;32;49m*** No errors detected
1: ESC[0;39;49m
 1/12 Test  #1: Boost_Tests ......................   Passed    0.00 sec

gcc 10

1: Test timeout computed to be: 10000000
1: Running 2 test cases...
1:
1: ESC[1;32;49m*** No errors detected
1: ESC[0;39;49mTest setup error:
 1/12 Test  #1: Boost_Tests ......................***Failed    0.00 sec

[claremacrae]

I can reproduce this on my Mac:

/Users/clare/Documents/develop/ApprovalTests/ApprovalTests.cpp.CMakeSamples/develop_approvaltests_fetch_content/cmake-build-default-g-9-brew/approvaltests.cpp_build/tests/Boost_Tests/Boost_Tests \
   --logger=HRF,all --color_output=false --report_format=HRF --show_progress=no
Testing started at 13:02 ...

Running 2 test cases...
Test setup error: 
*** No errors detected
Process finished with exit code 200

[claremacrae]

The above was obtained from running with CLion's Boost Test integration...

When I run the exe directly, I get this output:

/Users/clare/Documents/develop/ApprovalTests/ApprovalTests.cpp.CMakeSamples/develop_approvaltests_fetch_content/cmake-build-default-g-9-brew/approvaltests.cpp_build/tests/Boost_Tests/Boost_Tests

*** No errors detected
Test setup error: 
Running 2 test cases...

Process finished with exit code 200

[claremacrae]

Exit code 200 is:

const int exit_exception_failure = 200; // otherwise uncaught exception

[claremacrae]

By pruning code out, I've found that when this register_observer() function is called, the code crashes. If I don't call this, it doesn't crash.

    int register_our_listener(BoostApprovalListener& t)
    {
        boost::unit_test::framework::register_observer(t);  // <<< Calling this triggers the crash somehow.
        return 1;
    }

I'm guessing it's an object lifetime issue... Or could it be static order of initialisation fiasco?

[claremacrae]

I've tried various Boost versions with gcc 9 on my Mac, and observe the following:

• boost_1_71_0 - exits with status code 0
• boost_1_72_0 - exits with status code 0
• boost_1_73_0 - exits with status code 200
• boost_1_74_0 - exits with status code 200
• boost_1_75_0 - exits with status code 200

The version currently downloaded in this project is 1_74_0

[claremacrae]

Am thinking I should log a bug for this, or at least see if anyone else has done so...

https://github.com/boostorg/test/issues