Feature probes incorrectly fail on AL2 due to zlib issues

[toidiu]

Problem:

Some feature probes (eg. S2N_LIBCRYPTO_SUPPORTS_EVP_MD5_SHA1_HASH) fail with a zlib error in the s2nUnitAl2Arm and s2nUnitAl2Openssl111 CI test. This means we are not correctly testing some the feature probes on AL2 platforms.

I was able to see the error if I uncomment the message line in the cmake file.

zlib Failure:

Run Build Command(s):/usr/bin/gmake cmTC_ff5a3/fast && /usr/bin/gmake  -f CMakeFiles/cmTC_ff5a3.dir/build.make CMakeFiles/cmTC_ff5a3.dir/build
gmake[1]: Entering directory `/codebuild/output/src2287290689/src/github.com/aws/s2n-tls/build/CMakeFiles/CMakeTmp'
Building C object CMakeFiles/cmTC_ff5a3.dir/S2N_LIBCRYPTO_SUPPORTS_FLAG_NO_CHECK_TIME.c.o
/usr/bin/cc   -I /codebuild/output/src2287290689/src/github.com/aws/s2n-tls -include /codebuild/output/src2287290689/src/github.com/aws/s2n-tls/utils/s2n_prelude.h -c -Werror-implicit-function-declaration -Wno-unused-variable -pthread -o CMakeFiles/cmTC_ff5a3.dir/S2N_LIBCRYPTO_SUPPORTS_FLAG_NO_CHECK_TIME.c.o   -c /codebuild/output/src2287290689/src/github.com/aws/s2n-tls/tests/features/S2N_LIBCRYPTO_SUPPORTS_FLAG_NO_CHECK_TIME.c
Linking C executable cmTC_ff5a3
/usr/bin/cmake3 -E cmake_link_script CMakeFiles/cmTC_ff5a3.dir/link.txt --verbose=1
/usr/bin/cc      -rdynamic CMakeFiles/cmTC_ff5a3.dir/S2N_LIBCRYPTO_SUPPORTS_FLAG_NO_CHECK_TIME.c.o  -o cmTC_ff5a3  /usr/lib64/libcrypto.a -ldl -lrt -pthread 
/usr/lib64/libcrypto.a(c_zlib.o): In function `zlib_stateful_expand_block':
(.text+0x4c): undefined reference to `inflate'
/usr/lib64/libcrypto.a(c_zlib.o): In function `zlib_stateful_compress_block':
(.text+0xc8): undefined reference to `deflate'
/usr/lib64/libcrypto.a(c_zlib.o): In function `zlib_stateful_finish':
(.text+0x114): undefined reference to `inflateEnd'
(.text+0x11c): undefined reference to `deflateEnd'
/usr/lib64/libcrypto.a(c_zlib.o): In function `zlib_stateful_init':
(.text+0x1e8): undefined reference to `inflateInit_'
(.text+0x23c): undefined reference to `deflateInit_'
/usr/lib64/libcrypto.a(c_zlib.o): In function `bio_zlib_ctrl':
(.text+0x414): undefined reference to `deflate'
(.text+0x5cc): undefined reference to `zError'
/usr/lib64/libcrypto.a(c_zlib.o): In function `bio_zlib_write':
(.text+0x6e8): undefined reference to `deflate'
(.text+0x770): undefined reference to `zError'
(.text+0x7fc): undefined reference to `deflateInit_'
/usr/lib64/libcrypto.a(c_zlib.o): In function `bio_zlib_read':
(.text+0x904): undefined reference to `inflate'
(.text+0x968): undefined reference to `zError'
(.text+0x9ec): undefined reference to `inflateInit_'
/usr/lib64/libcrypto.a(c_zlib.o): In function `bio_zlib_free':
(.text+0xa70): undefined reference to `inflateEnd'
(.text+0xa90): undefined reference to `deflateEnd'
collect2: error: ld returned 1 exit status
gmake[1]: *** [cmTC_ff5a3] Error 1
gmake[1]: Leaving directory `/codebuild/output/src2287290689/src/github.com/aws/s2n-tls/build/CMakeFiles/CMakeTmp'
gmake: *** [cmTC_ff5a3/fast] Error 2

Investigation:

The s2nUnitAL2, even though it is AL2, does not have zlib errors. One difference I detected was which libcrypto we linked against.

# Fails: s2nUnitAl2Arm and s2nUnitAl2Openssl111
S2N_LIBCRYPTO=openssl-1.1.1

# Passes: s2nUnitAL2
S2N_LIBCRYPTO=default

• Is AL2 missing zlib? Can we add zlib?
• Does openssl 1.1.1 require zlib? (can we build it without zlib)?
• What libcrypto do we use in s2nUnitAL2, when it is default. Why doesnt it error on zlib?

Resolution:

• We should fix the zlib issue for AL2.
• Add a feature probe test for S2N_LIBCRYPTO_SUPPORTS_ENGINE feat: feature probe S2N_LIBCRYPTO_SUPPORTS_ENGINE #4878

// S2N_LIBCRYPTO_SUPPORTS_ENGINE feature probe test

    {
        if (s2n_libcrypto_is_openssl() && !s2n_is_in_fips_mode()) {
#if !defined(OPENSSL_NO_ENGINE) && !defined(S2N_LIBCRYPTO_SUPPORTS_ENGINE)
            FAIL_MSG("Expected ENGINE feature probe to be enabled");
#endif
        }
    };

[dougch]

Is AL2 missing zlib ?

Zlib, including headers are installed here: https://github.com/aws/s2n-tls/blob/main/codebuild/bin/install_al_dependencies.sh#L51

What libcrypto is default
AL2 shipped with openssl102

Might these BUILD_FLAGS be part of the issue?

    # AL2 case; Linker flags are a workaround for system openssl
    if [[ ${VERSION_ID} == '2' ]]; then
       BUILD_FLAGS=$(echo -e '-DCMAKE_EXE_LINKER_FLAGS="-lcrypto -lz" \
         -DCMAKE_EXPORT_COMPILE_COMMANDS=ON')
    fi