New Cloudflare test "kills" other tests

[mouse07410]

Describe the bug
Newly-added "External inter tests" totally fail to work on machines behind a firewall and Web proxy - unfortunately bringing all the other tests down with them.

To Reproduce
Steps to reproduce the behavior:

1. Clone and build this package on a machine that's behind a corporate firewall.
2. Run scripts/runtests.sh
3. See error

.  .  .
  oqsprovider
    name: OpenSSL OQS Provider
    version: 0.5.2-dev
    status: active
    build info: OQS Provider v.0.5.2-dev (a589273) based on liboqs v.0.9.0
    gettable provider parameters:
      name: pointer to a UTF8 encoded string (arbitrary size)
      version: pointer to a UTF8 encoded string (arbitrary size)
      buildinfo: pointer to a UTF8 encoded string (arbitrary size)
      status: integer (arbitrary size)
Cert gen/verify, CMS sign/verify, CA tests for all enabled OQS signature algorithms commencing: 
.......................
External interop tests commencing
 Cloudflare:
^C

Expected behavior

Cert gen/verify, CMS sign/verify, CA tests for all enabled OQS signature algorithms commencing: 
.......................
External interop tests commencing
no, we disabled them for now, as they fail to detect Web proxy...
Test project /Users/ur20980/src/oqs-provider/_build
    Start 1: oqs_signatures
1/5 Test #1: oqs_signatures ...................   Passed    2.57 sec
    Start 2: oqs_kems
2/5 Test #2: oqs_kems .........................   Passed    0.24 sec
    Start 3: oqs_groups
3/5 Test #3: oqs_groups .......................   Passed    0.35 sec
    Start 4: oqs_tlssig
4/5 Test #4: oqs_tlssig .......................   Passed    0.01 sec
    Start 5: oqs_endecode
5/5 Test #5: oqs_endecode .....................   Passed    4.85 sec

100% tests passed, 0 tests failed out of 5

Total Test time (real) =   8.02 sec

All oqsprovider tests passed.

Environment (please complete the following information):

• OS: MacOS Sonoma 14.0
• OpenSSL version: 3.1.3
• oqsprovider version: master

Please run the following commands to obtain the version information:

• For OpenSSL: openssl version
• For oqsprovider: openssl list -providers

$ openssl version
OpenSSL 3.1.3 19 Sep 2023 (Library: OpenSSL 3.1.3 19 Sep 2023)
$ openssl list -providers
Providers:
  base
    name: OpenSSL Base Provider
    version: 3.1.3
    status: active
  default
    name: OpenSSL Default Provider
    version: 3.1.3
    status: active
  legacy
    name: OpenSSL Legacy Provider
    version: 3.1.3
    status: active
  oqs
    name: OpenSSL OQS Provider
    version: 0.5.2-dev
    status: active
  pkcs11
    name: PKCS#11 Provider
    version: 3.1.3
    status: active
$ 

Additional context
As far as I understand, the cause of the problem is that the tests, and probably OpenSSL by itself, does not recognize and deal with Web proxy, pointed at by HTTP_PROXY= and HTTPS_PROXY= env vars.

It means that either the "External interior" test-runs should be conditional on, e.g., absence of HTTP_PROXY variable, or somehow make the test correctly deal with Web proxy.

[baentsch]

the "External interior" test-runs should be conditional on, e.g., absence of HTTP_PROXY variable, or somehow make the test correctly deal with Web proxy.

Makes perfect sense. Easiest would be to check for presence of "HTTP_PROXY" env var and disable the test in such case. But disabling tests should be the last option: Could I ask you to check whether the openssl s_client -proxy* command line parameter(s) could be utilized to work in your environment? I don't have such environment at my disposal to test/fix. As usual: PR very welcome!

[mouse07410]

Could I ask you to check whether the openssl s_client -proxy* command line parameter(s) could be utilized to work in your environment?

Sure. It works - see the included patch.

As usual: PR very welcome!

:-)

How about this (tested on my machines):

diff --git a/scripts/oqsprovider-externalinterop.sh b/scripts/oqsprovider-externalinterop.sh
index d90dff6..fb75275 100755
--- a/scripts/oqsprovider-externalinterop.sh
+++ b/scripts/oqsprovider-externalinterop.sh
@@ -18,9 +18,15 @@ if [ -z "$DYLD_LIBRARY_PATH" ]; then
     export DYLD_LIBRARY_PATH=$LD_LIBRARY_PATH
 fi
 
+if [ ! -z "${HTTP_PROXY}" ]; then
+    export USE_PROXY="-proxy ${HTTP_PROXY#http://} -allow_proxy_certs"
+else
+    export USE_PROXY=""
+fi
+
 echo " Cloudflare:"
 export OQS_CODEPOINT_X25519_KYBER512=65072
-(echo -e "GET /cdn-cgi/trace HTTP/1.1\nHost: cloudflare.com\n\n"; sleep 1; echo $'\cc') | $OPENSSL_APP s_client -connect pq.cloudflareresearch.com:443 -groups x25519_kyber768 -servername cloudflare.com -ign_eof 2>/dev/null | grep kex=X25519Kyber768Draft00
-(echo -e "GET /cdn-cgi/trace HTTP/1.1\nHost: cloudflare.com\n\n"; sleep 1; echo $'\cc') | $OPENSSL_APP s_client -connect pq.cloudflareresearch.com:443 -groups x25519_kyber512 -servername cloudflare.com -ign_eof 2>/dev/null | grep kex=X25519Kyber512Draft00
+(echo -e "GET /cdn-cgi/trace HTTP/1.1\nHost: cloudflare.com\n\n"; sleep 1; echo $'\cc') | "${OPENSSL_APP}" s_client ${USE_PROXY} -connect pq.cloudflareresearch.com:443 -groups x25519_kyber768 -servername cloudflare.com -ign_eof 2>/dev/null | grep kex=X25519Kyber768Draft00
+(echo -e "GET /cdn-cgi/trace HTTP/1.1\nHost: cloudflare.com\n\n"; sleep 1; echo $'\cc') | "${OPENSSL_APP}" s_client ${USE_PROXY} -connect pq.cloudflareresearch.com:443 -groups x25519_kyber512 -servername cloudflare.com -ign_eof 2>/dev/null | grep kex=X25519Kyber512Draft00

[baentsch]

Thanks! I could change #283 with this code -- but wouldn't it be more accurate in terms of attribution if you do a PR with this and I close #283?