Extend tlsfuzzer coverage

[Jakuje]

Description

This mostly extends the test coverage for TLS 1.2 and ECDSA and EdDSA keys. I did not touch the RSA-PSS coverage, as it needs to use the special RSA-PSS keys and certificates to work correctly and pkcs11-provider can not use them. This will likely be handled in some follow-up PR.

Checklist

• Test suite updated with functionality tests
• Test suite updated with negative tests

Reviewer's checklist:

• Any issues marked for closing are addressed
• There is a test suite reasonably covering new functionality or modifications
• This feature/change has adequate documentation added
• Code conform to coding style that today cannot yet be enforced via the check style test
• Commits have short titles and sensible commit messages
• Coverity Scan has run if needed (code PR) and no new defects were found

[simo5]

LGTM!

Covscan not needed.

[tomato42]

there are multiple scripts missing compared to instructions from openssl/openssl#25724

also, I think it would be nice to check that PKCS#11 can be used to delegate verification of signatures too...?

[Jakuje]

also, I think it would be nice to check that PKCS#11 can be used to delegate verification of signatures too...?

AFAIK this is implicitly tested by adding the propquery to the openssl cli in the second invocation of the tests:

https://github.com/latchset/pkcs11-provider/blob/6be8f7aedc119d5ceb7b9e4db5c4df8b73ee46af/tests/ttlsfuzzer#L81C1-L81C6

This should force the OpenSSL to import even public keys to the pkcs11 provider and do the operations on them. This already caught us some architectural issues in the past that have been fixed.

[Jakuje]

there are multiple scripts missing compared to instructions from openssl/openssl#25724

For RSA, I added test-dhe-rsa-key-exchange-signatures.py, but again, the SHA1 signatures are failing per default c-p so I can add that, but will have to skip these:

test-dhe-rsa-key-exchange-signatures.py:stdout: 'TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 sha1 signature'
test-dhe-rsa-key-exchange-signatures.py:stdout: 'TLS_DHE_RSA_WITH_AES_128_CBC_SHA sha1 signature'
test-dhe-rsa-key-exchange-signatures.py:stdout: 'TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 sha1 signature'
test-dhe-rsa-key-exchange-signatures.py:stdout: 'TLS_DHE_RSA_WITH_AES_256_CBC_SHA sha1 signature'

The test test-signature-algorithms.py again talks about SHA-1 signatures. Is there any point in keeping it or ok to skip altogether?

[tomato42]

The test test-signature-algorithms.py again talks about SHA-1 signatures. Is there any point in keeping it or ok to skip altogether?

if there are other cases that do pass, yes, modifying it with -x ... -X ... to expect failures is completely valid way to use those scripts

[tomato42]

also, I think it would be nice to check that PKCS#11 can be used to delegate verification of signatures too...?

AFAIK this is implicitly tested by adding the propquery to the openssl cli in the second invocation of the tests:

https://github.com/latchset/pkcs11-provider/blob/6be8f7aedc119d5ceb7b9e4db5c4df8b73ee46af/tests/ttlsfuzzer#L81C1-L81C6

This should force the OpenSSL to import even public keys to the pkcs11 provider and do the operations on them. This already caught us some architectural issues in the past that have been fixed.

ok, but isn't the point of those tests to ensure end-to-end support for that, in the TLS context specifically?

[Jakuje]

These should work, but do not for some reason. Running this separately gives the following:

(server)
$ openssl s_server -www -port 9999 -key "${ECPRIURI}"  -cert "${ECCRTURI}" -trace
(client)
$ PYTHONPATH=. python3 scripts/test-signature-algorithms.py -n 0 --ecdsa -p 9999 "explicit SHA-256+RSA or ECDSA"

(server log)
Using default temp DH parameters
ACCEPT
Received TLS Record
Header:
  Version = SSL 3.0 (0x300)
  Content Type = Handshake (22)
  Length = 91
    ClientHello, Length=87
      client_version=0x303 (TLS 1.2)
      Random:
        gmt_unix_time=0x00000000
        random_bytes (len=28): 00000000000000000000000000000000000000000000000000000000
      session_id (len=0): 
      cipher_suites (len=8)
        {0xC0, 0x09} TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
        {0xC0, 0x13} TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA
        {0x00, 0x33} TLS_DHE_RSA_WITH_AES_128_CBC_SHA
        {0x00, 0xFF} TLS_EMPTY_RENEGOTIATION_INFO_SCSV
      compression_methods (len=1)
        No Compression (0x00)
      extensions, length = 38
        extension_type=signature_algorithms(13), length=4
          ecdsa_secp256r1_sha256 (0x0403)
        extension_type=signature_algorithms_cert(50), length=18
          0000 - 00 10 06 03 05 03 04 03-03 03 02 03 08 1a 08   ...............
          000f - 1b 08 1c                                       ...
        extension_type=supported_groups(10), length=4
          secp256r1 (P-256) (23)

Sent TLS Record
Header:
  Version = TLS 1.2 (0x303)
  Content Type = Alert (21)
  Length = 2
    Level=fatal(2), description=handshake failure(40)

80924C670C7F0000:error:0A000076:SSL routines:tls_choose_sigalg:no suitable signature algorithm:ssl/t1_lib.c:3862:

[Jakuje]

@tomato42 This looks like the OpenSSL is looking at the signature in the certificate and given that we have RSA CA and EC key in the TLS server, it contains the Signature Algorithm: sha256WithRSAEncryption, which makes this connection to fail.

Tried the same test with ECDSA CA key and these work ok, but other bunch of RSA tests start failing when we have EC CA and RSA host key:

test-tls13-rsa-signatures.py:stdout:FAILED:
test-tls13-rsa-signatures.py:stdout:    'sanity'
test-tls13-rsa-signatures.py:stdout:    'sanity'
test-tls13-rsa-signatures.py:stdout:    'tls13 signature rsa_pss_rsae_sha256'
test-tls13-rsa-signatures.py:stdout:    'tls13 signature rsa_pss_rsae_sha384'
test-tls13-rsa-signatures.py:stdout:    'tls13 signature rsa_pss_rsae_sha512'

test-tls13-signature-algorithms.py:stdout:FAILED:
test-tls13-signature-algorithms.py:stdout:      'sanity'
test-tls13-signature-algorithms.py:stdout:      'sanity'
test-tls13-signature-algorithms.py:stdout:      'tolerance 32715 methods with sig_alg_cert'
test-tls13-signature-algorithms.py:stdout:      'tolerance legacy RSA PKCS#1.5'
test-tls13-signature-algorithms.py:stdout:      'tolerance unallocated 0x0A01 (10+RSA) method'
test-tls13-signature-algorithms.py:stdout:      'unique and well-known sig_algs, rsa algorithms last'

test-sig-algs.py:stdout:FAILED:
test-sig-algs.py:stdout:        'extra sigalgs'
test-sig-algs.py:stdout:        'MD5 first'
test-sig-algs.py:stdout:        'MD5 first, no SHA-1'
test-sig-algs.py:stdout:        'no SHA-1'
test-sig-algs.py:stdout:        'rsa_pss_rsae_sha256 only'
test-sig-algs.py:stdout:        'rsa_pss_rsae_sha384 only'
test-sig-algs.py:stdout:        'rsa_pss_rsae_sha512 only'
test-sig-algs.py:stdout:        'sanity'
test-sig-algs.py:stdout:        'sanity'

And some of the other tests do not seem to work at all.

Is the tlsfuzzer usage limited to use with self-signed certificates or certificates signed with the same key type? Or is there some bug in tlsfuzzer or openssl to cover the combination?

I do not think we want to change the whole testsuite to have self-signed certificates or to have different CA certificate chains just for tlsfuzzer @simo5 so I think this state is the best we can get now.

[tomato42]

there is no explicit policy for tlsfuzzer that the certificates have to be self-signed or anything like that

if openssl ignores signature_algorithms_cert and uses signatures_algorithms for selection of the server certificate, and we want to have heterogenous keys in the certificate chain, then the only option is to modify the test script to send both sha256+rsa and sha256+ecdsa, but then expect only one of the (depending on the --ecdsa option) in the ServerKeyExchange

all of that is already supported, just the test script needs to be updated

[simo5]

Ok I think we should merge as is then, and then will have a folllwup PR once the tlsfuzzer script can be updated to handle these cases.

[Jakuje]

Fine with me! Not sure if there is some followup related to the other comment from Alicja, but I think this should be good enough for now.

[tomato42]

I'd say there is... to have it actually tested we need to either have homogenous CA and server keys for the RSA and ECDSA cases, or update the tlsfuzzer scripts

I suggest the former.

but I don't think it's done unless either of those options is followed

[simo5]

LGTM