general OQS update

[baentsch]

General update to "everything OQS":

• Removal of support for EOL (oqs-)openssl111
• Generating new artifacts and storing them in ZIP again: (oqsprovider)/artifacts.zip
• moved old "artifacts.zip" to suitably named files should anyone want to test against those
• Improved certificate generation to include critical R3 extensions
• Added logic to generate CSV "compatMatrices" files
• Added "cross_verify" results
• Added logic to generate OCSP certificates (and test it via an OCSP responder)

Open issues:

• How to test (and report) OCSP results?
  - Collating the compatMatrices results into overall results file: Is there a script for this @johngray-dev ?
• Testing other implementations with these new artifacts: I wasn't able to run other "gen.sh" scripts (see Failure running check.sh in corey-digicert #63, Failure running make cross_verify in oqs-gnutls #64) -- but didn't go to the length of building implementations from scratch to try this. Thanks @CBonnell and @ueno for making available docker images.
• Evaluate results more deeply: It seems interop with digicert and BC is pretty OK; everything else seems to be bad.

[baentsch]

@johngray-dev @ounsworth @dghgit @CBonnell What is the "agreed-upon" merge strategy of PRs in this repo? Would anyone (of you?) want to check this (against your implementations?) before merging or should I just merge as-is? It would be good if new contributions (like yours, @dghgit) would be able to use this latest OQS data...

[dghgit]

I've had a look - it looks like OQS isn't up to date with https://github.com/pq-crystals/dilithium/tree/standard - the difference is pretty small the length of tr has increased, but it does change both the keys and signatures. The change was made about two weeks ago.

It looks like an issue has already been raised for this. @johngray-dev I think they previously changed OIDs on this, so they might again. We won't be changing the OID for kyber - it needs to be whatever the NIST official standard is (a failure to do so may result in patent trolling in the future (sigh)).

[baentsch]

Thanks @dghgit for taking a look!

it looks like OQS isn't up to date with https://github.com/pq-crystals/dilithium/tree/standard

This is expected as (lib)oqs is not following all upstream changes immediately (mistakes are not unheard of :) But your comment has me point some OQS colleagues to this issue as this may (IMO should) impact the OQS release strategy:
@bhess By when do you plan to have run "copy_from_upstream" again? Against "master" or "standard" branch? @SWilson4, @dstebila: Do we really want to wait for the ARM-optimizations before including this update in liboqs?

All that said, I do expect the IBM team to change OIDs due to this, so this PR therefore seems valid as per the OID encoded.

[dghgit]

Thanks for the confirmation. I've closed my pull request, I'll introduce a new update when the OIDs for Dilithium are updated.

[dstebila]

This is expected as (lib)oqs is not following all upstream changes immediately (mistakes are not unheard of :) But your comment has me point some OQS colleagues to this issue as this may (IMO should) impact the OQS release strategy: @bhess By when do you plan to have run "copy_from_upstream" again? Against "master" or "standard" branch? @SWilson4, @dstebila: Do we really want to wait for the ARM-optimizations before including this update in liboqs?

I don't want to have the ARM and Intel builds of the same library to be incompatible. If we don't have the ARM versions we could go with just unoptimized C code on ARM until it's available. But I think actually the ARM code will come rather quickly, so we may not have to wait too long.

[baentsch]

I don't want to have the ARM and Intel builds of the same library to be incompatible.

Me neither. But I don't see how this can happen: The new code will introduce a completely new (algorithm) name and OID, right? Once it's been added, it should work on all platforms (via generic code) and optimizations (will work just faster but producing the same KATs) later as and when available. Dilithium either ceases to exist or continues with old code and old OID but its use should end, no?

[dstebila]

I don't want to have the ARM and Intel builds of the same library to be incompatible.

Me neither. But I don't see how this can happen: The new code will introduce a completely new (algorithm) name and OID, right? Once it's been added, it should work on all platforms (via generic code) and optimizations (will work just faster but producing the same KATs) later as and when available. Dilithium either ceases to exist or continues with old code and old OID but its use should end, no?

True, assuming we introduce a new algorithm name for the FIPS versions. And if we do, then indeed we would presumably be dropping the things named Kyber and Dilithium right now as well.

[baentsch]

assuming we introduce a new algorithm name for the FIPS versions.

I'd vote for doing so sooner than later. FWIW, Rijndael changed (at least in OpenSSL) to AES (on Feb 7, 2001), before NIST published the final standard (November 26, 2001).

[dghgit]

We might need to wait in case NIST do what they did with SHA-3, there was a last minute tweak, there seems to be a lot of interest in trying these algorithms out, I think a move to "production names" might be one temptation too far.

Speaking of tweaks, just an additional heads up, the draft SP 800-204 included a comment in Section 1.3.2 about a change to c~ as well as tr. This was also queried on the PQC mailing list last night. The change only effects signatures. This change has now been made in

https://github.com/pq-crystals/dilithium/tree/standard

Bouncy Castle has also replicated this change and pushed it to our github repositories for Java and C#. It's also now in BC 1.77 beta 5.

[baentsch]

@johngray-dev any objections to merging this as-is -- so it can be used in #69 (should there be interest for that)?

[johngray-dev]

No objects to Merging this as is so it can be used with #69. John Gray From: Michael Baentsch ***@***.***> Sent: Tuesday, August 29, 2023 11:07 AM To: IETF-Hackathon/pqc-certificates ***@***.***> Cc: John Gray ***@***.***>; Mention ***@***.***> Subject: [EXTERNAL] Re: [IETF-Hackathon/pqc-certificates] general OQS update (PR #65) @johngray-dev any objections to merging this as-is -- so it can be used in #69 (should there be interest for that)? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned. Message @johngray-dev<https://urldefense.com/v3/__https:/github.com/johngray-dev__;!!FJ-Y8qCqXTj2!ZpjHHzWsYU1sazGeTQD1IhyL0BXmPJbbtMDygbNG-ij2hy_vM5fMAJJOR3wAQNMqEFBUcFAnc-HuCcjupoG7CbCU37eRA-M$> any objections to merging this as-is -- so it can be used in #69<https://urldefense.com/v3/__https:/github.com/IETF-Hackathon/pqc-certificates/pull/69__;!!FJ-Y8qCqXTj2!ZpjHHzWsYU1sazGeTQD1IhyL0BXmPJbbtMDygbNG-ij2hy_vM5fMAJJOR3wAQNMqEFBUcFAnc-HuCcjupoG7CbCUeHxp8q8$> (should there be interest for that)? — Reply to this email directly, view it on GitHub<https://urldefense.com/v3/__https:/github.com/IETF-Hackathon/pqc-certificates/pull/65*issuecomment-1697629591__;Iw!!FJ-Y8qCqXTj2!ZpjHHzWsYU1sazGeTQD1IhyL0BXmPJbbtMDygbNG-ij2hy_vM5fMAJJOR3wAQNMqEFBUcFAnc-HuCcjupoG7CbCUaubBYz0$>, or unsubscribe<https://urldefense.com/v3/__https:/github.com/notifications/unsubscribe-auth/ANFGAWPR625DOWJ2H7IRN4LXXYASRANCNFSM6AAAAAA3GPEVXU__;!!FJ-Y8qCqXTj2!ZpjHHzWsYU1sazGeTQD1IhyL0BXmPJbbtMDygbNG-ij2hy_vM5fMAJJOR3wAQNMqEFBUcFAnc-HuCcjupoG7CbCU1TsKpTM$>. You are receiving this because you were mentioned.Message ID: ***@***.******@***.***>> Any email and files/attachments transmitted with it are intended solely for the use of the individual or entity to whom they are addressed. If this message has been sent to you in error, you must not copy, distribute or disclose of the information it contains. Please notify Entrust immediately and delete the message from your system.