Pull Kyber division fixes from PQ-Crystals into dev-092

[praveksharma]

Pull recent fixes made to Kyber from PQ-Crystals into dev-092 branch.

TODO:

• Patch Kyber aarch64 implementation.

Fixes #1645 (see also #1649).

• [No] Does this PR change the input/output behaviour of a cryptographic algorithm (i.e., does it change known answer test values)? (If so, a version bump will be required from x.y.z to x.(y+1).0.)
• [No] Does this PR change the list of algorithms available -- either adding, removing, or renaming? Does this PR otherwise change an API? (If so, PRs in fully supported downstream projects dependent on these, i.e., oqs-provider and OQS-OpenSSH will also need to be ready for review and merge by the time this is merged.)

[SWilson4]

ARM patch done. I also checked out the copy_from_upstream / CI fixes from #1589 which were done after 0.9.0, as the branch isn't idempotent under copy_from_upstream otherwise.

[baentsch]

Where do these McEliece changes come from?

[baentsch]

Why Falcon changes in Kyber DIV fixes?

[baentsch]

Is there a simple explanation for the McEliece and Falcon (at least doc) changes? Code-wise LGTM, apart for the code tracing issue also commented in the "main" PR.

[SWilson4]

Is there a simple explanation for the McEliece and Falcon (at least doc) changes? Code-wise LGTM, apart for the code tracing issue also commented in the "main" PR.

Sorry, I should have expanded on that above. We didn't properly run copy_from_upstream to update documentation before 0.9.0 (#1586), and there was a bug in the script which was fixed in #1589. Since this branch is based on 0.9.1, which was based on 0.9.0, it didn't include the subsequent documentation updates or the copy_from_upstream bug fix. I checked out the post-#1589 version of copy_from_upstream, as well as the fixed CI workflow to check for idempotence, so that the release documentation would be correct.

[baentsch]

Thanks for the additional explanation @SWilson4 ! Makes sense (&explains why the "DIV fix" PR to "main" doesn't include the Falcon/McEliece doc changes). LGTM now.

[SWilson4]

Update for you @dstebila @praveksharma @bhess @baentsch: I ran the full constant-time tests (on dev-092) and oqs-provider release tests (on dev-092 / 092-tracker / OpenSSL master) overnight. The AVX2 CT tests and oqs-provider tests were all green, but the "generic" CT tests had failures for McEliece, Sphincs+, and BIKE. The BIKE failures were expected, since they were patched after 0.9.0. I haven't yet looked through the McEliece and Sphincs+ failures, but it could well be that they are known issues that we've only documented in the AVX2 implementations, since we haven't properly run the "generic" tests on those algorithms in a while. (Ever?)

My position is that we ought to go ahead with 0.9.2 regardless. Both Sphincs+ and McEliece currently have documented constant-time "issues" in our constant_time folder, and the current state of the algorithms is no worse than it was for 0.9.0 or 0.9.1. We can aim to have the "generic" CT tests all green for 0.10.0.

[SWilson4]

Further update: I ran the command grep -r '\/.*KYBER_Q src/kem/kyber on the dev-092 branch and manually inspected the results. There are still a few non-commented / KYBER_Q operations remaining in the source code, but as far as I can tell all of these have constant dividends.

Since that wouldn't catch the case where a division operation involving KYBER_Q was broken onto multiple lines, I also ran grep -r 'KYBER_Q' src/kem/kyber and manually scanned the results for partial expressions. I didn't find any, so I believe that there are indeed no remaining / KYBER_Q operations with secret-dependent dividends.

[stenya]

Hi.

As someone not deeply familiar with KEM algorithm implementations, I am interested in this:

Does this fix affect the backward compatibility of the Kyber algorithm with older versions of the liboqs library?
For instance, if ciphertext was generated by the new liboqs v0.9.2 (server side) but decrypted using the old liboqs v0.8.0 (client side).

Additionally, does liboqs generally maintain backward compatibility with older versions?

[dstebila]

Does this fix affect the backward compatibility of the Kyber algorithm with older versions of the liboqs library? For instance, if ciphertext was generated by the new liboqs v0.9.2 (server side) but decrypted using the old liboqs v0.8.0 (client side).

The fix does not affect the backward compatibility of Kyber.

Additionally, does liboqs generally maintain backward compatibility with older versions?

0.x.y releases are compatible with 0.x.z releases. But changing the second digit indicates that there may be an algorithm whose compatibility has changed. The release notes describe the changes. For Kyber, 0.9.* and 0.8.* are compatible.