Choose between ring and aws-lc-rs

[epompeii]

Rustls has chosen to move from ring to aws-lc as their default cryptography backend: https://github.com/rustls/rustls/releases/tag/v%2F0.23.0

This caused build failures on Windows: rustls/rustls#1913

The next time that I went to bump dependencies, this issue was resolved, so I updated tokio-rustls from 0.25.0 to 0.26.0 which meant I could then update mail-send from 0.4.8 to 0.4.9. Everything passed in CI and was deployed to production.

It wasn't until almost a day later that I noticed emails were not being sent. Looking at the logs, I found this:

thread 'tokio-runtime-worker' panicked at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/rustls-0.23.15/src/crypto/mod.rs:248:14:
no process-level CryptoProvider available -- call CryptoProvider::install_default() before this point

Full log output

2024-10-26T08:13:49.645 app[2874223f340278] iad [info] Oct 26 08:13:49.645 INFO accepted connection, remote_addr: 172.16.3.42:43518, local_addr: 0.0.0.0:61016

2024-10-26T08:13:49.645 app[2874223f340278] iad [info] Oct 26 08:13:49.645 INFO request completed, latency_us: 70, response_code: 200, uri: /v0/auth/login, method: OPTIONS, req_id: 0ca766ec-3d83-405b-9fda-a3aa48106279, remote_addr: 172.16.3.42:43518, local_addr: 0.0.0.0:61016

2024-10-26T08:13:49.659 app[2874223f340278] iad [info] Oct 26 08:13:49.659 INFO accepted connection, remote_addr: 172.16.3.42:43530, local_addr: 0.0.0.0:61016

2024-10-26T08:13:49.681 app[2874223f340278] iad [info] thread 'tokio-runtime-worker' panicked at /usr/local/cargo/registry/src/index.crates.io-6f17d22bba15001f/rustls-0.23.15/src/crypto/mod.rs:248:14:

2024-10-26T08:13:49.681 app[2874223f340278] iad [info] no process-level CryptoProvider available -- call CryptoProvider::install_default() before this point

2024-10-26T08:13:49.681 app[2874223f340278] iad [info] Oct 26 08:13:49.681 INFO request completed, latency_us: 21498, response_code: 202, uri: /v0/auth/login, method: POST, req_id: a3619be9-c6ff-4fec-98c9-82c0ff813d2e, remote_addr: 172.16.3.42:43530, local_addr: 0.0.0.0:61016 

Doing some digging, it looks like the Windows build issue wasn't the only problem. This is going to be an ongoing, ecosystem wide headache going forward.

If the feature configuration of your project is such that only one of rustls/ring or rustls/aws-lc-rs (the default) are enabled then the interfaces that work on the basis of there being a default provider will operate without code change. You only need to explicitly register a process-wide default if you're using the interfaces that require a default and the features are ambiguous.

algesten/ureq#765 (comment)

Taking a look, tokio-rustls 0.26.0 moved to using aws-lc as the default: https://github.com/rustls/tokio-rustls/blob/82b4a3e8c09fc362edc09a49b9b84fe609cc9842/Cargo.toml#L21

However, mail-send 0.4.9 chose to keep ring as the default: https://github.com/stalwartlabs/mail-send/blob/980a9271c30caa7252948792f70f2dd0105158a4/Cargo.toml#L37

There is ambiguity here on which to choose, so rustls panics at runtime.

I looked into following the suggestion in the panic message and calling CryptoProvider::install_default(): https://docs.rs/rustls/0.23.15/rustls/crypto/struct.CryptoProvider.html#method.install_default

In order to call this method, we need to choose between ring and aws-lc:

• ring: https://docs.rs/rustls/0.23.15/rustls/crypto/ring/fn.default_provider.html
• aws_lc_rs: https://docs.rs/rustls/0.23.15/rustls/crypto/aws_lc_rs/fn.default_provider.html

See also: https://rustls.github.io/rustls/prerelease/crypto/struct.CryptoProvider.html#using-the-per-process-default-cryptoprovider

We could try fixing this issue by just setting all of the feature flags for all crates that use rustls to our selection of ring or aws-lc. However, it is a bit unsettling that if any crate ever makes a change that makes this ambiguous going forward then we get a runtime panic. Therefore, I don't think aligning the feature flags is enough. We will need to call CryptoProvider::install_default() at the start of the API server and CLI in order to protect against runtime panics.