Error when TLS connection not properly closed

[vglfr]

After setting folder.alias.trash = "[Gmail]/Trash" message is deleted successfully but an error is thrown.

Command: himalaya message delete 27301 --trace

Log exert:

2024-12-13T15:09:26.712762Z  INFO moving imap messages 27301 from folder INBOX to folder Trash
2024-12-13T15:09:26.712869Z DEBUG utf7 encoded from folder: INBOX
2024-12-13T15:09:26.712936Z DEBUG utf7 encoded to folder: [Gmail]/Trash
C: a2 SELECT "INBOX"
S: * FLAGS (\Answered \Flagged \Draft \Deleted \Seen $NotPhishing $Phishing Junk NonJunk Old)
S: * OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen $NotPhishing $Phishing Junk NonJunk Old \*)] Flags permitted.
S: * OK [UIDVALIDITY 1] UIDs valid.
S: * 11044 EXISTS
S: * 0 RECENT
S: * OK [UIDNEXT 27302] Predicted next UID.
S: * OK [HIGHESTMODSEQ 4253978]
S: a2 OK [READ-WRITE] INBOX selected. (Success)
C: a3 UID MOVE 27301 "[Gmail]/Trash"
2024-12-13T15:09:27.146942Z  WARN cannot exec imap action: peer closed connection without sending TLS close_notify: https://docs.rs/rustls/latest/rustls/manual/_03_howto/index.html#unexpected-eof, attempt (1)
C: a4 UID MOVE 27301 "[Gmail]/Trash"
2024-12-13T15:09:27.147260Z  WARN cannot exec imap action: peer closed connection without sending TLS close_notify: https://docs.rs/rustls/latest/rustls/manual/_03_howto/index.html#unexpected-eof, attempt (2)
C: a5 UID MOVE 27301 "[Gmail]/Trash"
2024-12-13T15:09:27.147502Z  WARN cannot exec imap action: peer closed connection without sending TLS close_notify: https://docs.rs/rustls/latest/rustls/manual/_03_howto/index.html#unexpected-eof, attempt (3)
C: a6 UID MOVE 27301 "[Gmail]/Trash"
2024-12-13T15:09:27.147695Z  WARN cannot exec IMAP action after 3 attempts, aborting
C: a7 CLOSE
C: a8 LOGOUT
Error:
   0: cannot execute imap action after 3 retries
   1: cannot move imap messages 27301 from folder INBOX to folder [Gmail]/Trash
   2: peer closed connection without sending TLS close_notify: https://docs.rs/rustls/latest/rustls/manual/_03_howto/index.html#unexpected-eof

Location:
   /build/source/src/backend/mod.rs:810

Executing a35 uid move 27302 "[Gmail]/Trash" in openssl s_client throws 406788D0607F0000:error:0A000126:SSL routines::unexpected eof while reading:ssl/record/rec_layer_s3.c:687: and quits repl but the message got deleted.

Executing a32 store 11044 +flags \deleted deletes message successfully.

[soywod]

Just to be sure, which version of Himalaya CLI do you use? IMAP logs look like from the previous library. Make sure you are using the v1.0.0. Could you also share your IMAP config? The fact that you receive a TLS issue after deleting a message and not before (during mailbox selection for example) is really strange.

[vglfr]

himalaya 1.0.0-beta.4 on Nix unstable. Which apparently is April 2024.

Full config:

[accounts.vglfr]
default = true
email = "[email protected]"
display-name = "..."
downloads-dir = "~/Downloads"
folder.alias.trash = "[Gmail]/Trash"
backend = "imap"
message.send.backend = "smtp"
imap.host = "imap.gmail.com"
imap.port = 993
imap.encryption = "tls"
imap.login = "[email protected]"
imap.passwd.raw = "..."
smtp.host = "smtp.gmail.com"
smtp.port = 465
smtp.encryption = "tls"
smtp.login = "[email protected]"
smtp.passwd.raw = "..."

Installed it via Cargo but error persists:

2024-12-19T14:30:14.031524Z DEBUG client::build: email::imap: authentication succeeded! mechanism=Plain
2024-12-19T14:30:14.031888Z  INFO email::email::message::r#move::imap: moving imap messages 27430 from folder INBOX to folder Trash
2024-12-19T14:30:14.031955Z DEBUG email::imap: client 1/1 is free, locking it
2024-12-19T14:30:14.032010Z DEBUG email::email::message::r#move::imap: utf7 encoded from folder: INBOX
2024-12-19T14:30:14.032051Z DEBUG email::email::message::r#move::imap: utf7 encoded to folder: [Gmail]/Trash
2024-12-19T14:30:14.032347Z TRACE select_mailbox{client=1}: imap_client::stream: wrote 27/27 bytes
2024-12-19T14:30:14.209014Z TRACE select_mailbox{client=1}: imap_client::stream: read 408/1024 bytes
2024-12-19T14:30:14.209188Z  WARN select_mailbox{client=1}: imap_codec::response: Rectified missing `text` to "..."
2024-12-19T14:30:14.209230Z  WARN select_mailbox{client=1}: imap_client::tasks::resolver: received unsolicited unsolicited=Status(Untagged(StatusBody { kind: Ok, code: Some(Other(CodeOther(b"HIGHESTMODSEQ 4270937"))), text: Text("...") }))
2024-12-19T14:30:14.209370Z  WARN select_mailbox{client=1}: imap_client::tasks::tasks::select: missing required UNSEEN OK untagged response
2024-12-19T14:30:14.209635Z TRACE move_messages{client=1}: imap_client::stream: wrote 43/43 bytes
Error:
   0: cannot move IMAP message(s)
   1: stream error
   2: peer closed connection without sending TLS close_notify: https://docs.rs/rustls/latest/rustls/manual/_03_howto/index.html#unexpected-eof

Location:
   /home/vglfr/.cargo/registry/src/index.crates.io-6f17d22bba15001f/pimalaya-tui-0.2.1/src/himalaya/backend.rs:670

Version:

himalaya v1.0.0 +smtp +sendmail +wizard +pgp-commands +imap +maildir
build: linux gnu x86_64
git: unknown, rev unknown

Config (when installed via Cargo):

[accounts.vglfr]
default = true
email = "[email protected]"
display-name = "..."
downloads-dir = "/home/vglfr/Downloads"
folder.aliases.trash = "[Gmail]/Trash"
backend.type = "imap"
backend.host = "imap.gmail.com"
backend.port = 993
backend.login = "[email protected]"
backend.encryption.type = "tls"
backend.auth.type = "password"
backend.auth.raw = "..."
message.send.backend.type = "smtp"
message.send.backend.host = "smtp.gmail.com"
message.send.backend.port = 465
message.send.backend.login = "[email protected]"
message.send.backend.encryption.type = "tls"
message.send.backend.auth.type = "password"
message.send.backend.auth.raw = "..."

[soywod]

Please upgrade to the v1.0.0, it is definitely more stable. Either wait for the Nix package to land on unstable (NixOS/nixpkgs#363649), or use directly the Nix derivation available in the repository (you have access to a flake.nix and a default.nix). Be aware that there are breaking changes, see the CHANGELOG. Try again and let me know if the issue is still there.

[vglfr]

The error persists when installed with nix shell github:pimalaya/himalaya. Logs and config exactly as when installed with Cargo.

Version:

himalaya v1.0.0 +maildir +wizard +imap +smtp +pgp-commands +sendmail
build: linux gnu x86_64
git: nix-flake-20241219082013, rev 5eeda248fd90d161c8b346604f603d4a0a37ec6d

[soywod]

Both IMAP lib used (before and now) were based on Rustls, it looks like the issue is beyond Himalaya scope. I do not understand how and why it appears, especially that I do not reproduce it with my own Gmail account. According to Rustls:

Rustls treats an EOF without close_notify as an error of type std::io::Error with ErrorKind::UnexpectedEof. In some situations it’s appropriate for the application to handle this error the same way it would handle a normal EOF (a read returning Ok(0)). In particular if UnexpectedEof occurs on an idle connection it is appropriate to treat it the same way as a clean shutdown. And if an application always uses messages with length framing (in other words, messages are never delimited by the close of the TCP connection), it can unconditionally ignore UnexpectedEof errors from rustls.

So it is not really an error (do you confirm that the message is moved properly?), it is more like a security warning about the connection not properly closed. Not sure if as a lib consumer it is safe to ignore the error. @duesee any opinion?

[vglfr]

do you confirm that the message is moved properly?

Absolutely! In all settings - Cargo, nixpkgs, repo flake.

Please give me a day to test it with another Gmail account.

[soywod]

Since it is more like a warning, it maybe makes sense to treat it as such, by emitting a log instead?

[vglfr]

Log should be appropriate given no other reports.

Same behavior with a second account.

[duesee]

As far as I understand, some implementations don't do a clean TLS shutdown which results in this error/warning. The attack which one can be worried about is a truncation attack. For example, if a Bash script is piped into a shell and an attacker can truncate the script, it may leave a machine in an incosistent state.

For IMAP, I cannot come up with a good attack, though. If an attacker can truncate a message, it will propably always result in a parsing error. Maybe there is a case I miss, but I don't see how to truncate a message in a way that would change it's meaning (ignoring parse errors).

Examples:

<tag> ...\r\n

• No truncation leads to a valid message.

<tag> ...{10}\r\n
abc\r\n
def\r\n

• No truncation (before literal) leads to a valid message.
• No truncation (in literal) leads to a valid message (due to specific length of literal).

Maybe a warning is enough?

[duesee]

What is the worst that can happen if an attacker can truncate at any point? I feel it should be not too much because it would also be a reliability problem.

Non-atomic move comes to mind:

• copy message to new folder
• delete message from old folder

If truncation happens after copy and before delete, we'll end up habing a duplicate message. But... yeah :-)

[soywod]

Awesome, thank you for your replies. Let's go for a simple warning for the next release!

[soywod]

I started to work on this issue, and there is an edge case I'm not sure how to properly handle.

Let's say you move a message. The client sends the request to the server, and the server processes the move but returns a missing close_notify error. You end up with a moved message AND an error, you don't have access to the moved message output. In this particular case it's OK, because moving a message does not produce any output, but what about appending a message to a mailbox? The output is supposed to be Ok(NonZeroU32), but you get Err(std::io::Error). You end up with an appended message that you cannot know the UID.

How to handle this case? My naive intuition tells me to show a warning only when an IMAP task does not produce any output. Otherwise return a better error like "the action may not have been executed due to an unexpected closed stream".

[soywod]

To give more context about the @vglfr moving message issue, the close_notify error occurs just after writing the MOVE command, when reading the output:

https://github.com/pimalaya/imap-client/blob/02f2772983759528c59dc5c2f3a924ae18e5425f/src/stream.rs#L53