First attempt at supporting encrypted debug_info

[voughtdq]

Would close #1928

A few things need to be resolved before this can be changed from a draft:

1. There are two places where calling :beam_lib.crypto_key_fun/1 would make sense: the Retriever module or in the individual language modules. Based on a cursory glance of the Elixir compiler, it doesn't look like {:debug_info_key, ...} is supported as an option. This was the original intent behind calling :beam_lib.crypto_key_fun/1 in the Erlang language module. However, either this option really is supported by the Elixir compiler or could be in the future. This is where it might make sense to put it in the Retriever module instead. This draft does it both ways for now.
2. Surfacing debug_info_key/debug_info_fn to the user. I've added these to ExDoc.Config. But I'm unsure of the best course of action for CLI options. Supporting just the key in the options and then allowing the user to define a function in a custom config.exs breaks the existing order of things (i.e. being able to set the same config options with command line switches as with a config file) since something like --debug_info_fn "fn ..." isn't user friendly. Maybe only debug_info_key should be supported for now?
3. Finally, I was considering adding debug_info_fun as a synonym so that Erlang users don't have to worry about translating fun to fn.

(note to self: the test needs to test for the rest of the docs in the erlang module)

[voughtdq]

Oh and one more thing - :beam_lib supports looking for a file called .erlang.crypt as a way of providing the decryption key to get the debug info.

Maybe it's good enough that we just advise users to use .erlang.crypt?

[voughtdq]

Ugh, sorry about being so noisy, I had one more thing to mention:

:beam_lib says it stores the crypto_key_fun in the current process (although I'm not sure what mechanism is used to do that since I don't see any calls to get/put), so this is something anyone reviewing the code should keep in mind.

[josevalim]

Maybe it's good enough that we just advise users to use .erlang.crypt?

Yes, let's go down this route. In any case, we don't need to surface everything in the command line. The most complex options are expected to be set via config files.

However, either this option really is supported by the Elixir compiler or could be in the future. This is where it might make sense to put it in the Retriever module instead. This draft does it both ways for now.

Yes, let's do it in the retriever, because I am almost sure you can also encode .beam files after the fact too.

[voughtdq]

That edoc test is flaky... maybe worth removing. When I was writing tests for edoc, I realized sometimes the docs would be available, but sometimes they wouldn't when tested without crypto_key_fun. The first run usually succeeds and then subsequent runs don't (maybe on the first run the docs are emitted and then encrypted?)

[voughtdq]

Going to close for now, will re-open with something more orderly.

[josevalim]

Do you need to test it with edoc? Could you test by using Elixir own modules? Somehow by using beam_lib or something similar to manually encrypt the debug_info after the fact? Or maybe require Erlang/OTP 27 and test only with the new -doc attributes, to remove edoc from the equation?

[voughtdq]

Now that I've taken some time to think about this -

I'd like to support edoc if at all possible, so I want to have tests for it. I think calling erlc twice would get the intended result.

Here's the heart of the issue - I'm concerned that some kind of leak is happening here. That is, something that people thought was being kept secret is not secret. I don't want to encourage people or give them a false sense of security if that's the case. This is, of course, unfounded - I don't know enough about edoc internals to really know for sure.

If we do determine it is insecure, what do you think about supporting it, but warning users that they should migrate to EEP59 docs? I will look into also warning if there's a way to check if a module has edocs.

Sorry to ping you @garazdawi, but do you have any thoughts on this? Is edoc leaking docs that are supposed to be encrypted here? I was asserting that compiling a module with edocs returns nothing when it is encrypted, but it actually does return docs.

[josevalim]

I think there is some confusion. The docs are not encrypted, the debug info (which stores the AST) is. It should really be ortogonal to Edoc and the -doc attribute. But it may be easier to test only one of them (instead of the two).

[voughtdq]

🤦‍♂️ sorry, I get it now - the EEP59 docs being encrypted is incidental.
I will take the edoc test out.