Calling malloc from signal handler

[nathanwbrei]

Signal handler is calling malloc. Signal handlers should avoid all non-async-signal-safe (approx the same thing as non-reentrant) syscalls as per https://man7.org/linux/man-pages/man7/signal-safety.7.html. That includes malloc and all IO.

If the segfault corrupts malloc itself, such as here (eic/EICrecon#402), we lose the backtrace.

The solution is to pre-allocate a buffer for the backtrace instead.

[nathanwbrei]

As of #385, we pre-allocate buffers for the backtrace. We also move symbol lookup and demangling to the supervisor thread. The system seems robust in practice, and TSAN is down to exactly one complaint, about backtrace() itself calling malloc. It turns out this is an easy fix: As per the backtrace(3) man page, the malloc() call happens during dynamic library loading. We only need to ensure that the library is loaded before any signal handlers are invoked.