Fix likely deadlock issue in SafeClient

[Hyodar]

From our testing data on operator state root messaging, it seems quite likely there's a deadlock happening in SafeClient for block submissions. Would be good to get an overall review on it. Trying to change some of the approaches that seem problematic while I get more info on it, at the moment:

• Centralized resubbing in reinits, so there's no way a resub can happen while it's reinitializing
• Made resubs not block the event listening from the subscription task

Good to highlight the rollup clients are only used for header subscriptions, so it's down to SubscribeNewHead specifically.

[Hyodar]

One additional point here is - is it necessary to reinitialize the client? It makes sense to do it, but maybe it's not necessary. I'm starting to feel like it's not, and we could just simplify this module a lot.

[emlautarom1]

How was this deadlock detected? Any way to reproduce in a testing environment?

[Hyodar]

As mentioned there, not totally sure what's happening yet. Couldn't reproduce it as well, currently trying to subscribe to headers locally and see if I can catch it. The symptoms are simply that randomly after some time an operator stops sending StateRootUpdateMessages, also trying to get more info on it from operator logs.

[Hyodar]

Setting this as draft for now (actually should've set as draft when creating - not meant to be open rn), as I couldn't yet reproduce the issue and I didn't see it happening again. Also, it's quite likely I'm going to simplify the module a bit - we don't actually need to reinitialize the client.

This could also not actually be an issue. My previous understanding was that the way reinit notifications are working could cause a deadlock when resubbing, but this doesn't make too much sense.

[Hyodar]

Closing as this is not relevant anymore as per latest changes.