using Concurrency in EsploraConfig resulting in errors

[BitcoinZavior]

Describe the bug

Using concurrency value of more than 2 for EsploraConfig is resulting in following errors:

1. failed to lookup address information: nodename nor servname provided, or not known
2. timed out reading response
3. tls connection init failed / AlertReceived(HandshakeFailure)
4. ConnectionAborted / Software caused connection abort

Issue orginally raised for bdk-rn: LtbLightning/bdk-rn#51

To Reproduce

Initialise a EsploraConfig with a concurrency value of more than 2
Use the EsploraConfig to initialise a Blockchain object
User the Blockchain object to sync a wallet

Expected behavior

The sync should be faster when using a higher concurrency value and no result in errors

Build environment
bdk kotlin and bdk swift

This might be a possible cause: https://stackoverflow.com/questions/69732481/strange-error-when-multithreading-downloads-in-rust#comment123259499_69732481

[Czino]

I'd like to add that even after changing concurrency to 1

Most of these errors still occur (albeit less frequently)

1. failed to lookup address information: nodename nor servname provided, or not known
2. timed out reading response
3. failed to lookup address information: No address associated with hostname
4. ConnectionAborted / Software caused connection abort

[thunderbiscuit]

Weird. It looks like this would be Rust related because the ffi layer simply passes those options along to the Rust library without (as far as I can tell) transforming them or applying any kind of logic.

Do you get the same errors when leaving the value as null? (the config should default to 4)

[Czino]

That's hard to test, because I don't get these errors locally but on production for our users. My phones seem to be unaffected from seeing these kind of errors, so I cannot inspect it for myself.

What I can tell you is that when I changed the concurrency from 5 to 1, the error rate lessened.

[Czino]

Also worth mentioning, we are currently using the REST API instead of RPC because our server is behind cloudflare that does not forward RPC calls.

[thunderbiscuit]

This issue has gone stale and I'll close it for now, but if it's still a problem and you'd like to revive the discussion feel free to reopen!