[Bug] BRPC Error

[eleven-f]

Search before asking

• I had searched in the issues and found no similar issues.

Version

BE: doris-2.1.7-rc03-443e87e203

What's Wrong?

The following errors may be reported when querying, writing or deleting. occur more frequently!
Error:
1105, 'errCode = 2, detailMessage = (192.168.2.105)[CANCELLED]failed to send brpc when exchange, error=Resource temporarily unavailable, error_text=[E11]Resource temporarily unavailable @192.168.2.101:8060, client: 192.168.2.105, latency = 843'

What You Expected?

What causes this problem? How can I avoid it?

How to Reproduce?

No response

Anything Else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[stevqin]

I've encountered the same situation as well.

[Mryange]

目前版本可以通过调小下面两个变量来缓解这个问题。

parallel_pipeline_task_num // 一个qurry的并发个数，如果为0则并发数为cpu核心的一半
parallel_exchange_instance_num //  显示一个query exchange后的并发个数

原因是brpc内部有个限制。

| EAGAIN         | 11   | 是    | 同时发送的请求过多。软限，很少出现。                       | Resource temporarily unavailable         |

Doris be 与 be 之间的exchange 是共用一个长连接。
而且上游的所有并发都会给下游的所有并发发送数据。（例如并发为n，则同时有n * n 个rpc请求
这导致并发比较高的情况下就很容易出现这个问题。