[CHIA-1829] don't drop outgoing response messages

[arvidn]

Purpose:

With regards to long sync, there are two serious issues with the current rate limits.

1. send- vs. receive limits

request_blocks has a rate limit of 500 messages / minute and respond_blocks has a rate limit of only 100 messages / minute. This discrepancy was probably an oversight, as it doesn't make sense to allow responses at a lower rate than request.

Additionally, all outgoing messages are subject to a reduction of limit to 30%, so the effective limit for response messages is much lower than the corresponding request message. This was probably an attempt to avoid exceeding limits because of phase shifted 1-minute windows. What appears to staying within the limits on the sender may exceed the limits on the receiver, if the windows overlap in the right way.

This reduction causes the peer serving blocks to hit its send rate limit (of response messages) long before the requesting peer hits its rate limit for request messages. Since #18907 landed, this problem can be seen in the logs, quite often. When the response message is dropped, the requesting peer waits for 30 seconds before timing out and banning the other peer. This significantly slows down long sync.

The issue of sending request_blocks at too high rate, causing disconnects, was attempted to be addressed in #18729. Not only did this patch make it clear just how rate limited long-sync is, but it turned out this wasn't enough. Because of the second issue, described below, to make the request pacing work well, it would have to be even more complex.

2. overall bytes limit

Even though the respond_blocks message has a limit of 250 MB / minute, there's an independent limit of the combined sizes of all non-TX messages). This limit is just 100 MB / minute. This was probably an oversight. It appears as if the intention was to limit respond_blocks to the higher value, but it never takes effect. Additionally, both of these limits are also subject to the upload rate factor of 30%, so the effective limit of all (non-TX) messages sent is just 30 MB/minute, or 512 kiB/s.

This change

This change removes the rate limit entirely for the following messages:

• RespondBlock
• RespondBlocks
• RejectBlock
• RejectBlocks

Implemented here and here.

To prevent abuse, we also need to disconnect any peer sending these messages unsolicited. Implemented here.

This change is essentially backwards compatible because the response messages are effectively rate limited by the corresponding request messages, and if you exceed the rate limit, disconnecting (without banning) is still a better outcome than waiting 30 seconds and then disconnect and ban the peer.

message	count/min	scaled to 30%
request_block	200	60
request_blocks	500	150

Previous incoming response limits:

message	count/min	size/min
respond_blocks	100	250 MiB
respond_block	200	20 MiB
reject_blocks	100	10 kiB
reject_block	200	20 kiB

With this patch, if you send the maximum number of request_blocks, 150 you may receive 150 respond_blocks and disconnect the peer. The current behavior is that you receive 30 respond_blocks, wait 30 seconds and then ban the peer.

Current Behavior:

RespondBlock, RespondBlocks, RejectBlock, RejectBlocks are subject to unreasonably low rate limits, causing issues syncing.

New Behavior:

RespondBlock, RespondBlocks, RejectBlock, RejectBlocks are not subject to any rate limits other than the implied rate limit of the corresponding request message.

Peers sending unsolicited responses are banned.

Testing Notes:

I've been running a node on testnet11 to ensure I no longer drop respond_blocks messages.

[coveralls-official]

Pull Request Test Coverage Report for Build 12212149266

Details

• 80 of 80 (100.0%) changed or added relevant lines in 6 files are covered.
• 18 unchanged lines in 7 files lost coverage.
• Overall coverage increased (+0.004%) to 91.472%

---

Files with Coverage Reduction	New Missed Lines	%
chia/wallet/util/wallet_sync_utils.py	1	86.07%
chia/daemon/client.py	1	74.58%
chia/plotters/plotters.py	1	90.94%
chia/consensus/blockchain.py	2	95.39%
chia/wallet/wallet_node.py	3	88.03%
chia/server/ws_connection.py	4	88.15%
chia/plotters/madmax.py	6	44.58%

Totals
Change from base Build 12209661102:	0.004%
Covered Lines:	104612
Relevant Lines:	114179

---

💛 - Coveralls

[emlowe]

nice analysis