Think about fragmentation

[TimonPost]

Currently, we fragment our packets, however, there has been some problems with this. I am not sure about the exact problems because it was discussed a while ago. However, our current implementation is not completely finished yet. We do fragment big packets, but we do not acknowledge individual fragments yet. So if you just send big chunks it might happen that individual parts would not arrive at the destination, which would result in the case that a packet will never be returned back to the user, because the fragmentation is not completed. The IP-layer, however, should provide fragmentation it'self already. And it will resent a whole packet if one fragment is not received.

The problem we are facing is that we keep packets under the MTU-size, but next don't offer any reliability guarantee on top of it. And thus it would be better if we just removed the fragmentation and let the IP-layer handle it. Or we should implement acknowledgment for each fragment so that we do add value to our fragmentation.

[OvermindDL1]

UDP will not be transparently merged back together if it is broken up, at most usual it's dropped. Fragmentation needs to be handled in this library, and it needs to be able to scale it down as MTU can occasionally be lower than expected...

[ckaran]

Look into using raptorq. I've been using it in my PhD work in a network simulator I'm writing, and I'm very happy with how it performs. The algorithm is defined by RFC 6330; skim the first few sections to get an idea of what it provides.

Why use it

• Can fragment messages that are up to 946,270,874,880 bytes in length (section 3.3.2 of the RFC).
• It is a Forward Error Correcting (FEC) building block. That means that if you lose a portion of the packets, you can still recover the original message. There are tuning parameters within the algorithm (which the raptorq exposes) that allow you to select how much protection you want to have. E.g., with a particular tuning parameter you are guaranteed that you can lose 20% of the packets, and still be able to recover the original message. The encoding is such that it doesn't matter which subset of packets are lost. This is extremely useful for UDP multicast or broadcast, where the clients are each getting a different subset of the packets. See https://tools.ietf.org/html/rfc5052 and https://en.wikipedia.org/wiki/Forward_error_correction for more information.
• Focuses on fragmentation & assembly only; the inputs and outputs are byte strings, which you are then required to do something with. That means it gets out of the way of the actual networking layer, so you could potentially use it on any kind of network you could think of.
• Is written in pure rust 😁
• Seems to be quite fast (but I haven't done any significant profiling, so take this with a grain of salt).

Why not to use it

• Documentation is quite sparse; the author expects you to read RFC 6330.
• Last time I checked (pre-1.0) the output packets are always exactly the size of the MTU. I don't know if this is an artifact of the algorithm or the crate, nor do I know if this has been fixed.
• While FEC improves the odds of a message getting through, it doesn't guarantee reliable delivery; if you want reliability, you will need acknowledgements. But that is a problem for a different issue.

Tips & tricks

When you're setting up communications, create a pair of channels, one over TCP and one over UDP. The TCP channel will let you communicate the tuning parameters (ObjectTransmissionInformation) to all clients securely. This is useful if you want to be able to adapt to changing network conditions (new players joining or leaving, or the network getting bad over time) by changing the tuning parameters.

This approach is also how you get low-latency reliability, but that is something that should be discussed in a different issue.