Processing of sampled traffic

[pavel-odintsov]

Hello!

I'm really happy with your library! Thank you so much! It's working fine!

But I'm looking for ways to handle sampled traffic. In modern 100GE+ networks usage of sampled mirrors or sampled traffic monitoring protocols (i.e. sflow) is growing so fast.

As you know we could not use flow tracking because it's too small chance to get another packet for same TCP session. Usual sampling rate of 1024-4096 do not allow us to use this approach at all.

So only one way to classify at lease small amount of whole traffic is per-packet processing without any per-flow processing.

Do you have experience or research papers with this approach? Could libprotoident work in this mode?

Thank you!

[salcock]

No, libprotoident won't work very well at all with packet sampling, as it is very reliant on seeing very specific packets from within each flow to be able to classify that flow correctly.

The only way libprotoident could work in the presence of sampling is if you were able to do flow-based sampling, i.e. only capture packets for 1 / 100 flows, but I'm not aware of any high-speed sampling protocols that operate at the flow level. The other benefit of flow-based sampling is that libprotoident only needs to see the first payload-bearing packet in each direction for a flow, so your sampling process could automatically discard any subsequent packets for that flow.

Maybe you could use IPFIX to do some of what I describe, but I suspect it would require a few extensions on your part -- it wouldn't work "out of the box", so to speak.