Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Heplify tcpassembly / sipassembly #313

Open
brownee3210 opened this issue Dec 14, 2024 · 3 comments
Open

Heplify tcpassembly / sipassembly #313

brownee3210 opened this issue Dec 14, 2024 · 3 comments

Comments

@brownee3210
Copy link

Hi,

I have a SIP data stream I am monitoring, and it is SIP over TCP and the SIP message are large so are being segmented by the TCP layer.

Some of these messages have 3 to 4 segments that need to be re-assembled.

I have been trying to get heplify's “-tcpassembly” and “-sipassembly” options to re-assemble this SIP data but so far unsuccessfully.

If I run the heplify command without the “-tcpassembly” ” and “-sipassembly” options I see the message in the database, but it is truncated.

If I run with the “-tcpassembly” ” and / or “-sipassembly” options the message seems to be skipped all together and are not written to the database.

This is the heplify command I have been testing with and I have added the option “assembly_debug_log” to log debugging for TCP assembly. I have also used the “-e” option to send logging to stderr. I am using heplify version 1.66.10.

/usr/local/sbin/hep/heplify -i eno2 -dd -nt udp -hs 192.168.4.161:9998 -hi 4100 -m SIP -pr 5060-5070 -t pcap -prometheus 192.168.4.194:7998 -vlan -tcpassembly -sipassembly -e -assembly_debug_log

The output is quite verbose but one of the messages I see is: -
“hit_ max buffer size: {MaxBufferedPagesTotal:1 MaxBufferedPagesPerConnection:1}, 1, 1”

Not sure if this is a problem / error message that could be contributing to this problem?

After this not really sure what else I can try to solve this so hoping someone can help out with this?

Regards,
Graham Brown

@adubovikov
Copy link
Member

can you please share the original pcap file with us ? The issue can be out of order fragments. Can you also use not the "pcap" interface, but the "afpacket"

@brownee3210
Copy link
Author

Hi,

Thank you for your response.

I have changed pcap to af_packet. Not exactly sure about this change. Are you able to explain the advantages of af_packet and why we should use it instead of pcap?

I can also see that there are 2 further options that can be considered in the config for af_packet: -

  1. -fg uint Fanout group ID for af_packet
  2. -fw int Fanout worker count for af_packet (default 4)

Can you also explain how these 2 config elements should be configured and dimensioned when using af_packet?

I am still experiencing problems with TCP segment re-assembly.

As stated in your reply “The issue can be out of order fragments” and in the PCAP file(s) I do indeed see out of order TCP segments.

Where there are missing SIP messages in my Homer database, I have checked in the associated PCAP file, and they are indeed SIP messages with out of order TCP segments.

The odd thing is that some SIP messages with out of order TCP segments don’t always to fail re-assembly by heplify, it seems a bit “random” though as to whether re-assembly will work or not with out of order TCP segments.

For SIP messages with their TCP segments correctly ordered there is never a problem, re-assembly is always successful, it is only when TCP segments are out of order there seems to be some sort of intermittent re-assembly problem with heplify.

The heplify agent I am using is version 1.66.10.

Unfortunately, I do have a problem with the sharing of PCAP files. The company I work for does not want any PCAP files to be published on a public forum.

Do you have and suggestions/ideas how we can proceed without sharing the PCAP file?

Regards,
Graham Brown

@lmangani
Copy link
Member

Hello @brownee3210 and thanks for the additional details

Are you able to explain the advantages of af_packet and why we should use it instead of pcap?

This is not specific to heplify and you can look up what af_packet is/does as its features/settings as they are generic.

As stated in your reply “The issue can be out of order fragments” and in the PCAP file(s) I do indeed see out of order TCP segments.

This is most definitely related to the problem but a guess won't take us much futher.

Unfortunately, I do have a problem with the sharing of PCAP files. The company I work for does not want any PCAP files to be published on a public forum.

That's common with opensource.... Companies are fine using it for free without ever contributing anything back and when its time to provide material to resolve a problem for everyone, things hit a wall. We understand your frustration but with homer we do everything in public so unless you can provide a way to reproduce the issue I'm afraid we won't be able to do much. Our company does offer commercial support for hepic with the ability to troubleshoot privately and securely in case that's useful but when it comes to community opensource we require for issues and data to be open as well to benefit all users rather than just some. Thanks for understanding!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants