Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When exporting the syslog dump, some fields such as TLS_SERVER_NAME, SRC_NAME, DST_NAME are no longer exported in the 6.0 version or higher #8898

Open
ioesoft opened this issue Jan 7, 2025 · 0 comments
Assignees
Labels
Ready to Test a feedback is needed on a proposal or implementation

Comments

@ioesoft
Copy link

ioesoft commented Jan 7, 2025

Environment:

  • OS name: Ubuntu
  • OS version: 20.04.2 LTS
  • Architecture: x86_64
  • ntopng version/revision: 6.2.250103

What happened:
When exporting the syslog dump, some fields such as TLS_SERVER_NAME, SRC_NAME, DST_NAME are no longer exported in the 6.0 version or higher.
FYI, here is the comparison of the exported dumps between version 5.5 and the latest version 6.2.
As you can see in the sample below, there is no TLS_SERVER_NAME, SRC_NAME, DST_NAME field in the version 6.2.

1) v.5.5.220901
Jan 6 02:16:05 ntop ntopng[110569]: { "IN_SRC_MAC": "00:0C:29:97:EA:8A", "OUT_DST_MAC": "58:86:94:29:2E:D7", "JA3C_HASH": "bd0bf25947d4a37404f0424edf4db9ad", "IPV4_SRC_ADDR": "192.168.0.221", "SRC_ADDR_LOCAL": false, "SRC_ADDR_BLACKLISTED": false, "SRC_ADDR_SERVICES": 0, "SRC_NAME": "", "IPV4_DST_ADDR": "104.208.16.92", "DST_ADDR_LOCAL": false, "DST_ADDR_BLACKLISTED": false, "DST_ADDR_SERVICES": 0, "DST_NAME": "v10.events.data.microsoft.com", "SRC_TOS": 2, "DST_TOS": 2, "L4_SRC_PORT": 58260, "L4_DST_PORT": 443, "PROTOCOL": 6, "L7_PROTO": 219, "L7_PROTO_NAME": "TLS.Microsoft365", "TCP_FLAGS": 223, "IN_RETRASMISSIONS": 0, "OUT_RETRASMISSIONS": 0, "IN_OUT_OF_ORDER": 0, "OUT_OUT_OF_ORDER": 0, "IN_LOST": 0, "OUT_LOST": 0, "IN_PKTS": 16, "IN_BYTES": 3836, "OUT_PKTS": 13, "OUT_BYTES": 5357, "FIRST_SWITCHED": 1736097243, "LAST_SWITCHED": 1736097244, "CLIENT_NW_LATENCY_MS": 0.001, "SERVER_NW_LATENCY_MS": 79.093000000000004, "SRC_IP_COUNTRY": "", "SRC_IP_LOCATION": [ 0.0, 0.0 ], "DST_IP_COUNTRY": "US", "DST_IP_LOCATION": [ -93.625, 41.586799621582031 ], "NTOPNG_INSTANCE_NAME": "ntop", "INTERFACE": "tcp://*:5556c", "COMMUNITY_ID": "1:pQJmoLbLpkmkKJ9MpGAGCTx5y0Q=", "EXPORTER_IPV4_ADDRESS": "192.168.0.77", "TLS_SERVER_NAME": "v10.events.data.microsoft.com" }

Jan 6 02:13:40 ntop ntopng[110569]: { "IN_SRC_MAC": "28:D0:EA:C9:22:7D", "OUT_DST_MAC": "33:33:00:01:00:03", "IPV6_SRC_ADDR": "fe80::62e:9fb2:60c8:a022", "SRC_ADDR_LOCAL": true, "SRC_ADDR_BLACKLISTED": false, "SRC_ADDR_SERVICES": 0, "SRC_NAME": "fe80::62e:9fb2:60c8:a022", "IPV6_DST_ADDR": "ff02::1:3", "DST_ADDR_LOCAL": true, "DST_ADDR_BLACKLISTED": false, "DST_ADDR_SERVICES": 0, "DST_NAME": "ff02::1:3", "SRC_TOS": 0, "DST_TOS": 0, "L4_SRC_PORT": 62614, "L4_DST_PORT": 5355, "PROTOCOL": 17, "L7_PROTO": 154, "L7_PROTO_NAME": "LLMNR", "IN_PKTS": 2, "IN_BYTES": 142, "OUT_PKTS": 0, "OUT_BYTES": 0, "FIRST_SWITCHED": 1736097099, "LAST_SWITCHED": 1736097099, "SRC_IP_COUNTRY": "", "SRC_IP_LOCATION": [ 0.0, 0.0 ], "DST_IP_COUNTRY": "", "DST_IP_LOCATION": [ 0.0, 0.0 ], "NTOPNG_INSTANCE_NAME": "ntop", "INTERFACE": "tcp://*:5556c", "COMMUNITY_ID": "1:fGHvlRH3MooPgDaIIqoiY3k0Y/g=", "EXPORTER_IPV4_ADDRESS": "192.168.0.77" }

2) v.6.2.250103
Jan 6 16:41:19 ntop ntopng[721087]: { "IN_SRC_MAC": "00:0C:29:B7:A3:94", "OUT_DST_MAC": "58:86:94:29:2E:D7", "IPV4_SRC_ADDR": "192.168.0.126", "IPV4_DST_ADDR": "52.113.194.132", "SRC_TOS": 0, "DST_TOS": 0, "L4_SRC_PORT": 63457, "L4_DST_PORT": 443, "PROTOCOL": 6, "L7_PROTO": 219, "L7_PROTO_NAME": "TLS.Microsoft365", "L7_PROTO_RISK": 32768, "L7_PROTO_RISK_NAME": "TLS (probably) Not Carrying HTTPS", "FLOW_END_REASON": "idle_timeout", "TCP_FLAGS": 26, "IN_RETRANSMISSIONS": 0, "OUT_RETRANSMISSIONS": 0, "IN_OUT_OF_ORDER": 0, "OUT_OUT_OF_ORDER": 0, "IN_LOST": 0, "OUT_LOST": 0, "APPL_LATENCY_MS": 4, "IN_PKTS": 10, "IN_BYTES": 1680, "OUT_PKTS": 8, "OUT_BYTES": 7509, "FIRST_SWITCHED": 1736149158, "LAST_SWITCHED": 1736149158, "CLIENT_NW_LATENCY_MS": 0.001, "SERVER_NW_LATENCY_MS": 1.621, "SRC_IP_COUNTRY": "", "SRC_IP_LOCATION": [ 0.0, 0.0 ], "DST_IP_COUNTRY": "CA", "DST_IP_LOCATION": [ -73.567398071289062, 45.501899719238281 ], "COMMUNITY_ID": "1:QBqIoVw1XiAnBuH6sZMUKGgCptY=", "L7_RISK_SCORE": 10, "EXPORTER_IPV4_ADDRESS": "192.168.0.77" }

How did you reproduce it?
Just export the dump via syslog by setting "-F=syslog;local3" option in the ntopng.conf.

Debug Information:

  1. ntopng.conf
    -G=/var/run/ntopng.pid
    -i=tcp://*:5556c
    -F=syslog;local3
    -W=192.168.0.30:443
    -v=2
    -n=3

  2. systemctl status ntopng result
    ntopng@ntop:~$ systemctl status ntopng
    ● ntopng.service - ntopng high-speed web-based traffic monitoring and analysis tool
    Loaded: loaded (/etc/systemd/system/ntopng.service; disabled; vendor preset: en>
    Active: active (running) since Mon 2025-01-06 18:22:07 KST; 21h ago
    Process: 1571 ExecStartPre=/bin/sh -c /usr/bin/ntopng-utils-manage-config -a che>
    Process: 1585 ExecStartPre=/bin/sh -c /bin/cat /etc/ntopng/ntopng.conf > /run/nt>
    Process: 1587 ExecStartPre=/bin/sh -c /bin/cat /etc/ntopng/ntopng.conf.d/.conf >
    Process: 1589 ExecStartPre=/bin/sh -c /bin/sed "/^[ ]
    -e.$|^[ ]-G.|^[ ]--d>
    Main PID: 1591 (ntopng-main)
    Tasks: 43 (limit: 1048)
    Memory: 149.8M
    CGroup: /system.slice/ntopng.service
    └─1591 /usr/bin/ntopng /run/ntopng.conf

Jan 07 16:05:10 ntop ntopng[1591]: { "IN_SRC_MAC": "28:D0:EA:C9:22:7D", "OUT_DST_MAC>
Jan 07 16:05:11 ntop ntopng[1591]: { "IN_SRC_MAC": "00:0C:29:B7:A3:94", "OUT_DST_MAC>
Jan 07 16:05:11 ntop ntopng[1591]: { "IN_SRC_MAC": "28:D0:EA:C9:22:7D", "OUT_DST_MAC>
Jan 07 16:05:11 ntop ntopng[1591]: { "IN_SRC_MAC": "28:D0:EA:C9:22:7D", "OUT_DST_MAC>
Jan 07 16:05:12 ntop ntopng[1591]: { "IN_SRC_MAC": "28:D0:EA:C9:22:7D", "OUT_DST_MAC>
Jan 07 16:05:15 ntop ntopng[1591]: { "IN_SRC_MAC": "A8:A1:59:A5:B6:3A", "OUT_DST_MAC>
Jan 07 16:05:15 ntop ntopng[1591]: { "IN_SRC_MAC": "00:0C:29:B7:A3:94", "OUT_DST_MAC>
Jan 07 16:05:16 ntop ntopng[1591]: { "IN_SRC_MAC": "00:0C:29:B7:A3:94", "OUT_DST_MAC>
Jan 07 16:05:16 ntop ntopng[1591]: { "IN_SRC_MAC": "A8:A1:59:A5:B6:3A", "OUT_DST_MAC>
Jan 07 16:05:16 ntop ntopng[1591]: { "IN_SRC_MAC": "A8:A1:59:A5:B6:3A", "OUT_DST_MAC>

@MatteoBiscosi MatteoBiscosi self-assigned this Jan 7, 2025
@MatteoBiscosi MatteoBiscosi added the Ready to Test a feedback is needed on a proposal or implementation label Jan 7, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Ready to Test a feedback is needed on a proposal or implementation
Projects
None yet
Development

No branches or pull requests

2 participants