APM logs with incorrect hostname value

[nunofabarros]

Environment

• Telemetry Streaming Version: 1.32.0
• BIG-IP Version: 16.1.3.1
• Splunk Enterprise: 8.0.3

Summary

In some rare occasions, at the logs for the APM module the hostname field is being set either as null or as the value of the Packet_L7_Hostname field.

Steps To Reproduce

Steps to reproduce the behavior:

1. Output declaration submitted:

{
    "message": "success",
    "declaration": {
        "class": "Telemetry",
        "controls": {
            "class": "Controls",
            "logLevel": "info",
            "debug": false,
            "memoryThresholdPercent": 90
        },
        "SDE_System": {
            "class": "Telemetry_System",
            "systemPoller": {
                "interval": 60,
                "actions": [
                    {
                        "setTag": {
                            "env": "dev",
                            "region": "lab"
                        },
                        "locations": {
                            "system": true
                        },
                        "enable": true
                    }
                ],
                "enable": true
            },
            "enable": true,
            "host": "localhost",
            "port": 8100,
            "protocol": "http",
            "allowSelfSignedCert": false
        },
        "SDE_Listener": {
            "class": "Telemetry_Listener",
            "port": 6514,
            "actions": [
                {
                    "setTag": {
                        "env": "dev",
                        "region": "lab"
                    },
                    "enable": true
                }
            ],
            "enable": true,
            "trace": false,
            "match": ""
        },
        "SDE_Consumer": {
            "class": "Telemetry_Consumer",
            "type": "Splunk",
            "host": "xxxxxxxxxxx",
            "protocol": "https",
            "port": 443,
            "allowSelfSignedCert": true,
            "passphrase": {
                "cipherText": "xxxxxxxx",
                "class": "Secret",
                "protected": "SecureVault"
            },
            "compressionType": "none",
            "enable": true,
            "trace": false,
            "format": "default"
        },
        "schemaVersion": "1.32.0"
    }
}

2. Observe the following on a minority of the logs entries in Splunk:

• Where the hostname is null:

{
    "hostname": "",
    "errdefs_msgno": "01490248:5:",
    "partition_name": "euw5_specific",
    "session_id": "a025c8ce",
    "Access_Profile": "/Common/gcp-infra-api_ts20210716210259",
    "Partition": "euw5_specific",
    "Session_ID": "a025c8ce",
    "Client_Type": "Mozilla",
    "Client_Version": "1",
    "Client_Platform": "Win",
    "Client_CPU": "unknown",
    "Client_UI_Mode": "Full",
    "Client_JS_Support": "1",
    "Client_Activex_Support": "0",
    "Client_Plugin_Support": "0",
    "telemetryEventCategory": "APM",
    "f5telemetry_timestamp": "2022-10-27T09:10:50.863Z",
    "originalRawData": "hostname=\"clouda900106.c.xxx-xxx-xxxx-xxx.internal\",errdefs_msgno=\"01490248:5:\",partition_name=\"euw5_specific\",session_id=\"a025c8ce\",Access_Profile=\"/Common/gcp-infra-api_ts20210716210259\",Partition=\"euw5_specific\",Session_ID=\"a025c8ce\",Client_Hostname=\"\",Client_Type=\"Mozilla\",Client_Version=\"1\",Client_Platform=\"Win\",Client_CPU=\"unknown\",Client_UI_Mode=\"Full\",Client_JS_Support=\"1\",Client_Activex_Support=\"0\",Client_Plugin_Support=\"0\"",
    "env": "dev",
    "region": "euw5"
}

• Where hostname is equal to Packet_L7_Hostname field:

{
    "hostname": "anthropos-ui.xxxxxx.xxxx.xxx",
    "errdefs_msgno": "01580005:5:",
    "partition_name": "euw5_specific",
    "session_id": "bd083c25",
    "Access_Profile": "/Common/gcp-infra-ui_ts20210716210438",
    "Partition": "euw5_specific",
    "Session_ID": "bd083c25",
    "Action": "reject",
    "Name": "/Common/gcp-infra-ui_ts20210716210438-gcp-infra-ui_ts20200207103539-gcp-csm_ts1575456858-default_deny",
    "Index": "0",
    "Packet_L7_Scheme": "https",
    "Packet_L7_Path": "/ruxitagentjs_ICA2Vfqrux_10249220905100923.js",
    "Packet_Protocol": "tcp",
    "Packet_Source_IP": "193.16.224.4",
    "Packet_Source_Port": "34258",
    "Packet_Destination_IP": "xx.xx.xx.xx",
    "Packet_Destination_Port": "443",
    "telemetryEventCategory": "APM",
    "f5telemetry_timestamp": "2022-10-27T07:42:02.894Z",
    "originalRawData": "hostname=\"clouda900106.c.xxx-xxx-xxx-xxx.internal\",errdefs_msgno=\"01580005:5:\",partition_name=\"euw5_specific\",session_id=\"bd083c25\",Access_Profile=\"/Common/gcp-infra-ui_ts20210716210438\",Partition=\"euw5_specific\",Session_ID=\"bd083c25\",Action=\"reject\",Name=\"/Common/gcp-infra-ui_ts20210716210438-gcp-infra-ui_ts20200207103539-gcp-csm_ts1575456858-default_deny\",Index=\"0\",Packet_L7_Scheme=\"https\",Packet_L7_Hostname=\"anthropos-ui.xxxxxx.xxxx.xxx\",Packet_L7_Path=\"/ruxitagentjs_ICA2Vfqrux_10249220905100923.js\",Packet_Protocol=\"tcp\",Packet_Source_IP=\"193.16.224.4\",Packet_Source_Port=\"34258\",Packet_Destination_IP=\"xxxxxxxxx\",Packet_Destination_Port=\"443\"",
    "tenant": "Common"
}

Expected Behavior

The hostname should be always the equal to the hostname filed on the originalRawData.

Actual Behavior

The hostname is set on rare occasions either null or the same as the Packet_L7_Hostname field.

Thanks for help,

Nuno