A cache of securityStateReference will not be popped when error situation happens without the reportableFlag for SNMPv3.

[y-iwata-bl]

Expected behavior

I expect the cache will be popped as follows.

            # 7.1.3b
            if (
                pdu is None
                and not reportableFlag
                or pduType is not None
                and pduType not in rfc3411.confirmedClassPDUs
            ):
                if securityModel in snmpEngine.securityModels:                 # added
                    smHandler = snmpEngine.securityModels[securityModel]       # added
                    smHandler.releaseStateInformation(securityStateReference)  # added

                raise error.StatusInformation(errorIndication=errind.loopTerminated)

Actual behavior

My snmp trap receiver using pysnmp library received a SNMPv3 trap message having invalid password for authentication without the reportableFalg.
The cache for securityStateReference was not popped properly and the memory of the trap receiver was getting increased.

Following is my analysis of the behavior.

prepareDataElement() in proto/mpmod/rfc3412.py calls processIncomingMsg() in proto/secmod/rfc3414/service.py.
processIncomingMsg() pushes a cache to create a message to report later.

        # 3.2.11 (moved up here to let Reports be authenticated & encrypted)
        self._cache.pop(securityStateReference)
        securityStateReference = self._cache.push(
            msgUserName=securityParameters.getComponentByPosition(3),
            usmUserSecurityName=usmUserSecurityName,
            usmUserAuthProtocol=usmUserAuthProtocol,
            usmUserAuthKeyLocalized=usmUserAuthKeyLocalized,
            usmUserPrivProtocol=usmUserPrivProtocol,
            usmUserPrivKeyLocalized=usmUserPrivKeyLocalized,
        )

And then, the authenticationFailure exception raises because of the invalid password.

prepareDataElement() calls returnResponsePdu() in proto/rfc3412.py to report the error to the sender of the trap.
returnResponsePdu() calls prepareResponseMessage() in proto/mpmod/rfc3412.py.
prepareResponseMessage() is terminated at the following location without sending the message to report because the reportableFlag is 0.

            # 7.1.3b
            if (
                pdu is None
                and not reportableFlag
                or pduType is not None
                and pduType not in rfc3411.confirmedClassPDUs
            ):
                raise error.StatusInformation(errorIndication=errind.loopTerminated)

So, nobody pops the securityStateReference chache, the cache will be accumulated and the memory of the trap receiver will be increased.

Detailed steps

Following is the detailed steps.

1. Run a trap receiver using pysnmp library.
2. Send the SNMPv3 trap message containing one of the followings to the receiver by the snmptrap command.
   Invalid Engine ID.
   Invalid security name.
   Invalid password for authentication.
   Invalid password for privacy.

Note:
The problem does not happen under the following conditions.

• The trap message is received successfully.
• The version of SNMP is v1 or v2c.
• Executing the snmptrap command with the "-Ci" option or the snmpinfom command.

Python package information

pysnmp 6.2.4

Operating system information

Red Hat Enterprise Linux release 9.2

Python information

python 3.9

(Optional) Contents of your test script

snmptrap -e 80001f8880452e6e351154796600000000 -v 3 -l authPriv -u trapuser -a SHA -A testtesttest -x AES -X testtesttest 172.16.41.133 '' 1.3.6.1.4.1.119.2.3.126.10.2.22.1.3.0.4 1.3.6.1.4.1.119.2.3.126.10.2.22.1.1.3.1.2.882401 s 'admin123' 1.3.6.1.4.1.119.2.3.126.10.2.22.1.1.3.1.3.882401 i 1 1.3.6.1.4.1.119.2.3.126.10.2.22.1.1.3.1.4.882401 x c0a80165

Relevant log output

2024-08-08 22:08:43,459 pysnmp: receiveMessage: msgVersion 3, msg decoded
2024-08-08 22:08:43,460 pysnmp: prepareDataElements: SNMPv3Message:
 msgVersion=3
 msgGlobalData=HeaderData:
  msgID=1265346651
  msgMaxSize=65507
  msgFlags=0x03
  msgSecurityModel=3

 msgSecurityParameters=0x303b041180001f8880452e6e35115479660000000002010002010004087472617075736572040cde51366c6d3069ab18d4def804088d7d58ca1f6fb41a
 msgData=ScopedPduData:
  encryptedPDU=0x8ba4b74659ca8fc86f495419e264adcf6fdf4c61ab90a6eebae1584acd207742aa2cd2b2de696babcfaf6e5279121a2dfd17731183c0b635a9df4b514e65a347f41d784c94741d800138f26e74a8ad20a4078b6158e98f0dfcbcd4beacbcd367a9b3d91807b46b07ab6ff035b9ac4657dc3d75081b1dbf4e04114a2f4da8f644fa5af1935000b838a1a3837aacd51d0e678016c82d7bd354850bd33c6ba1e0049afe03e8a5dcfc862977e22bc11caf885108a7d95c3337


2024-08-08 22:08:43,460 pysnmp: prepareDataElements: msg data msgVersion 3 msgID 1265346651 securityModel 3
2024-08-08 22:08:43,460 pysnmp: processIncomingMsg: securityParameters
00000: 30 3B 04 11 80 00 1F 88 80 45 2E 6E 35 11 54 79
00016: 66 00 00 00 00 02 01 00 02 01 00 04 08 74 72 61
00032: 70 75 73 65 72 04 0C DE 51 36 6C 6D 30 69 AB 18
00048: D4 DE F8 04 08 8D 7D 58 CA 1F 6F B4 1A
2024-08-08 22:08:43,461 pysnmp: processIncomingMsg: UsmSecurityParameters:
 msgAuthoritativeEngineId=0x80001f8880452e6e351154796600000000
 msgAuthoritativeEngineBoots=0
 msgAuthoritativeEngineTime=0
 msgUserName=trapuser
 msgAuthenticationParameters=0xde51366c6d3069ab18d4def8
 msgPrivacyParameters=0x8d7d58ca1f6fb41a

2024-08-08 22:08:43,461 pysnmp: processIncomingMsg: cache write securityStateReference 10612125 by msgUserName trapuser
2024-08-08 22:08:43,461 pysnmp: processIncomingMsg: non-synchronized securityEngineID <OctetString value object, tagSet <TagSet object, tags 0:0:4>, encoding iso-8859-1, payload [0x80001f8880452e...1154796600000000]>
2024-08-08 22:08:43,461 pysnmp: processIncomingMsg: read from securityParams msgAuthoritativeEngineId <OctetString value object, tagSet <TagSet object, tags 0:0:4>, encoding iso-8859-1, payload [0x80001f8880452e...1154796600000000]> msgUserName <OctetString value object, tagSet <TagSet object, tags 0:0:4>, subtypeSpec <ConstraintsIntersection object, consts <ValueSizeConstraint object, consts 0, 32>>, encoding iso-8859-1, payload [trapuser]>
2024-08-08 22:08:43,461 pysnmp: processIncomingMsg: read user info from LCD
2024-08-08 22:08:43,462 pysnmp: processIncomingMsg: now have usmUserName <SnmpAdminString value object, tagSet <TagSet object, tags 0:0:4>, subtypeSpec <ConstraintsIntersection object, consts <ValueSizeConstraint object, consts 0, 65535>, <ValueSizeConstraint object, consts 0, 255>, <ValueSizeConstraint object, consts 1, 32>>, encoding utf-8, payload [trapuser]> usmUserSecurityName <SnmpAdminString value object, tagSet <TagSet object, tags 0:0:4>, subtypeSpec <ConstraintsIntersection object, consts <ValueSizeConstraint object, consts 0, 65535>, <ValueSizeConstraint object, consts 0, 255>>, encoding utf-8, payload [trapuser]> usmUserAuthProtocol <AutonomousType value object, tagSet <TagSet object, tags 0:0:6>, payload [1.3.6.1.6.3.10.1.1.3]> usmUserPrivProtocol <AutonomousType value object, tagSet <TagSet object, tags 0:0:6>, payload [1.3.6.1.6.3.10.1.2.4]> for msgUserName <OctetString value object, tagSet <TagSet object, tags 0:0:4>, subtypeSpec <ConstraintsIntersection object, consts <ValueSizeConstraint object, consts 0, 32>>, encoding iso-8859-1, payload [trapuser]>
2024-08-08 22:08:43,462 pysnmp: StatusInformation: {'errorIndication': AuthenticationFailure('Authenticator mismatched')}
2024-08-08 22:08:43,463 pysnmp: StatusInformation: {'errorIndication': AuthenticationFailure('Authenticator mismatched'), 'oid': (1, 3, 6, 1, 6, 3, 15, 1, 1, 5, 0), 'val': <Counter32 value object, tagSet <TagSet object, tags 64:0:1>, subtypeSpec <ConstraintsIntersection object, consts <ValueRangeConstraint object, consts 0, 4294967295>>, payload [1]>, 'securityStateReference': 10612126, 'securityLevel': 3, 'contextEngineId': <SnmpEngineID value object, tagSet <TagSet object, tags 0:0:4>, subtypeSpec <ConstraintsIntersection object, consts <ValueSizeConstraint object, consts 0, 65535>, <ValueSizeConstraint object, consts 5, 32>>, encoding iso-8859-1, payload [0x80004fb8056f65...7665723153eea940]>, 'contextName': b'', 'msgUserName': <OctetString value object, tagSet <TagSet object, tags 0:0:4>, subtypeSpec <ConstraintsIntersection object, consts <ValueSizeConstraint object, consts 0, 32>>, encoding iso-8859-1, payload [trapuser]>, 'maxSizeResponseScopedPDU': 65398}
2024-08-08 22:08:43,463 pysnmp: prepareDataElements: SM failed, statusInformation {'errorIndication': AuthenticationFailure('Authenticator mismatched'), 'oid': (1, 3, 6, 1, 6, 3, 15, 1, 1, 5, 0), 'val': <Counter32 value object, tagSet <TagSet object, tags 64:0:1>, subtypeSpec <ConstraintsIntersection object, consts <ValueRangeConstraint object, consts 0, 4294967295>>, payload [1]>, 'securityStateReference': 10612126, 'securityLevel': 3, 'contextEngineId': <SnmpEngineID value object, tagSet <TagSet object, tags 0:0:4>, subtypeSpec <ConstraintsIntersection object, consts <ValueSizeConstraint object, consts 0, 65535>, <ValueSizeConstraint object, consts 5, 32>>, encoding iso-8859-1, payload [0x80004fb8056f65...7665723153eea940]>, 'contextName': b'', 'msgUserName': <OctetString value object, tagSet <TagSet object, tags 0:0:4>, subtypeSpec <ConstraintsIntersection object, consts <ValueSizeConstraint object, consts 0, 32>>, encoding iso-8859-1, payload [trapuser]>, 'maxSizeResponseScopedPDU': 65398}
2024-08-08 22:08:43,463 pysnmp: returnResponsePdu: PDU <empty>
2024-08-08 22:08:43,463 pysnmp: prepareResponseMessage: stateReference 12688717
2024-08-08 22:08:43,463 pysnmp: StatusInformation: {'errorIndication': LoopTerminated('Infinite SNMP entities talk terminated')}
2024-08-08 22:08:43,464 pysnmp: prepareDataElements: error reported

[lextm]

It took us a while to validate this and confirm the results you provided.

Can you create a pull request so that we can merge your fix?

[y-iwata-bl]

Thank you for validating my fix.

Sure, I can do that. Please wait a while for me to make the request.

By the way, I have found additional similar potential issues as follows. I would like to include the additional fixes in the request as well. Is that okay?

diff -r pysnmp/proto/mpmod/rfc3412.py /usr/local/lib/python3.9/site-packages/pysnmp/proto/mpmod/rfc3412.py
402a403,406
>                 if securityModel in snmpEngine.securityModels:
>                     smHandler = snmpEngine.securityModels[securityModel]
>                     smHandler.releaseStateInformation(securityStateReference)
>
511a516,518
>             if securityModel in snmpEngine.securityModels:
>                 smHandler = snmpEngine.securityModels[securityModel]
>                 smHandler.releaseStateInformation(securityStateReference)

diff -r pysnmp/proto/secmod/rfc3414/service.py /usr/local/lib/python3.9/site-packages/pysnmp/proto/secmod/rfc3414/service.py
999a1000
>                         self._cache.pop(securityStateReference)
1024a1026
>                     self._cache.pop(securityStateReference)
1105a1108
>                 self._cache.pop(securityStateReference)
1206a1210
>                     self._cache.pop(securityStateReference)
1297a1302
>                     self._cache.pop(securityStateReference)
1368a1374
>                     self._cache.pop(securityStateReference)
1377a1384
>                 self._cache.pop(securityStateReference)
1382a1390
>                 self._cache.pop(securityStateReference)
1445a1454
>                 self._cache.pop(securityStateReference)
1452a1462
>                 self._cache.pop(securityStateReference)

[lextm]

Sure. Please feel free to include all changes you suggest. We will review them all together.

[lextm]

Fixed in #143. Further conversation will go on there.