BUG : Corrupted events leads to stuck batch uploads with status code 400 and error Request neither has anonymousId nor userId

[bradtgmurray]

Describe the bug
Our Android app wasn't successfully sending analytics at all with the Rudderstack SDK.

Looks like we somehow corrupted the events database, and then the batch send loop got stuck on these invalid events on the response Request to endpoint https://rudderstack.beeper-tools.com/v1/batch failed with status code 400 and error Request neither has anonymousId nor userId.

We did have some bad proguard rules that we think may have created this bad data, and have since fixed those rules, but even then having this upload loop get stuck is unfortunate.

I would propose that if events without a userId/anonymousId were invalid they shouldn't be allowed to be submitted to the database for later sending, or if we get a bad request response from the data plane we drop the data instead of just trying to send the same bad payload again in the future?

13:03:04.735 RudderSDK       W  Warn: CloudModeManager: cloudModeProcessor: Retrying in 1s
13:03:05.737 RudderSDK       D  Debug: DBPersistentManager: getDBRecordCount: countSQL: SELECT count(*) FROM events ;
13:03:05.740 RudderSDK       I  Info: DBPersistentManager: getDBRecordCount: fetched count from DB
13:03:05.741 RudderSDK       D  Debug: CloudModeManager: getPayloadFromMessages: DBRecordCount: 1113
13:03:05.741 RudderSDK       D  Debug: CloudModeManager: cloudModeProcessor: Fetching events to flush to server
13:03:05.741 RudderSDK       D  Debug: DBPersistentManager: fetchCloudModeEventsFromDB: selectSQL: SELECT * FROM events WHERE status IN (0, 1) ORDER BY updated ASC LIMIT 30
13:03:05.748 RudderSDK       I  Info: DBPersistentManager: fetchEventsFromDB: fetched messages from DB
13:03:05.750 RudderSDK       D  Debug: FlushUtils: getPayloadFromMessages: recordCount: 30
13:03:05.753 RudderSDK       D  Debug: FlushUtils: getPayloadFromMessages: sentAtTimestamp: 2023-11-01T17:03:05.751Z
13:03:05.756 RudderSDK       D  Debug: CloudModeManager: cloudModeProcessor: payload: {"sentAt":"2023-11-01T17:03:05.751Z","batch": [{"context":{"traits":{},"device":{}},"type":"track","event":"Application Installed","properties":{"build":30159,"version":"3.0.159"},"integrations":{"All":true},"sentAt":"2023-11-01T17:03:05.751Z"},{"context":{"traits":{},"device":{}},"type":"track","event":"Application Opened","properties":{"from_background":false,"version":"3.0.159"},"integrations":{"All":true},"sentAt":"2023-11-01T17:03:05.751Z"},{"context":{"traits":{},"device":{}},"type":"identify","event":"identify","integrations":{"All":true},"sentAt":"2023-11-01T17:03:05.751Z"},{"context":{"traits":{},"device":{}},...
13:03:05.757 RudderSDK       I  Info: CloudModeManager: cloudModeProcessor: 30
13:03:05.757 RudderSDK       D  Debug: RudderNetworkManager: sendNetworkRequest: Request URL: https://rudderstack.beeper-tools.com/v1/batch
13:03:05.758 RudderSDK       D  Debug: RudderNetworkManager: sendNetworkRequest: Gzip is enabled
13:03:05.913 RudderSDK       E  Error: RudderNetworkManager: sendNetworkRequest: Request to endpoint https://rudderstack.beeper-tools.com/v1/batch failed with status code 400 and error Request neither has anonymousId nor userId
13:03:05.913 RudderSDK       I  Info: CloudModeManager: cloudModeProcessor: ServerResponse: 400
13:03:05.923 RudderSDK       D  Debug: CloudModeManager: cloudModeProcessor: SleepCount: 11
13:03:05.923 RudderSDK       W  Warn: CloudModeManager: cloudModeProcessor: Retrying in 1s
13:03:06.925 RudderSDK       D  Debug: DBPersistentManager: getDBRecordCount: countSQL: SELECT count(*) FROM events ;
13:03:06.929 RudderSDK       I  Info: DBPersistentManager: getDBRecordCount: fetched count from DB
13:03:06.929 RudderSDK       D  Debug: CloudModeManager: getPayloadFromMessages: DBRecordCount: 1113
13:03:06.930 RudderSDK       D  Debug: CloudModeManager: cloudModeProcessor: Fetching events to flush to server
13:03:06.930 RudderSDK       D  Debug: DBPersistentManager: fetchCloudModeEventsFromDB: selectSQL: SELECT * FROM events WHERE status IN (0, 1) ORDER BY updated ASC LIMIT 30
13:03:06.938 RudderSDK       I  Info: DBPersistentManager: fetchEventsFromDB: fetched messages from DB
13:03:06.940 RudderSDK       D  Debug: FlushUtils: getPayloadFromMessages: recordCount: 30
13:03:06.944 RudderSDK       D  Debug: FlushUtils: getPayloadFromMessages: sentAtTimestamp: 2023-11-01T17:03:06.942Z
13:03:06.947 RudderSDK       D  Debug: CloudModeManager: cloudModeProcessor: payload: {"sentAt":"2023-11-01T17:03:06.942Z","batch": [{"context":{"traits":{},"device":{}},"type":"track","event":"Application Installed","properties":{"build":30159,"version":"3.0.159"},"integrations":{"All":true},"sentAt":"2023-11-01T17:03:06.942Z"},{"context":{"traits":{},"device":{}},"type":"track","event":"Application Opened","properties":{"from_background":false,"version":"3.0.159"},"integrations":{"All":true},"sentAt":"2023-11-01T17:03:06.942Z"},{"context":{"traits":{},"device":{}},"type":"identify","event":"identify","integrations":{"All":true},"sentAt":"2023-11-01T17:03:06.942Z"},{"context":{"traits":{},"device":{}},...
13:03:06.947 RudderSDK       I  Info: CloudModeManager: cloudModeProcessor: 30
13:03:06.947 RudderSDK       D  Debug: RudderNetworkManager: sendNetworkRequest: Request URL: https://rudderstack.beeper-tools.com/v1/batch
13:03:06.948 RudderSDK       D  Debug: RudderNetworkManager: sendNetworkRequest: Gzip is enabled
13:03:07.106 RudderSDK       E  Error: RudderNetworkManager: sendNetworkRequest: Request to endpoint https://rudderstack.beeper-tools.com/v1/batch failed with status code 400 and error Request neither has anonymousId nor userId
13:03:07.106 RudderSDK       I  Info: CloudModeManager: cloudModeProcessor: ServerResponse: 400
13:03:07.115 RudderSDK       D  Debug: CloudModeManager: cloudModeProcessor: SleepCount: 12

To Reproduce
Not sure how we managed to get users into this state, might have to be a speculative fix.

Expected behavior
The SDK shouldn't get stuck in a way that requires the app to be uninstalled to get events flowing.

Screenshots
n/a

Version of the Android SDK
rudderstack = { module = "com.rudderstack.android.sdk:core", version = "1.20.1" }

[itsdebs]

We are looking into it

[desusai7]

Hey @bradtgmurray,

We handled this issue in version 1.21.1 of the Android SDK, can you try upgrading to this version and let us know how this goes?

[Marus]

We've been occasionally seeing this same issue on the iOS side

[desusai7]

Hey @Marus,

On which version of iOS SDK were you seeing this issue and Can you try upgrading to the latest version of the iOS SDK and verify if that resolves the issue ?

[mabdullahsaeed]

Hi @desusai7 we were using iOS SDK version 2.4.0. but downgraded to 1.8.0 after finding out that versions > 2.0.0 are in beta.
After downgrading we have been continuously seeing these errors. Any idea on how to resolve this?

[desusai7]

Hey @mabdullahsaeed,

Can you please try moving to latest version of the iOS SDK (1.26.1) and see if you are still seeing the same issue ?

Also, please open an issue in iOS SDK repo, if you are still facing the same issue.