syncing an entity with large # objects causes server error

[Rollaway]

When authentication and / or networking is turned off and objects managed by simperium are created, enabling authentication and networking later may generate an error caused by simperium trying to sync that entity in a single POST. If the number of objects is large enough, the POST becomes too large for the server to handle and a "413 Request Entity Too Large" error is returned while simperium continues to try sending.

[Rjonhson]

You are not using websocks, are you?

[Rollaway]

No.

[Rjonhson]

You should!

[Rollaway]

And that would alleviate this issue...?

[mikejohnstn]

@Rollaway Yes, the HTTP interface for Simperium will soon be deprecated. We strongly recommend enabling websockets in the latest develop branch: [simperium setUseWebSockets: YES];

Sorry this wasn't clear. We'll be doing this and updating docs in the coming weeks.

Could you please try doing this and confirm it fixes your problem?

[Rollaway]

@mikejohnstn I made the change and I no longer get the same POST error. But it still appears to be doing the same thing in that after a certain number of objects are synced, the JSON String saved to NSUserDefaults as changesPending- doesn't get smaller as you would expect as the objects are synced. It does at first then stays the same at around 1200 objects processed. It just keeps spinning through the change processor until memory is exhausted and the app crashes. Let me know if you'd like to see logs or parts of a log (they get large fast!).

[mikejohnstn]

Thanks @Rollaway. With 1200 offline objects, there are going to be performance/memory issues with NSUserDefaults access. @jleandroperez could you add this as a test at some point to see if we can reproduce the crash?

We have #124 to move metadata storage to the database. Once that's fixed, I believe this will be fixed as well, but let's leave this open as a reminder.

[Rollaway]

Good stuff...thanks @mikejohnstn!

[jleandroperez]

@Rollaway i'll be working on a unit test this week, perhaps we can reproduce this crash. Just in case, did you get a crashlog generated?. Did you get any low memory warnings? or it was a bad access crash?.

Thanks!

[Rjonhson]

Memory gets exhausted fast indeed. Testing on an iPad mini, with 4000 objects, 2000 in each entity, in no time it hits almost 200MB of live bytes and shortly after this it crashes.

[jleandroperez]

@Rjonhson Sounds fair enough. I'll be working on this ASAP, perhaps we've got a retain cycle somewhere (the whole project has gone ARC enabled just a couple weeks ago!). Thanks for the feedback!

[Rjonhson]

@jleandroperez Alright! let me know if you need anything!

[Rollaway]

You guys are great! Thanks for getting right on this...and let me know if you need any logs, etc.

[jleandroperez]

We'll be working on merging the 'Memory Footprint' pull request, in the next couple weeks. @Rollaway please, if you're still experiencing any "413 Request Entity Too Large" erros, just let us know.

Thanks!