Skip to content
This repository has been archived by the owner on Apr 8, 2020. It is now read-only.

Max-size determination and storage algorithm to be hardened. #341

Closed
MartijnR opened this issue Nov 5, 2015 · 9 comments
Closed

Max-size determination and storage algorithm to be hardened. #341

MartijnR opened this issue Nov 5, 2015 · 9 comments
Labels
Milestone

Comments

@MartijnR
Copy link
Member

MartijnR commented Nov 5, 2015

Needs to check regularly for the max-size that is supported by the server, and especially if the maxSize value stored in indexedDb is null for any reason.

It might be good to not store the default size in the db (if this is done currently - it might not be).

Investigate why max-size retrieval often fails (or maybe for certain OpenRosa servers?)

@MartijnR MartijnR added this to the KoBoSurge milestone Nov 5, 2015
@MartijnR
Copy link
Member Author

MartijnR commented Nov 6, 2015

Also check if HEAD request to /submission passes credentials (Enketo Legacy doesn't).

@MartijnR
Copy link
Member Author

MartijnR commented Nov 6, 2015

That bug is not present in Enketo Express

@MartijnR MartijnR added the KoBo label Nov 12, 2015
@MartijnR
Copy link
Member Author

#370

@ukanga
Copy link

ukanga commented Jan 20, 2016

Is this related to X-OpenRosa-Accept-Content-Length? ODK Collect usually checks this and sends the submission with attachments in batches, each batch is of a size less than the specified content length.
X-OpenRosa-Accept-Content-Length should be available in the head request of pretty much any OpenRosa FormSubmissionAPI.

@MartijnR
Copy link
Member Author

Yes. Enketo checks this too.
On Jan 20, 2016 2:56 AM, "ukanga" [email protected] wrote:

Is this related to X-OpenRosa-Accept-Content-Length? ODK Collect usually
checks this and sends the submission with attachments in batches, each
batch is of a size less than the specified content length.
X-OpenRosa-Accept-Content-Length should be available in the head request
of pretty much any OpenRosa FormSubmissionAPI.


Reply to this email directly or view it on GitHub
#341 (comment)
.

Revolutionizing data collection since 2012.

Enketo https://enketo.org/ | LinkedIn
http://www.linkedin.com/company/enketo-llc | GitHub
https://github.com/enketo | Twitter https://twitter.com/enketo
| Blog http://blog.enketo.org/

@ukanga
Copy link

ukanga commented Jan 20, 2016

Does enketo do the upload in batches?

@MartijnR
Copy link
Member Author

Yes
On Jan 20, 2016 10:58 AM, "ukanga" [email protected] wrote:

Does enketo do the upload in batches?


Reply to this email directly or view it on GitHub
#341 (comment)
.

Revolutionizing data collection since 2012.

Enketo https://enketo.org/ | LinkedIn
http://www.linkedin.com/company/enketo-llc | GitHub
https://github.com/enketo | Twitter https://twitter.com/enketo
| Blog http://blog.enketo.org/

@MartijnR
Copy link
Member Author

issues found (but no actual bugs):

  • maxSize is stored in Db and used in offline-capable views (necessary for file input!). It is only updated when the formCache is refreshed. This seems to make sense. Maybe consider not storing a default size though, as it seems better to attempt to request it when it's missing. Figure out how the maxSize value actually gets stored though. Is it actually stored the first time a form is launched?
  • the logic for previews to avoid requesting maxSize in connection.js is weird and wrong for e.g enke.to/preview/::abcd -> make this smarter, e.g. determine type of view (if settings.type === 'preview')
  • let Enketo server not set a default size (in communicator.js), but return null or something
  • file-manager.js also sets a default size. Not necessary?
  • at least log an error to the console, if a response was received from Enketo server (= online) && Ona/KoBo did not include header or did not return a response or returned an invalid response.

@MartijnR
Copy link
Member Author

The only 'bug' was the fact that under certain very rare circumstances the default max submission size gets stored in db and won't update until the form changes.

That rare circumstance is: the OpenRosa server does not serve the X-OpenRosa-Accept-Content-Length header (or does not return a response to HEAD /submission) at exactly the moment the form is cached (or re-cached).

MartijnR added a commit to OpenClinica/enketo-express-oc that referenced this issue Feb 23, 2016
closes kobotoolbox#341

- never store default max size in db (offline-capable views)
- always attempt to obtain max size from server if not stored in db (offline-capable views)
- only set default max size at 1 point in code (instead of 3)
- do not make submission/max-size request during previews without enketo ID
- log error to console if OpenRosa server does not return X-OpenRosa-Accept-Content-Length header
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

No branches or pull requests

2 participants