Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stream out #63

Open
SymbolixAU opened this issue Feb 27, 2020 · 0 comments
Open

Stream out #63

SymbolixAU opened this issue Feb 27, 2020 · 0 comments

Comments

@SymbolixAU
Copy link
Collaborator

When parsing to large strings fails, need a method to stream to a connection

Just ran into this on to_ndjson in a production pipe:
Error in rcpp_to_ndjson(x, unbox, digits, numeric_dates, factors_as_string, : R character strings are limited to 2^31-1 bytes
This was an extremely large write and (remarkably) the first issue I've had in ~6Tb worth of json writes since I installed this branch, over lots of different data types/sizes/nesting structures/etc. So far it's all just working.

Originally posted by @sheffe in #29 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

0 participants