-
Notifications
You must be signed in to change notification settings - Fork 912
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Adding assertion to check for regular JSON inputs of size greater than INT_MAX
bytes
#17057
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor comment, may need style fix. LGTM otherwise!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
does check_input_size
need an update as well ? because it uses
std::numeric_limits<cudf::io::json::SymbolOffsetT>::max()
aka uint32_t
That's correct, I've updated |
/merge |
Description
Addresses #17017
Libcudf does not support parsing regular JSON inputs of size greater than
INT_MAX
bytes. Note that the batched reader can only be used for JSON lines inputs.Checklist