Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make ignore_comments not default (next major release) #74

Open
mverleg opened this issue Apr 5, 2020 · 2 comments
Open

Make ignore_comments not default (next major release) #74

mverleg opened this issue Apr 5, 2020 · 2 comments
Labels

Comments

@mverleg
Copy link
Owner

mverleg commented Apr 5, 2020

Stripping comments from files that are multiple megabytes takes an obscene amount of time, even if there are no comments.

I don't think I can make it much faster without writing a whole json parser (in something other than Python, or it can never compete).

So I'm thinking of something like:

  • Make ignore_comments=False the default
  • Ignore comments only for small files, unless ignore_comments is used to override
  • Only parse comments if the file starts with a comment, unless ignore_comments is used to override
  • Only strip comments on uncompressed files, unless ignore_comments is used to override
@mverleg
Copy link
Owner Author

mverleg commented Nov 3, 2022

As a way to address

  • Performance
  • Excessive deprecation messages

I think it might be good to always try to

  • Parse without stripping comments if ignore_comments is None
  • Then only if it fails (or ignore_comments is True) re-try with stripping comments

This way

  • The deprecation warning is only shown if strip_comments was None and there were actual comments (and only the first time)
  • The performance without comments is better, at the cost of performance of implicitly stripping comments

@mverleg
Copy link
Owner Author

mverleg commented Nov 4, 2022

Released v3.16.1 with this optimization,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant