Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[M1] Performance optimizations (if necessary) #2308

Open
Tracked by #1919
matt-codecov opened this issue Aug 16, 2024 · 0 comments
Open
Tracked by #1919

[M1] Performance optimizations (if necessary) #2308

matt-codecov opened this issue Aug 16, 2024 · 0 comments

Comments

@matt-codecov
Copy link

matt-codecov commented Aug 16, 2024

Ideas that come up during M1 that we want to throw somewhere so they aren't forgotten.

Matt's ideas

  • Throw inserts on a SQLite worker thread
  • Get rid of joint PK in measurement tables in favor of rowid [wip] get rid of joint PK, just use local_*_id codecov-rs#15
  • Make chunks parser a lazy iterator over LineSessionModelSet (or whatever the type is) (example)
    • Gets the side effects out of the parser
    • Batches inserts more efficiently; you might be able to get through 5 or 10 chunks before needing to insert and start a new batch
    • May mean not using winnow anymore, or using it less

Arpad's ideas

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant