Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Chore/writer optimization #83

Merged
merged 12 commits into from
Jan 12, 2024
Merged

Chore/writer optimization #83

merged 12 commits into from
Jan 12, 2024

Conversation

picciama
Copy link
Contributor

@picciama picciama commented Jan 12, 2024

PR Checklist

  • This comment contains a description of changes (with reason)
  • Referenced issue is linked
  • If you've fixed a bug or added code that should be tested, add tests!
  • Documentation in docs is updated

Description of changes

  • rewritten spectronaut and msp writer to make them faster
  • fixes a bug when reading instrumentConfiguration from mzML (upper/lowercase) issue depending on ThermoRawFileParser / MSConvert

Technical details

  • removed the intermediate pandas dataframe creation
  • offer async writing while keeping file handle open
  • only write many lines at a time
  • support async writing by accepting iput from a queue and providing a threading function
  • numpy vectorization

Additional context

@picciama picciama merged commit 69e81c5 into development Jan 12, 2024
28 checks passed
@picciama picciama deleted the chore/writer_optimization branch January 29, 2024 08:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CE calibrating FragPipe results generated from mzML formatted data fails
1 participant