Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

5 parser bug #6

Merged
merged 6 commits into from
Oct 11, 2024
Merged

5 parser bug #6

merged 6 commits into from
Oct 11, 2024

Conversation

lpicci96
Copy link
Collaborator

@lpicci96 lpicci96 commented Oct 11, 2024

  • Improves dtype formatting. Previously numeric cols that were not OBS_VALUE were set to int16 which was not sufficient to store values such as SCALE: 1000000. This change first coerces non numeric values to nulls and then converts specific columns to int64
  • Improves logging
  • Improved error handling

Closes #5

@lpicci96 lpicci96 linked an issue Oct 11, 2024 that may be closed by this pull request
@lpicci96 lpicci96 requested a review from jm-rivera October 11, 2024 09:35
@codecov-commenter
Copy link

codecov-commenter commented Oct 11, 2024

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

Attention: Patch coverage is 86.66667% with 2 lines in your changes missing coverage. Please review.

Project coverage is 90.71%. Comparing base (27d3606) to head (4613b3b).

Files with missing lines Patch % Lines
src/imf_reader/weo/parser.py 75.00% 1 Missing ⚠️
src/imf_reader/weo/reader.py 90.90% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main       #6      +/-   ##
==========================================
- Coverage   91.01%   90.71%   -0.31%     
==========================================
  Files           6        6              
  Lines         178      183       +5     
==========================================
+ Hits          162      166       +4     
- Misses         16       17       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@jm-rivera
Copy link
Contributor

Thanks @lpicci96 my only question would be about setting types vs doing something like on the importers package + pyarrow. That way we let that engine figure out the right data types, instead of hardcoding. What do you think?

@jm-rivera
Copy link
Contributor

@lpicci96 we should incorporate into bblocks

@lpicci96 lpicci96 merged commit 773678c into main Oct 11, 2024
6 checks passed
@lpicci96 lpicci96 deleted the 5-parser-bug branch October 11, 2024 12:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

parser bug
3 participants