Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add BUFR file creation script and fix unit test bug in test_get_bufr.py #263

Merged
merged 16 commits into from
Aug 12, 2024

Conversation

ladsmund
Copy link
Contributor

PR Description

  • New Feature: Added script for recreating BUFR files, create_bufr_files, as a CLI script in setup.py.
  • Bug Fix: Resolved issue in test_get_bufr.py that caused unit tests to fail.

@ladsmund ladsmund changed the title ### PR Title Add BUFR file creation script and fix unit test bug in test_get_bufr.py Add BUFR file creation script and fix unit test bug in test_get_bufr.py Jun 24, 2024
Copy link
Member

@BaptisteVandecrux BaptisteVandecrux left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

I did not have much to say about your create_bufr_files. But going through those files triggered a few questions that I listed below. Sorry if they fall outside the scope of the PR. Feel free to move them to issues if you want to deal with them later.

I'm approving anyway since the code is fully functional.

src/pypromice/postprocess/station_configurations.toml Outdated Show resolved Hide resolved
src/pypromice/postprocess/real_time_utilities.py Outdated Show resolved Hide resolved
src/pypromice/postprocess/get_bufr.py Outdated Show resolved Hide resolved
@ladsmund ladsmund force-pushed the feature/bufr_regenerate branch from 3958fb1 to b5de50d Compare July 3, 2024 13:27
@ladsmund ladsmund changed the base branch from main to develop July 3, 2024 13:29
@ladsmund ladsmund force-pushed the feature/bufr_regenerate branch 2 times, most recently from 34afcce to 0c3fcd5 Compare July 9, 2024 09:39
@ladsmund ladsmund force-pushed the feature/bufr_regenerate branch from 0c3fcd5 to e4adfe9 Compare August 6, 2024 11:29
PennyHow
PennyHow previously approved these changes Aug 12, 2024
Copy link
Member

@PennyHow PennyHow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

src/pypromice/postprocess/create_bufr_files.py Outdated Show resolved Hide resolved
Configuration variables were to strictly validated.
* Made bufr_integration_test explicit
* Updated read_bufr_file to use wmo_id as index
* Added corresponding unit tests
* Added flag to raise exceptions on errors
* Added create_bufr_files.py to setup
Updated station config:
* Added sonic_ranger_from_gps
* Changed height_of_gps_from_station_ground from 0 to 1
- Ensure get_bufr_variables raises AttributeError when station dimensions are missing
* Bedrock stations shouldn’t depend on the noisy GPS signal for elevation.
* Added station dimension values for WEG_B
* Added corresponding unittest
Added eccodes installation
* Removed station_configurations.toml from repository
* Updated bufr_utilities.set_station to validate wmo id
* Implemented StationConfig io tests
* Extracted StationConfiguration utils from get_bufr
* Added support for loading multiple station configuration files

Other
* Made ArgumentParser instantiation inline
* Added detailed descriptions with references to the attributes in BUFRVariables
* Change the attribute order to align with the exported schema
* Changed variable roundings to align with the scales defined in the BUFR schemas:
  * Latitude and longitude is set to 5. Was 6
  * heightOfStationGroundAboveMeanSeaLevel is set to 1. Was 2
  * heightOfBarometerAboveMeanSeaLevel is set to to 1. Was 2
  * pressure is set to -1. Was 1. Note: The BUFRVariable unit is Pa and not hPA
  * airTemperature is set to 2. Was 1.
  * heightOfSensorAboveLocalGroundOrDeckOfMarinePlatformTempRH is set to 2. Was 4
  * heightOfSensorAboveLocalGroundOrDeckOfMarinePlatformWSPD is set to 2. Was 4
 * Added unit tests to test the roundings
* Updated existing unit tests to align with corrected precision
* The station position determination (AWS_latest_locations) is separated from the bufr file export
* Updated the unit tests

Corrected minimum data check to allow p_i or t_i to be nan

Renamed process_station parameters for readability
* Rename now_timestamp -> target_timestamp
* Rename time_limit -> linear_regression_time_limit

Applied black
@ladsmund ladsmund force-pushed the feature/bufr_regenerate branch from e273da6 to 99659fa Compare August 12, 2024 13:56
@ladsmund ladsmund merged commit 09950de into develop Aug 12, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants