Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Version 1 4 1 drivers 1km 2023 tcl update #58

Open
wants to merge 27 commits into
base: master
Choose a base branch
from
Open

Version 1 4 1 drivers 1km 2023 tcl update #58

wants to merge 27 commits into from

Conversation

melrose-wri
Copy link
Member

@melrose-wri melrose-wri commented Jan 2, 2025 •
edited
Loading

Pull request checklist

Please check if your PR fulfills the following requirements:

  • Make sure you are requesting to pull a topic/feature/bugfix branch (right side). Don't request your master!
  • Make sure you are making a pull request against the develop branch (left side). Also you should start your branch off our develop.
  • Check the commit's or even all commits' message styles matches our requested structure.
  • Check your code additions will fail neither code linting checks nor unit test.

Pull request type

Please check the type of change your PR introduces:

  • Bugfix
  • Feature
  • Code style update (formatting, renaming)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • Documentation content changes
  • Other (please describe):

What is the current behavior?

The model currently uses an older version of the 1km drivers of tree cover loss data (driver tile processing date: 20241121) which was trained on tree cover loss data through 2022.

What is the new behavior?

The 1km drivers of tree cover loss data was retrained on tree cover loss data through 2023 (driver tile processing date: 20241224) resulting in changes to the classification. In this version, the model has been updated to use the newest version of the 1 km drivers of tree cover loss map (updated through 2023) in the gross emissions calculations and subsequent steps.

Does this introduce a breaking change?

  • Yes
  • No

Other information

melrose-wri and others added 27 commits July 15, 2024 14:49
@melrose-wri
Readme Update:
0a1ea90 
Changed typo in Purpose and Scope section from NH4 to CH4.
@melrose-wri
Update from previous Curtis driver to 1km drivers:
429bacf 
-Updates drivers path and pattern in constants_and_names.py and constants.h
-Added hard commodites flu value and new drivers class output paths to constants.h
-Added new drivers classes (hard commodites and other natural disturbances) to preprocessing steps in calc_gross_emissions_generic.cpp (lines 1 - 530).
-TODO: Update equations in calc_gross_emissions_generic.cpp from line 530 on.
-Changed permanant agriculture (formerly commodity driven deforestation) flu vlaues in flu_val.cpp with updated values from IPCC 2019 Refinement for boreal/ temperate and tropical forests.
-TODO: Decide what flu value to use for (climate_zone == 9) // Montane
-TODO: Update equations.cpp file with information for new drivers.
@melrose-wri
Commit:
1e42338 
- Changed drivers path and patterns and Flu values back to 10km drivers inputs for testing purposes
- Deleted calc_gross_emissions_no_shifting_ag.cpp and calc_gross_emissions_convert_to_grassland.cpp since they are outdated and no longer being used for sensitivity analysis
- Updated dataset, gdal, etc names in calc_gross_emissions_generic.cpp from int values to descriptive names (e.g. INGDAL1 >> INGDAL_AGC)
- Added empty objects for CH4only and N2O only output but they have not been added to the decision tree/ calculation yet
@melrose-wri
Commit:
d6f49ee 
- Changed python files for gross emission calculations
- Successfully ran 1 tile locally using 10km drivers with new c++ code for biomass_soil. Processing time was slightly longer than v1.3.2 (6:42 vs 8:50) so pushing these changes so that I can test biomass_soil for 00N_000E on a spot machine.
- TODOs: QC output then update soil_only c++ code and run with soil_only version
@melrose-wri
Commit:
7d42cd6 
- Ran previous commit in EC2 instance and calc_gross_emissions_generic.cpp couldn't compile so I realized that the previous local run was from the old calc_gross_emissions_generic.exe
- Fixed bugs so that new calc_gross_emissions_generic.cpp can compile
- Successfully ran 1 tile with -sp then 3 tiles with mp locally using 10km drivers with new c++ code for biomass_soil. Processing time was approx 4 minutes faster than old code (5:05 vs 8:50).
- TODOs: QC output then update soil_only c++ code and run with soil_only version
@melrose-wri
Commit:
9eba1b2 
- Successfully ran calc_gross_emissions_soil_only.cpp for 00N_110E with -sp using 10km drivers
@melrose-wri
Commit:
e3d953b 
- Changing emis_run_date_biomass_soil in constants_and_names.py to '20248888' for full ec2 run
@melrose-wri
Commit:
2cdafa8 
- In constants_and_names.py, updated version to 1.4.0, changed drivers to 1km drivers and deleted driver-specific emissions output
- In constants.h, changed drivers to 1km drivers and deleted driver-specific emissions output
- In flu_val.cpp, updated flu values for 1km drivers. Note: Currently using averages for "montane" climate zone, but in the future we will reclassify "montane" as "dry" or "moist/wet" and use those values instead of using averages.
- In calc_gross_emissions_generic.cpp and calc_gross_emissions_soil_only.cpp, updated decision tree for permanent agriculture, shifting cultivation, forest management, wildfire, and settlements & infrastructure. Updated non-co2, fire emissions equation for forest management and wildfire to include litter and deadwood.
TODO: Still need to update decision tree and node codes for  hard commodities, other natural disturbances, and no driver. Also need to update equation.cpp.
@melrose-wri
Commit:
9a050b7 
- Updated calc_gross_emissions_generic.cpp, calc_gross_emissions_soil_only.cpp, and node codes with updated decision tree for hard commodities, other natural disturbances, and no driver. Added commented out code w/ TODOs to split non co2 emissions into CH4 only and N2O only.
- Updated constants_and_names.py with dir and pattern for soil_only CH4_only and N2O_only outputs to use after splitting non co2 emissions.
- Updated equations.cpp with new drivers classes. Changed combustion factor for other natural disturbances/ no drivers to boreal = 0.34 and temperate = 0.45. Added commented out code w/ TODOs to split non co2 emissions into CH4 only and N2O only. Q: How many sig figs to use for peat_drain_annual_CH4_only and peat_drain_annual_N2O_only
- Added commented out code w/ TODOs to split non co2 emissions into CH4 only and N2O only in  mp_calculate_gross_emissions.py.
@melrose-wri
Commit:
40f7d4e 
- Fixed type issue (non_soil_bgc_c[x] to non_soil_bgc_c). calc_gross_emissions_generic.cpp compiles correctly now. calc_gross_emissions_soil_only.cpp also compiles
@melrose-wri
Commit:
840d0dc 
- Updated run_full_model.py to run test tiles with master script on spot machine from emissions stage onwards using new drivers
@melrose-wri
Commit:
9c2e056 
- Updated run_full_model.py to run test tiles with master script on spot machine from emissions stage onwards using new drivers
@melrose-wri
Commit:
7e0bc68 
- Deleted 10km drivers path/pattern and driver-specific outputs for soil_only mode in constants_and_names.py.
- Added a comment in calc_gross_emissions_generic.cpp, calc_gross_emissions_soil_only.cpp, and equations.cpp where peatburn_CH4_only is intitialized that there are no N2O emissions from burning peat. That way, we don't wonder in the future why N2O emissions has one less term than CH4 emissions (i.e. no peatburn_N2O_only)
- Added note about making unassigned driver the same as other natural disturbances in calc_gross_emissions_generic.cpp and calc_gross_emissions_soil_only.cpp
- Added link to the carbon_budget_flowchart.pptx at the top of the constants.h, equations.cpp, and flu_val.cpp since all three files use inputs from those slides.
- Added note to constants.h and flu_val.cpp that the function in flu_val.cpp is specifically for the permanent agriculture driver
- Updated peat_drain_annual_non_CO2 to 2 significant figures in equations.cpp
Both the calc_gross_emissions_generic.cpp and calc_gross_emissions_soil_only.cpp compile without any errors
@melrose-wri
Commit:
a09bfd9 
- Updated all relevant files to split the non-co2 emissions into CH4-only and N2O-only emissions output. Both the calc_gross_emissions_generic.cpp and the calc_gross_emissions_soil_only.cpp files compile correctly.
@melrose-wri
Commit:
a850a29 
- Updated emis_run_date in constants_and_names.py to run updated output with 1km drivers and seperate output for CH4 and N2O
@melrose-wri
Commit:
1ffeeab 
- Updated flux output dates in constants_and_names.py to run updated output with 1km drivers and seperate output for CH4 and N2O (forgot to include in last commit)
@melrose-wri
Commit:
768b47d 
- During the full tile run, got the following error when trying to download the 1km tiles:"An error occurred (AccessDenied) when calling the GetObject operation: Access Denied". Troubleshooted in a small ec2 instance and realized that the --no-sign-request flag in the s3_folder_download utility was causing the issue. Added an additional if statement in the s3_folder_download utility without the --no-sign-request flag so that driver tiles can be downloaded.
- soil_only CH4 and N2O rasters were not being uploaded to s3 even though they were created during the mode run. Realized that the upload tile step was looking for tiles with "gross_emis_ch4_only_all_drivers_Mg_CO2e_ha_soil_only_2001_23 pattern" but the tiles being created by the C++ code had pattern "gross_emis_CH4_only_all_drivers_Mg_CO2e_ha_soil_only_2001_23 pattern" (ie capitalized GHG). Changed pattern_gross_emis_ch4_only_all_drivers_soil_only and pattern_gross_emis_n2o_only_all_drivers_soil_only in constants_and_names.py from lower cased ch4 and n2o to capitalized CH4 and N2O.
@melrose-wri
Commit:
af691a9 
- Changed drivers path and pattern in constants_and_names.py and constants.h from 20241004 to 20241121
@melrose-wri
Merge pull request #55 from wri/version_1_4_0_drivers_1km
926494d 
Version 1 4 0 drivers 1km
@dagibbs22
Update readme.md
c58734d
@dagibbs22
Merge branch 'master' into develop
96950dc
@melrose-wri
Commit:
77ac3fa 
- Changed drivers path and pattern in constants_and_names.py to use the new version which includes TCL training data through 2023.
 - Chenged all outputs from the emissions stage onward from using the data 20241121 to using a test date of 20248888
@melrose-wri
Commit:
f9c95e9 
- Changed drivers path and pattern in constants.h to use the new version which includes TCL training data through 2023.
@melrose-wri
Commit:
503e1c1 
- Changed drivers path and pattern in constants_and_names.py to use the new version (20241224) which includes TCL training data through 2023.
@melrose-wri
Commit:
1471cf7 
- Changed version to 1.4.1
- Changed test date 20248888 to run date 20241230 in constants_and_names.py
@melrose-wri
Merge branch 'develop' into version_1_4_1_drivers_1km_2023_TCL_update
5b1a751
@melrose-wri
Merge pull request #57 from wri/version_1_4_1_drivers_1km_2023_TCL_up…
ed854ba 
…date

Version 1 4 1 drivers 1km 2023 tcl update
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@melrose-wri @dagibbs22