Skip to content

Commit

Permalink
updating with new data
Browse files Browse the repository at this point in the history
updating app with data from April 11
  • Loading branch information
cameronkruse committed May 20, 2024
1 parent 1784396 commit 9ebe15d
Show file tree
Hide file tree
Showing 21 changed files with 172,528 additions and 493,731 deletions.
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,8 @@ In the application, crop profiles were built using data from AgMIP. They assesse
- Roots & Tubers Cassava
- Cereals - Maize
- Legume - Soybean
Note that this data is actively being refined by the AgMIP team and with further research these values may be adjusted.

Note that all data is actively being refined by the AgMIP team and with further research these values may be adjusted.

## Additional data
A few additional data sources were used in the basemap of the application. Population data is from World Pop. Soil organic carbon and sand content data was sourced from ISRIC. This data was processed as raster data tilesets. More information on how we process raster data can be found in a previous blog post.
31,117 changes: 6,034 additions & 25,083 deletions data/crop-data-processing.ipynb

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion data/synced-data/crop-yield-data-quantiles.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion data/synced-data/crop-yield-data-ratio-quantiles.json

Large diffs are not rendered by default.

10,236 changes: 10,236 additions & 0 deletions data/synced-data/crop-yields-mean-models.csv

Large diffs are not rendered by default.

583,944 changes: 135,773 additions & 448,171 deletions data/synced-data/crop-yields-mean-models.geojson

Large diffs are not rendered by default.

11 changes: 9 additions & 2 deletions vacs-map-app/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,15 @@ _Disclaimer: there are a number of remnants from prototyping in this directory t
## Updating The Data
Most of this process is automated, but there are a few things to keep track of and make sure are in sync.

### Step 1: Data Processing
1. download raw data and replace existing data in folder `data/unsynced-data/crop-yield-data/` This directory is not synced with the repo, so you may have to create this directory and add the data.
2. open the notebook `data/crop-data-processing.ipynb` and run all cells
3. copy the newly updated `/data/synced-data/crop-yields-mean-models.csv` into the `vacs-map-app/public/data-raw` directory and continue with step 2

### Step 2: Updating App Data

**If crop yields data gets updated, but crops are all the same:**
1. replace `public/data-raw/crop-yields-mean-models.csv`
1. ensure the `public/data-raw/crop-yields-mean-models.csv` has been replaced with the new data
2. run `node scripts/format-yields-data.js` (this files contains an input filename which can be edited)
3. update minimaps (see below)

Expand All @@ -64,7 +71,7 @@ Most of this process is automated, but there are a few things to keep track of a
2. update the other crop-info files in `public/data` (general, genetic, nutritional, regional-potential)
- these files come directly from [this spreadsheet](https://docs.google.com/spreadsheets/d/1mU91Pbc--wN14cb_sNCVWaScF0nwho7T/edit#gid=892630347). Make sure each spreadsheet has a row for all desired crops, and that the `id` field matches the crop variable name used in the yields data. Once they are updated, download each sheet as a csv and replace the crop-info files, leaving the filenames as they were.

**Updating minimaps**
### Step 3: Updating Minimaps
1. run `node scripts/generate-minimaps.js`
2. because this script generates a new image for each crop in each scenario and using both the default and colorblind friendly color schemes, it can reach the js stack limit if all are generated at once
3. to deal with this there are some arrays at the top of the file which control the minimaps that are generated in a run of the script (see comments there for more detail)
Expand Down
20,472 changes: 10,236 additions & 10,236 deletions vacs-map-app/public/data-raw/crop-yields-mean-models.csv

Large diffs are not rendered by default.

20,472 changes: 10,236 additions & 10,236 deletions vacs-map-app/public/data/crop-yields-mean-models.csv

Large diffs are not rendered by default.

Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified vacs-map-app/public/minimaps/default/okra_future_ssp126.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified vacs-map-app/public/minimaps/default/okra_future_ssp370.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified vacs-map-app/public/minimaps/default/pigeonpea_future_ssp126.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified vacs-map-app/public/minimaps/default/pigeonpea_future_ssp370.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified vacs-map-app/public/minimaps/default/soybean_future_ssp126.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified vacs-map-app/public/minimaps/default/soybean_future_ssp370.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified vacs-map-app/public/minimaps/default/yams_future_ssp126.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified vacs-map-app/public/minimaps/default/yams_future_ssp370.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 9ebe15d

Please sign in to comment.