forked from AlabamaWaterInstitute/CloudInfra
-
Notifications
You must be signed in to change notification settings - Fork 19
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
b80cce9
commit c91299b
Showing
4 changed files
with
105 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
Write-Host "$(Get-Date) Downloading Run Package AWI_16_2863657_007.tar.gz" | ||
Invoke-WebRequest https://ciroh-ua-ngen-data.s3.us-east-2.amazonaws.com/AWI-007/AWI_16_2863657_007.tar.gz -o AWI_16_2863657_007.tar.gz | ||
Write-Host "$(Get-Date) Decompressing Run Package AWI_16_2863657_007.tar.gz" | ||
tar -xf .\AWI_16_2863657_007.tar.gz | ||
rm .\AWI_16_2863657_007.tar.gz | ||
cd .\AWI_16_2863657_007\ | ||
Write-Host "$(Get-Date) Running NextGen National Water Model Framework Package AWI_16_2863657_007.tar.gz" | ||
docker run --rm -it -v .:/ngen/ngen/data awiciroh/ciroh-ngen-image:latest /ngen/ngen/data auto | ||
Write-Host "Finished at $(Get-Date)" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,47 @@ | ||
{ | ||
"RUN_CONFIG": { | ||
"description": "Run Config for Hurricane Zach", | ||
"Model": { | ||
"realization": { | ||
"Name": "CFE_SLOTH_realization", | ||
"Type": "filesystem", | ||
"Path": "/ngen/realization.json", | ||
"Hash": "d2c6fbda93c134de495d69745fae11087784d2aa" | ||
}, | ||
"configuration": { | ||
"Name": "CFE_SLOTH_ini", | ||
"Type": "filesystem", | ||
"Path": "/ngen/cfe_sloth.ini", | ||
"Hash": "ab0c3dff59c4b282b172b90128159fda3386d012" | ||
} | ||
}, | ||
"Forcings": { | ||
"inputs": { | ||
"Name": "Hurricane Zach", | ||
"Type": "bucket", | ||
"Path": "s3://awi-ciroh-ngen-data/AWI_001/forcings/", | ||
"Hash": "220fff8bdd3b85f23d93e73b4bc7e3bc2c7c0f35" | ||
}, | ||
"Hydrofabric": { | ||
"catchment": { | ||
"Name": "Catchment(s) File v1.0", | ||
"Type": "bucket", | ||
"Path": "s3://awi-ciroh-ngen-data/AWI_001/catchments.geojson", | ||
"Hash": "da39a3ee5e6b4b0d3255bfef95601890afd80709" | ||
}, | ||
"nexus": { | ||
"Name": "Nexus File v1.0", | ||
"Type": "bucket", | ||
"Path": "s3://awi-ciroh-ngen-data/AWI_001/nexus.geojson", | ||
"Hash": "cae054f62f697080d822fea9c7d9c268be8b7ac9" | ||
}, | ||
"crosswalk": { | ||
"Name": "Crosswalk File v1.0", | ||
"Type": "bucket", | ||
"Path": "s3://awi-ciroh-ngen-data/AWI_001/crosswalk.geojson", | ||
"Hash": "4c39964d1e30779f9992d3c00e94a39952cb102a" | ||
} | ||
} | ||
} | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
--- CONFIG START --- | ||
### Model | ||
| Name of Model files | type | PATH to .json file | HASH of file | | ||
| ----------- | ----------- | ----------- | ----------- | | ||
| CFE_SLOTH_realization | filesystem | /ngen/realization.json | d2c6fbda93c134de495d69745fae11087784d2aa | | ||
| CFE_SLOTH_ini | filesystem | /ngen/cfe_sloth.ini | ab0c3dff59c4b282b172b90128159fda3386d012 | | ||
### Forcings | ||
| Name of Forcings | type | PATH to forcings file(s) | HASH of file(s) | | ||
| ----------- | ----------- | ----------- | ----------- | | ||
| Hurricane Zach | bucket | s3://awi-ciroh-ngen-data/AWI_001/forcings/ | 220fff8bdd3b85f23d93e73b4bc7e3bc2c7c0f35 | | ||
### Hydrofabric | ||
| Name of Hydrofabric | type | PATH to .json file | HASH of file | | ||
| ----------- | ----------- | ----------- | ----------- | | ||
| Catchment(s) File | bucket | s3://awi-ciroh-ngen-data/AWI_001/catchments.geojson | da39a3ee5e6b4b0d3255bfef95601890afd80709 | | ||
| Nexus File | bucket | s3://awi-ciroh-ngen-data/AWI_001/nexus.geojson | cae054f62f697080d822fea9c7d9c268be8b7ac9 | | ||
| Crosswalk File | bucket | s3://awi-ciroh-ngen-data/AWI_001/crosswalk.geojson | 4c39964d1e30779f9992d3c00e94a39952cb102a | | ||
|
||
All values within each section are defined for the run to evaluate when a run is unique and the change that makes it so. For example changing the hash of any of the fields for the referenced files should change the RUN CONFIG into something new. | ||
|
||
Note: the Realization, Catchment, Nexus and other model-required files must be searching by name. | ||
|
||
| Valid | Not Valid | | ||
| ----- | --------- | | ||
| hurricane_marty_realization.json | hurricane_marty.json | | ||
| Houston_catchments.geojson | Houston.geojson | | ||
| Nexus_2012_flood.geojson | 2012_flood.geojson | |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
# Notes on a "NextGen Universal Package" | ||
|
||
There's been some discussion in the community about the combinatorial effects of opening up the possibility to build ever-more-complicated systems while making it more difficult to resproduce results in the community, or to share results for integration into our collective understanding of the world. | ||
|
||
The first step to building that collective understanding is based on having the same interpretation and categorization of the data we have and will produce. | ||
|
||
Firstly there must be a heirarchical underestanding of how we represent data and configurations. Note that these aren't 1:1 with the data and configurations, these are for the heirarchical framing of how those are organized. | ||
|
||
The identified components are as follows: | ||
|
||
- _Data_ | ||
- Hydrofabric | ||
- Underlying physical understanding of the domain | ||
- Forcings | ||
- Input data over a temporal and spacial domain | ||
- _Configuration_ | ||
- Realization of model allocation | ||
- Models (version of BMI-compatible model) | ||
- Model configurations | ||
|
||
Each Instance of the combination of these things is hereafter called a RUN and the grouping of all the aforementioned items necessitates a standardized RUN CONFIG. | ||
|
||
A preliminary Specification for NextGen RUN CONFIGs has been proposed. |