-
Notifications
You must be signed in to change notification settings - Fork 23
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
5 changed files
with
144 additions
and
2 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,97 @@ | ||
| --- | ||
| title: "Accessing National Water Model (NWM) Data via Google Cloud BigQuery API" | ||
| description: "September Monthly Blog Update" | ||
| slug: September Monthly Blog Update | ||
|
|
||
| tags: [BigQuery, BYU, NWM, CIROH, Google, Cloud, API] | ||
| hide_table_of_contents: false | ||
| --- | ||
|
|
||
| <img src="/img/gcp_architecture_diagram.png" alt="gcp architectrure diagram"/> | ||
| <i >Image Source: <a href="https://github.com/BYU-Hydroinformatics/api-nwm-gcp">https://github.com/BYU-Hydroinformatics/api-nwm-gcp</a> </i> | ||
|
|
||
| <br /> | ||
| <br /> | ||
|
|
||
| # Accessing National Water Model (NWM) Data via Google Cloud BigQuery API | ||
| Several important historical and ongoing National Water Model (NWM) datasets are now available on Google Cloud BigQuery, which makes them queryable through SQL using Google Cloud console. Some of these data sets are also accessible through an API (e.g. using Python). These datasets and their current status are as follows: | ||
|
|
||
| <table style={{ width: '100%', display: 'inline-table' }}> | ||
| <tr> | ||
| <td>Product</td> | ||
| <td>Cloud Console SQL</td> | ||
| <td>CIROH API</td> | ||
| <td>Historical</td> | ||
| <td>Daily Updates</td> | ||
| </tr> | ||
| <tr> | ||
| <td>Medium-range forecasts</td> | ||
| <td>X</td> | ||
| <td>X</td> | ||
| <td>X</td> | ||
| <td>X</td> | ||
| </tr> | ||
| <tr> | ||
| <td>Long-range forecasts</td> | ||
| <td>X</td> | ||
| <td>X</td> | ||
| <td>X</td> | ||
| <td>X</td> | ||
| </tr> | ||
| <tr> | ||
| <td>Analysis and Assimilation</td> | ||
| <td>X</td> | ||
| <td>X</td> | ||
| <td>X</td> | ||
| <td>X</td> | ||
| </tr> | ||
| <tr> | ||
| <td>Retrospective Data (NWM v3)</td> | ||
| <td>X</td> | ||
| <td></td> | ||
| <td>X</td> | ||
| <td></td> | ||
| </tr> | ||
| <tr> | ||
| <td>Return Periods</td> | ||
| <td>X</td> | ||
| <td></td> | ||
| <td>X</td> | ||
| <td></td> | ||
| </tr> | ||
| </table> | ||
|
|
||
| To gain access and run queries on BigQuery: | ||
|
|
||
| - Submit the NWM BigQuery API Access Form, available at: https://docs.ciroh.org/docs/products/Data%20Management%20and%20Access%20Tools/bigqeury-api/ | ||
| - Once approved, the CIROH IT team will cover the costs associated with your queries. | ||
| - Initially, each organization or project may request a usage limit of up to $500 per month for accessing the data through BigQuery. | ||
|
|
||
|
|
||
| This service offers a convenient and efficient way to interact with NWM data for your research and analysis needs. | ||
|
|
||
|
|
||
|
|
||
| ## Public-Private Partnership: Advancing Water Resource Management | ||
|
|
||
|
|
||
|
|
||
| The National Water Model (NWM) BigQuery project exemplifies a successful collaboration between public and private sectors, uniting government-generated data with modern, cutting-edge cloud technology. This collaboration addresses several key aspects: | ||
|
|
||
|
|
||
|
|
||
| - **Improved Data Access**: By leveraging Google Cloud BigQuery, a private sector platform, the project dramatically improves access to public NWM data. This partnership makes valuable water resource information more readily available to researchers, policymakers, and the public. | ||
| - **Technological Innovation**: The integration of NWM data with BigQuery showcases how private sector technology can enhance the utility of public sector data. This synergy promotes innovation in data analysis and visualization techniques. | ||
| - **Cost-Effective Solutions**: The CIROH DevOps team's commitment to covering query costs demonstrates how public funding can be strategically used to make private sector tools accessible to a wider audience, particularly in the academic and research communities. | ||
| - **Capacity Building**: This initiative helps build capacity across sectors by providing researchers and organizations with powerful tools to analyze water resource data, potentially leading to better-informed decision-making in water management. | ||
| - **Scalability and Efficiency**: By utilizing Google's cloud infrastructure, the project ensures that the growing volume of NWM data can be efficiently stored, accessed, and analyzed, addressing the scalability challenges often faced by public sector IT resources. | ||
| - **Cross-Sector Collaboration**: This project fosters collaboration between government agencies, academic institutions, and private technology companies, creating a model for future partnerships in environmental and resource management. | ||
| - **Open Science Promotion**: By making NWM data more accessible, this partnership supports the principles of open science, encouraging transparency and reproducibility in water resource research. | ||
|
|
||
|
|
||
| This public-private partnership not only enhances the value of the National Water Model but also sets a precedent for future collaborations that can drive innovation in environmental data management and analysis. | ||
|
|
||
|
|
||
|
|
||
|
|
||
|
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,6 +1,6 @@ | ||
|
|
||
| import News from '../components/news-js'; | ||
|
|
||
| # Community NextGen and NGIAB Updates | ||
| # Community NextGen Updates | ||
|
|
||
| <News/> |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters