Update to new nwp-consumer

[peterdudfield]

Detailed Description

Update to new NWP-consumer, after major refactor

Context

• some new env vars? - Done ✅ in secrets, will need updating ❌ for non-secrets in terraform
• might need to update the forecast apps too ❌

Possible Implementation

• India dev ECMWF
• India dev GFS
• India dev MO
• India pro ECMWF
• India pro GFS
• India pro MO
• Uk dev UKV
• Uk dev ECMWF
• Uk pro ECMWF
• Uk pro UKV

[peterdudfield]

Started doing this in #715

[peterdudfield]

This is for version 1.0.5.

[peterdudfield]

Version 1.0.5 is 331 MB (4556 files), where as 0.5.33 is 34.3MB (182 files)

[devsjc]

This is because it pulls in all the ECMWF live data, which covers both the UK and India. Crops are taken when the data is loaded anyway by the forecasters so it shouldn't be a problem downstream.

[peterdudfield]

This is because it pulls in all the ECMWF live data, which covers both the UK and India. Crops are taken when the data is loaded anyway by the forecasters so it shouldn't be a problem downstream.

ok, I could clip to UK, before it does some regridding to help

[peterdudfield]

NWP Consumer 1.0.7 works for PVnet 2.4.18 and for PVNet DA 2.4.18 for ECMWF

EDIT: not yet!

[peterdudfield]

Made a bug report here - joblib/joblib#1637, just to see if anyone can help

[peterdudfield]

PVnet and PNVnet DA works for 2.4.19

[peterdudfield]

PVnet on dev, makes the foreacst look weird. need to investigate

[peterdudfield]

tested ECMWF India and saved to s3://india-nwp-development/ecmwf/data//2025010706.zarr/