Skip to content
This repository has been archived by the owner on Sep 11, 2023. It is now read-only.

Allow different size regions of interest for history and forecast #610

Open
JackKelly opened this issue Feb 16, 2022 · 3 comments
Open

Allow different size regions of interest for history and forecast #610

JackKelly opened this issue Feb 16, 2022 · 3 comments
Labels
enhancement New feature or request

Comments

@JackKelly
Copy link
Member

Detailed Description

Implement a config option to specify different sized regions of interest for the history versus the forecast.

Context

The MetNet papers used much larger regions of interest for the history than for the forecast. We should probably do the same 🙂

@JackKelly JackKelly added the enhancement New feature or request label Feb 16, 2022
@JackKelly JackKelly moved this to Todo in Nowcasting Feb 16, 2022
@JackKelly JackKelly added this to the v17 dataset milestone Feb 18, 2022
@peterdudfield
Copy link
Contributor

This is a good point, obvious to see in the UCL ML competition, where a bigger input image would have made better results

Perhaps in Satellite we switch to two datavars

  • data_historic, save images of size X
  • data_future, saves images of size Y

@JackKelly
Copy link
Member Author

I'm by no means certain but, in order for the XArray coords to work correctly, I think we might have to save two separate sets of NetCDF files: one set for historic data, and a different set for future data. Which is maybe getting complicated! The only way I can think of putting two different size images into a single NetCDF file would be to have two sets of coords, too (e.g. a x_osgb_future and an x_osgb_historic) which would break all our existing code that reads from the x_osgb coords.

Maybe, for the time being, we can keep things simple and use the large size for both historic and future data in a single NetCDF file and see how insane the file sizes get 🙂

@peterdudfield
Copy link
Contributor

Yea, for the moment let's do a single set.

Probably is the right way to do it, split it up with two different netcdf files. This also reduces the chance of 'data leakage', i.e using the future to help the model

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
enhancement New feature or request
Projects
No open projects
Status: Todo
Development

No branches or pull requests

2 participants