site torch dataset

[peterdudfield]

Pull Request

Description

First try at adding "Site" workflow

Things to do

• change to use site_id not system_id
• Add in site dropout

#73

How Has This Been Tested?

• added a new test
• old tests
• test out with wind RUVNL data

from ocf_data_sampler.torch_datasets.site import SitesDataset
site_pipeline = SitesDataset(config_filename='test_config.yaml')
batch = site_pipeline.__getitem__(0)
print(batch)

with the following config

general:
  description: Test config for site PVNet
  name: site_pvnet_test

input_data:

  site:
    filename: "/mnt/storage_ssd_4tb/india_wind_data.nc"
    metadata_filename:  "/mnt/storage_ssd_4tb/india_wind_metadata2.csv"
    time_resolution_minutes: 60
    history_minutes: 60
    forecast_minutes: 120
  nwp:
    ecmwf:
      nwp_provider: ecmwf
      nwp_zarr_path: "/mnt/storage_b/ecmwf_india_ad_crop/*.zarr"
      history_minutes: 60
      forecast_minutes: 120
      time_resolution_minutes: 60
      nwp_channels:
        - t2m # 2-metre temperature
      nwp_image_size_pixels_height: 2
      nwp_image_size_pixels_width: 2
      dropout_timedeltas_minutes: [-180]
      dropout_fraction: 1.0

Checklist:

• My code follows OCF's coding style guidelines
• I have performed a self-review of my own code
• I have made corresponding changes to the documentation
• I have added tests that prove my fix is effective or that my feature works
• I have checked my code and corrected any misspellings

[peterdudfield]

Thanks for you comments @AUdaltsova and @Sukh-P

[Sukh-P]

Brilliant work adding this in!

[AUdaltsova]

Thanks a lot for doing this! Looks so good!