Polar masks output do not follow path when using SICDownloader

[bnubald]

• IceNet version: v0.2.7
• Python version: 3.11.7
• Operating System: Linux x64

Description

When using non-default output path for SIC data download, the masks generation does not pass the path argument to the Masks class for the masks.params output file containing the polar hole mask radii and dates. So, it creates the file under the default ./data/masks/south/masks/masks.params instead of the expected ./notebook/data/masks/south/masks/masks.params.

What I Did

sic = SICDownloader(
    dates=[
        pd.to_datetime(date).date()
        for date in pd.date_range("2020-01-01", "2020-04-30", freq="D")
    ],
    path=os.path.join(".", "notebook", "data"), # <-- Relevant non-default value.
    delete_tempfiles=True,
    north=False,
    south=True,
    parallel_opens=True,
)

Related error when running above command follows.

It traces along the following path (generation of mask path via self.base_path, which is defined by self._path from DataProducer class which is set by path argument):

• icenet/icenet/data/sic/mask.py

  Lines 213 to 221 in cb1cb78

  	mask_path = os.path.join(
  	self.get_data_var_folder("masks"),
  	"active_grid_cell_mask_{:02d}.npy".format(month))
  	if not os.path.exists(mask_path):
  	raise RuntimeError("Active cell masks have not been generated, "
  	"this is not done automatically so you might "
  	"want to address this!")

• icenet/icenet/data/producers.py

  Lines 55 to 58 in cb1cb78

  	@property
  	def base_path(self) -> str:
  	"""The base path of the data collection."""
  	return self._path

• icenet/icenet/data/producers.py

  Line 48 in cb1cb78

  self._path: str = os.path.join(path, identifier)

• icenet/icenet/data/producers.py

  Line 30 in cb1cb78

  path: str = os.path.join(".", "data"),

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
Cell In[8], line 1
----> 1 sic = SICDownloader(
      2     dates=[
      3         pd.to_datetime(date).date()             # Dates to download the variable data for
      4         for date in pd.date_range("2020-01-01", "2020-04-30", freq="D")
      5     ],
      6     path=os.path.join(".", "notebook", "data"), # Location to download data to (default is `./data`)
      7     delete_tempfiles=True,                      # Whether to delete temporary downloaded files
      8     north=False,                                # Boolean: Whether to use mask for this region
      9     south=True,                                 # Boolean: Whether to use mask for this region
     10     parallel_opens=True,                        # Boolean: Whether to use `dask.delayed` to open and preprocess multiple files in parallel
     11 )

File /data/hpcdata/users/username/miniconda3/envs/icenet-edsbook/lib/python3.11/site-packages/icenet/data/sic/osisaf.py:380, in SICDownloader.__init__(self, additional_invalid_dates, chunk_size, dates, delete_tempfiles, download, dtype, parallel_opens, *args, **kwargs)
    377 self._ftp_osi430b = "/reprocessed/ice/conc-cont-reproc/v2p0/{:04d}/{:02d}/"
    378 self._ftp_osi430a = "/reprocessed/ice/conc-cont-reproc/v3p0/{:04d}/{:02d}/"
--> 380 self._mask_dict = {
    381     month: self._masks.get_active_cell_mask(month)
    382     for month in np.arange(1, 12 + 1)
    383 }
    385 # Load dates that previously had a file size of zero.
    386 # To recheck they haven't been fixed since last download.
    387 zero_dates_path = os.path.join(self.get_data_var_folder("siconca"),
    388                                "zero_size_days.csv")

File /data/hpcdata/users/username/miniconda3/envs/icenet-edsbook/lib/python3.11/site-packages/icenet/data/sic/osisaf.py:381, in <dictcomp>(.0)
    377 self._ftp_osi430b = "/reprocessed/ice/conc-cont-reproc/v2p0/{:04d}/{:02d}/"
    378 self._ftp_osi430a = "/reprocessed/ice/conc-cont-reproc/v3p0/{:04d}/{:02d}/"
    380 self._mask_dict = {
--> 381     month: self._masks.get_active_cell_mask(month)
    382     for month in np.arange(1, 12 + 1)
    383 }
    385 # Load dates that previously had a file size of zero.
    386 # To recheck they haven't been fixed since last download.
    387 zero_dates_path = os.path.join(self.get_data_var_folder("siconca"),
    388                                "zero_size_days.csv")

File /data/hpcdata/users/username/miniconda3/envs/icenet-edsbook/lib/python3.11/site-packages/icenet/data/sic/mask.py:218, in Masks.get_active_cell_mask(self, month)
    213 mask_path = os.path.join(
    214     self.get_data_var_folder("masks"),
    215     "active_grid_cell_mask_{:02d}.npy".format(month))
    217 if not os.path.exists(mask_path):
--> 218     raise RuntimeError("Active cell masks have not been generated, "
    219                        "this is not done automatically so you might "
    220                        "want to address this!")
    222 # logging.debug("Loading active cell mask {}".format(mask_path))
    223 return np.load(mask_path)[self._region]

RuntimeError: Active cell masks have not been generated, this is not done automatically so you might want to address this!