Confusion of extracted parameters in the Geosphere Dataset

[mhuber89]

Describe the bug

When extracting hourly 'radiation_global' data from the Geosphere dataset, wind_direction data are extracted. The issue here is a confusion between the extraction parameters set in the file wetterdienst/wetterdienst/provider/geosphere/observation/api.py.
For the hourly data starting from line 121 the RADIATION_GLOBAL value is set to "D6X" (which is the wind direction) and the parameter WIND_DIRECTION is set to "GSX" (which is the global irradiation in J/cm2). these two parameters have to be exchanged :)

To Reproduce
Steps to reproduce the behavior:
With the following code you can access the global irradiation even though the wind direction was requested:

from datetime import datetime
from wetterdienst import Parameter, Resolution, Settings
from wetterdienst.provider.geosphere.observation import GeosphereObservationRequest, GeosphereObservationDataset, GeosphereObservationResolution, GeosphereObservationParameter
settings = Settings( # default
    ts_si_units=True  # convert values to SI units
)

stations_at = GeosphereObservationRequest(
    parameter=[Parameter.WIND_DIRECTION],
    resolution=GeosphereObservationResolution.HOURLY, 
    start_date=datetime(2022, 6, 1),
    end_date=datetime(2022, 6, 2),
    settings=settings
)

station_at = stations_at.filter_by_station_id("4821")

df = station_at.values.all().df.to_pandas()
df 

Expected behavior
When both parameters are exchanged the correct values are extracted

Screenshots

Desktop (please complete the following information):

• OS: Mac-OS
• Python-Version 3.11.4
• wetterdienst 0.59.0

[mhuber89]

In addition there is also a bug in the unit conversion in the 10 minute dataset for the parameter radiation_global in the geosphere dataset. the origin unit is W/m2 not Joule/cm2

[gutzbenj]

Dear @mhuber89 ,
thanks for reporting this!

The parameter confusion is fixed with latest version.

Regarding the unit I think we need a bit of discussion! I think so far I took at as the unit should always refer to the time interval that is requested e.g. if you request daily data the energy shouldn't refer to seconds (which watt / joule per second would do). However I'm now unsure whether this is really smart for the case of watt?

Another example: if you request daily precipitation you'd always expect the amount to be per 86400 seconds rather then the amount of precipitation per 1 second.

Do you have other experiences with this?

[mhuber89]

Hi @gutzbenj ,
thanks a lot again for the quick fix :)

I think having the SI units of wetterdienst referring to the duration of the requested time interval is fine (at least from my side there is nothing against it). On the other hand I think having normalised (to seconds) SI units would be also fine as long as it is done consistently. as far as I can see the unit scheme seems very consistent in the code :)

what I was referring to in the comment above was more that the origin unit of the extracted geosphere parameter radiation_global is W/m2 (for the 10 minute dataset). but in the code the origin value is interpreted as joule / cm2. so I guess just the marked part would need to be exchanged to have the values correct again.

[mhuber89]

Sorry for the many comments, but I have one hopefully last question. I think the units for the radiation values in geosphere that are give in Joule per ... are always given in per cm2 (at least according to your documentation and the geosphere documentation). So in the above file most of the unit Values for radiation related values should be changed to JOULE_PER_SQUARE_CENTIMETER, right?

[gutzbenj]

Dear @mhuber89 ,

had busy days but will look into this the upcoming weekend and respond to you asap.

[gutzbenj]

Sorry, couldn't look into the issue during the weekend but hopefully next weekend!

[mhuber89]

no worries, I have a workaround for now :)

[gutzbenj]

Hey again,

I checked the units and it should be W / m2, see this: https://dataset.api.hub.geosphere.at/v1/station/historical/klima-v1-10min/metadata

So as it stands, our documentation is wrong!

Also after considering everything again I'm a bit frustrated:

It seems there are two ways used to express energy per area:

• At Geosphere they use Watt which includes the time scale (s)
• At DWD they use sums (J / cm2 per ... e.g. day) which implicitly tells the time over the queried data

After thinking over it again it would actually be more logical to use Watts as it includes the time period but the user would obviously have to do the hard work of multiplying it and even there are even some other logical implications like leap years...

I think at the moment easiest would be to keep both units and I'll fix the documentation.

Addition

Turns out Geosphere is using J / cm2 for daily sums as well! https://dataset.api.hub.geosphere.at/v1/station/historical/klima-v1-1d/metadata

[gutzbenj]

Another addon to this: it seems as W / m2 is only used for 10 minute values a unit conversion would be reasonable effort to get all units straight as J / cm2, I'll hand in a PR soon.

[amotl]

Hi again.

Is GH-998 already the intended fix for your issue, @mhuber89? Did you have the chance to validate whether it works well for you? It looks like the improvement has been included into release 0.59.2 already.

With kind regards,
Andreas.