Accessing Particle Tracking DFS3-files

[MartinKyndeMST]

Hi,

When attempting to access the DFS3-files from particle tracking in groundwater simulations ( "projectname_PT_3DSZ.dfs3" in https://doc.mikepoweredbydhi.help/webhelp/2023/MIKESHE/index.htm --> Particle Tracking (PT) --> Output from the PT simulation), I get an error ( with mikeio.read() ):

pandas._libs.tslibs.np_datetime.OutOfBoundsDatetime: Cannot generate range with start=939600000000000000 and periods=11

Do you know of any way to work around this? Or what might cause this?

Best,
Martin

[ecomodeller]

The error indicates that the datetime is outside the range supported by Pandas, but I wonder why. Is this a reproducible based on installation examples, or could you attach a small file that has this problem?

[MartinKyndeMST]

Thank you!
Sadly, the file is too big to send, but the timesteps in the dfs3-file is in 50 year intervals - and the number of seconds in the frequency gets too big in the line 163 of the "_dfs.py"-file: time = pd.date_range(start_time_file, periods=nt, freq=freq). When I in the "_dfs.py"-file reduce the frequency in the function "def _valid_timesteps", I can get the file to be read. So you are definently right, that the problem i with pandas ability to create the datetime range.

When using time=0 in the read, i get get output:
<mikeio.Dataset>
dims: (z:10, y:676, x:805)
time: 1999-10-11 00:00:00 (time-invariant)
geometry: Grid3D(nz=10, ny=676, nx=805)
items:
0: Number of particles (Integer)
1: Accumulated particle count (Integer)
2: Number of registered particles (Integer)
3: Most recent registration zone code (Integer)
4: Average age (year)
5: Average transport time to nearest registration cell (year)

Do you have any idea, to work around this, when even the first timestep is too big?
Thank you!

[ecomodeller]

I think there is a way forward.

Pandas uses nanosecond time resolution by default, which is overkill for your use-case.
Since Pandas 2.0, it is possible to change the time resolution (unit) to something more appropriate.
https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.date_range.html

Note the difference in the dtype ([ns] vs [s]):

>>> import pandas as pd
>>> pd.date_range("2000", freq='10s', periods=2)
DatetimeIndex(['2000-01-01 00:00:00', '2000-01-01 00:00:10'], dtype='datetime64[ns]', freq='10s')
>>> pd.date_range("2000", freq='10s', periods=2, unit='s')
DatetimeIndex(['2000-01-01 00:00:00', '2000-01-01 00:00:10'], dtype='datetime64[s]', freq='10s')

See related issue #633