Resampling over large gaps in data

[TorSy]

Question:
I have a large dataset (1000 GB + ) worth of JSON data that is IoT sensor data. In order to regularize time-series to a set sampling frequency I use tempo.
One issue I experience is that the sensor data can have gaps in it. Sometimes months worth of missing data. The interpolate function fills all this in, which is really not what I want, as this interpolated data is not based on anything substantial. It should be left as Null or NA.

Could a parameter "interpolation limit" or similar be implemented? Currently I have a very computationally expensive workaround to this problem.

[tnixon]

Hi @TorSy - thanks for your feedback. We plan to separate out the interpolation and resampling functions in a future update so that you should be able to upsample and just fill with "default" values (nulls, zeros or empty strings). The interpolation function would be separate and could be chained together with resampling if that's the desired effect, but not necessary if not. Would this solve your issue?

[epa095]

@TorSy Sounds like #147 no?

[tnixon]

Agreed @epa095 - I think this is a duplicate of #147.