Interpolation creates null values in resampled dataframe

[lambertsbennett]

I am having an issue with resampling/interpolation that I think must be a simple misunderstanding. I have a signal at 10 Hz and I want to upsample to 100 Hz and fill values with a linear interpolation. My current code is as follows:

interp_df = start_tsdf.resample(freq=f"{1/100} sec", func='mean').interpolate(target_cols=['value'], method='linear', show_interpolated=True)

However this results in a value column that contains only nulls in between actual sensor readings. Is it not possible to upsample a signal and linearly interpolate to fill missing values?

[tnixon]

You should be able to do this @lambertsbennett, if not this is definitely a bug. Could you provide some sample data we could test with?

[lambertsbennett]

@tnixon thanks for the response, I will take a small sample of the data when I'm next at work and provide it as soon as possible!

[lambertsbennett]

sample.csv
@tnixon - Hi here are some sample data that I can use to reproduce the error. In this case I am using the signal and the file_name columns as partition columns. What I am thinking could be the problem is that many of the values are identical and it could be that this is a strange case?

[tnixon]

Thanks @lambertsbennett - I will investigate ASAP!

[lambertsbennett]

@tnixon I have also been looking more into this and things are even a bit stranger... with a different signal in the data there is a large time gap and this large time gap is interpolated properly, but the smaller gaps are filled with nulls.

[lambertsbennett]

Hi @tnixon, I was just wondering if there are any updates on this issue?

Thanks!

[lambertsbennett]

@tnixon we eventually completely switched up our interpolation process, but it would be nice in the future to use tempo. Did anything ever come out of this?

[tnixon]

Hi @lambertsbennett, thanks for providing the sample data. I'm sorry we haven't been able to follow up on this yet, but it is still in the queue and valuable to us to figure out what is going on here. I am setting aside some time to look at it in the near term

[jplavins]

Hello! I have the same problem where I end up with null values as a result of resampling/interpolation.
I've attached the sample data. Some combination of parameters work fine, some return null values (func='ceil' and method='linear', freq = 60seconds).
data_sample.csv

[tnixon]

Thanks for letting us know @jplavins - we are currently all focused on a major update to the Tempo API, v0.2 which should be released in the new year. We will put in test code to make sure this issue does not come up in the new version.