long-term goal: integrate with PeCAn tools for gap-filling and standardization.

[teixeirak]

Info from Mike Dietze, Sept. 2020:

All the met tools are within pecan/modules/data.atmosphere are here.

The default gapfilling, metgapfill.R, was written by Ankur Desai and leverages the Ameriflux MDS approach with a few extra special cases. It is generally good for filling small gaps (e.g. QC flagged data), but not for filling large gaps (e.g. months) when systems are down.

There’s an even simpler version based on splines and linear models -- again it’s really only good for small gaps

For larger gaps I’d use the tdm_* scripts, which is something that Christy Rollinson built with help from one of Ankur’s grad students. This code is really for downscaling, not gapfilling, but for large gaps what I’d do is to use the downscaling code to downscale a spatially-coarser reanalysis product (I’m particularly fond of ERA5, and there’s code for downloading that in the data.atmosphere module). Christy’s code needs a training data set at both scales (local and coarse) and build a complex series of GAMs across variables and across a moving average through day-of-year. It can also produce ensembles of outputs to capture the uncertainty associated with downscaling/gapfilling. It’s more computationally demanding, but more sophisticated than anything else I’ve got. Once you do the downscaling it should be pretty easy to just substitute these values for any NAs in your original time-series.

---

From Christy, Sept. 2020:

Like Mike says, the TDM scripts weren’t built for gap filling, but what Mike suggests with downscaling a coarser product and inserting the values into your gaps should work. However, right now this could be a bit buggy since the code was developed with producing ensembles that propagate uncertainty in mind. In the work for a different project (MANDIFORE), I discovered that the single-use instance where you don’t produce ensembles appears to be buggy and needs more than the quick hack I had put in there.

One key thing that might be important depending on what kind of weather station variables you need is that I spent a lot of time playing with the equations in my scripts to at least try to preserve met variable covariance. I suspect this is more focused on variables/resolutions you may not care about like sub daily long/shortwave radiation, temperature, and wind, but if that is important, my workflow might be worth the trouble.