Caching work (data loads)

[AJAnderson]

I'm working to make a small plotting front end to a data loading library I'm working on.

The library itself doesn't cache data it's asked to load. So I was wondering what the best way to cache data would be for egui, would it be to make a field in the main app struct to take the data? If so, I'm not sure how I would make that generic so that it could accept any datatype that gets loaded, any suggestions?

[emilk]

"How to cache things" is a pretty general question; it depends a lot on your requirements. In particular, cache eviction is a complex subject.

For something simple, try https://crates.io/crates/memoize ?

[AJAnderson]

Thanks, I know it was a bit vague. Along the way of tinkering I thought what if we cached the line that's being drawn? But as far as I can see egui::plot::Plot::new().line(xx) takes ownership of the Line xx. Is this something that it would make sense to cache possibly? Regenerating the line on each loop is pretty expensive for large datasets

[emilk]

Profile and see if it is worth it.

Note that a very dense line will be slow for egui to paint, and since egui is immediate mode it will be repainted everytime you move the mouse. If you have a lot of datapoints you may want to simplify it before handing it to egui (e.g. reduce the density along smooth sections of the line).

[AJAnderson]

I tried caching to avoid the file parsing step in my code (i.e. holding my data vector in the template app after load). It sped it up a little bit but still slow.

I had assumed it was because the construction of an iterator that implements Iterator<Item = Value> and the construction of a line from that iterator was costly. I hadn't realised the paint itself would be slow for dense lines. I will try downsampling the data to make it easier to render.

Thanks for your help!