Research Request - switch dask.delayed to dask.from_map

[tiffanychu90]

Complete the below when receiving a research request, and continue to add to this issue as you receive additional details and produce deliverables. Be sure to also add the appropriate project-level label to this issue (eg gtfs-rt, DLA).

Research Question

Single sentence description: The concatenation of segment speeds (lots of segments + geometry) over the last 2 years is taking quite awhile to produce year averages. The concatenation relies on dask.delayed, but the docs indicate there's a dask.from_map syntax that could be more desirable to use.

Detailed description:

• Start with the rt_segment_speeds/scripts/quarter_year_averages.py script and see if these can move to dask.from_map
  • The year averages currently take ~25 min, which feel too long for squashing down a gdf. segment_geometry is present and merged in every single date we have, but this is not necessarily desirable...we should dedupe more efficiently.
• Take a look at time_series_utils to see if we can switch out the concatenation step and generalize a bit more to take any processed dataframe going into GTFS digest

Data sources

• Cal-ITP data sources: GTFS analytics pipeline...start with outputs that go into GTFS digest / longer term averages

Deliverables

Utility functions + updated scripts