You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently the /pipeline endpoint only accepts one parameter; year. This means statistics are recalculated for the whole year every time the pipeline is run. This is not necessary since the statistics are calculated on a daily basis, thus a change in a past day doesn't change it's corresponding histogram rows unless the underlying data changed -- and we want to manage such special cases differently anyway. In general, we can just upload the past 1-2 days every day without removing all the previous days' rows and calculating them again.
To do so, the /pipeline endpoint should support the from and to parameters.
The export phase won't need to be changed as a single histogram_daily_stats.json file includes data for a whole year and we can't generate one day at a time.
The text was updated successfully, but these errors were encountered:
Currently the
/pipeline
endpoint only accepts one parameter;year
. This means statistics are recalculated for the whole year every time the pipeline is run. This is not necessary since the statistics are calculated on a daily basis, thus a change in a past day doesn't change it's corresponding histogram rows unless the underlying data changed -- and we want to manage such special cases differently anyway. In general, we can just upload the past 1-2 days every day without removing all the previous days' rows and calculating them again.To do so, the
/pipeline
endpoint should support thefrom
andto
parameters.The export phase won't need to be changed as a single histogram_daily_stats.json file includes data for a whole year and we can't generate one day at a time.
The text was updated successfully, but these errors were encountered: