You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
With #3827 we introduced a sync mechanism that stores the full grapher configs in R2 so that we can fetch them via a simple API instead of extracting this information from HTML like we did before.
These R2 files are updated whenever a chart is saved via the admin rest API. This covers all cases of normal config write scenarios (e.g. inside the admin, the ETL updates, ...) but it leaves out one point of modification: direct database changes, especially migrations.
We don't change chart configs in our db via migrations all the time but in the average year we probably do this 2-4 times. We could either:
manually call the syncGraphersToR2 script
try to run in after migrations run in the correct context
just run it periodically (e.g. every 10 min).
A single invocation if all charts are up to date only takes ~10 seconds so running the sync script is not particularly expensive but we should probably ensure that only of these processes runs at any given time (probably best via buildkite)
The text was updated successfully, but these errors were encountered:
danyx23
changed the title
Periodically run the syncGraphersToR2 devtool to avoid drifting after migrations
Periodically run the syncGraphersToR2 devtool to avoid R2 configs drifting via DB ones after migrations
Oct 7, 2024
With #3827 we introduced a sync mechanism that stores the full grapher configs in R2 so that we can fetch them via a simple API instead of extracting this information from HTML like we did before.
These R2 files are updated whenever a chart is saved via the admin rest API. This covers all cases of normal config write scenarios (e.g. inside the admin, the ETL updates, ...) but it leaves out one point of modification: direct database changes, especially migrations.
We don't change chart configs in our db via migrations all the time but in the average year we probably do this 2-4 times. We could either:
A single invocation if all charts are up to date only takes ~10 seconds so running the sync script is not particularly expensive but we should probably ensure that only of these processes runs at any given time (probably best via buildkite)
The text was updated successfully, but these errors were encountered: