Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support a --refresh and --step parameters #8

Open
stephen-soltesz opened this issue Nov 15, 2017 · 0 comments
Open

Support a --refresh and --step parameters #8

stephen-soltesz opened this issue Nov 15, 2017 · 0 comments

Comments

@stephen-soltesz
Copy link
Contributor

stephen-soltesz commented Nov 15, 2017

Queries can be expensive. It may be helpful in some cases to run a query that adds a timestamp column. The interval of timestamps would be smaller than the --refresh interval.

For example:

timestamp, site, value
2017-11-10 10:00:00, foo01, 100
2017-11-10 11:00:00, foo01, 100

The query would run every --refresh interval, (say 24hrs), and bqe would iterate through the timestamps at --step intervals of an hour. Arguably the step could be determined automatically...

This feature could help work around behavior we've observed from bqe with small refresh intervals where bigquery apparently stops returning updates around midnight (though that symptom could have other causes, such as time-to-availability of bq streaming inserts).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant