Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

All the fixes #339

Merged
merged 6 commits into from
May 22, 2024
Merged

All the fixes #339

merged 6 commits into from
May 22, 2024

Conversation

alexsnaps
Copy link
Member

@alexsnaps alexsnaps commented May 21, 2024

If you test this in INFO level log as such:

$ ./target/debug/limitador-server limitador-server/examples/limits.yaml -vv --grpc-reflection-service redis_cached  redis://localhost:6379

And then start the driver, you'd get a first log for the priority flush:

INFO flush_batcher_and_update_counters: limitador::storage::redis::counters_cache: Priority flush!

Then further for the per second flushes:

2024-05-21T15:43:58.456434Z  INFO flush_batcher_and_update_counters: limitador::storage::redis::redis_cached: Flushing 1 counter updates
2024-05-21T15:43:59.458124Z  INFO flush_batcher_and_update_counters: limitador::storage::redis::redis_cached: Flushing 1 counter updates

Then shut redis down:

2024-05-21T15:44:03.472454Z ERROR flush_batcher_and_update_counters: limitador::storage::redis::redis_cached: Partition to Redis detected!
2024-05-21T15:44:03.472467Z  WARN flush_batcher_and_update_counters: limitador::storage::redis::redis_cached: Error flushing error while accessing the limits storage: broken pipe

No more logs for the duration of the partition, restart redis, and...

2024-05-21T15:44:08.480444Z  WARN flush_batcher_and_update_counters: limitador::storage::redis::redis_cached: Partition to Redis resolved!
2024-05-21T15:44:09.481024Z  INFO flush_batcher_and_update_counters: limitador::storage::redis::redis_cached: Flushing 1 counter updates

@alexsnaps alexsnaps force-pushed the fix_partion_flush branch from 922bf2a to 45b191e Compare May 21, 2024 16:42
@@ -215,6 +215,14 @@ impl AsyncRedisStorage {
.clone()
.incr::<&str, i32, u64>("LIMITADOR_LIVE_CHECK", 1)
.await
.map_err(|err| {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is_alive(&self) isn't used anymore afaict... but there is the .then wrt liveness_latency... So... what shall we do?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't found a #[allow(dead_code)] that might affect this (not complaining), also, method is_alive is not found in any other part of the project than here (method definition)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@adam-cattermole what you recon we do? 🔥 the metric? Call it from a dedicated Future?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reckon we just burn the metric/method. We can add this back if needed but for now we could just use the datastore_latency from the batcher calls to redis (I know even this metric isn't perfect due to size of batch etc). If we decide we want this down the line it shouldn't be too hard to add it back

Copy link
Member

@didierofrivia didierofrivia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Everything looks good to me, with the exception of cargo not complaining for the unused method is_alive. Maybe @adam-cattermole would have a saying on this?

Copy link
Member

@adam-cattermole adam-cattermole left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@alexsnaps alexsnaps merged commit dfc5093 into main May 22, 2024
18 checks passed
@alexsnaps alexsnaps deleted the fix_partion_flush branch May 22, 2024 12:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

3 participants